branimir.com
RSS 1.0/RDF/XML
| Sunday, September 17, 2006 |
|---|
ShiftN: Automatic perspective correction [www.shiftn.de]
This is a program that makes it incredibly easy to correct convergent lines. Thanks Carsten!|
Posted by Branimir Dolicki at 16:31 |
|
# - G - Add comment |
| Tuesday, September 5, 2006 |
|---|
TIP: Cool incremental search in Apple's Address Book
Try searching for a phone number in your Address Book (on Tiger), but type the number backwards. Note the following:- The search works, even though you are typing backwards.
- Because you don't have to start with the country and area codes the search converges much more quickly. This is especially useful if some of the numbers are stored with country code, some only with area code etc.
- The search will find numbers even if they are stored with hyphens, brackets, spaces etc. This is not the case with normal, "forward" search.
|
Posted by Branimir Dolicki at 18:22 |
|
# - G - Add comment |
| Thursday, December 1, 2005 |
|---|
Learning about the World via satellites [bfi.org]
Isn't it amazing how much we can learn about the World from a single photo? A larger version is here. More photos here.
|
Posted by Branimir Dolicki at 23:39 |
|
# - G - Add comment |
| Wednesday, November 23, 2005 |
|---|
Planet Sony [www.doxpara.com]
Check out this picture.No, the map doesn't show known cases of the bird flu. It is about another plague.
Let's recap (full article is here):
You buy a CD. You put the CD into your PC in order to enjoy your music. Sony grabs this opportunity to sneak into your house like a virus and set up camp, and it leaves the backdoor open so that Sony or any other enterprising intruder can follow and have the run of the place. If you try to kick Sony out, it trashes the place. And what does this software do once it's on your PC? ... The DRM itself is almost unbelievably restrictive, and some have suggested that the reasoning behind it is part of Sony's ongoing war over digital music supremacy with the decidedly more supreme Apple.Now, about that picture: When Sony Rootkit spies on you it needs to query a name server (DNS) to convert a domain name to IP address. Due to a loophole in the DNS system it is possible to get lists of cached IP addresses from outside. IP addresses can be mapped to geographic locations. This way it is possible to show location of your house on the map.
If you are still considering to put a Sony product under Christmas tree be sure to read the End User Licence Agreement (EULA) Sony's customers have to agree to before listening to their music.
|
Posted by Branimir Dolicki at 09:22 |
|
# - G - Add comment |
| Tuesday, November 22, 2005 |
|---|
Please don't upgrade me! [www.jnd.org]
I don't think I want to be upgraded to Lufthansa Business Class. Read the horrifying account by Don Norman on complete failure of Lufthansa in human-computer interaction. I just hope that the same people don't get to design the cockpit!
|
Posted by Branimir Dolicki at 20:04 |
|
# - G - Add comment |
| Wednesday, August 31, 2005 |
|---|
It is not over when the boss says it is over [www.pacifict.com]
This amazing story about the guy who kept sneaking in at Apple after his project was cancelled made my day. (via http://www.37signals.com/svn/)
|
Posted by Branimir Dolicki at 10:16 |
|
# - G - Add comment |
| Thursday, May 5, 2005 |
|---|
Getting your Sony Ericsson K700i to use your IMAP4 server with self-signed SSL certificate
A nice feature of Sony Ericsson K700i is a built-in IMAP4 support with encryption. So, today I decided to get it to work with my IMAP server (dovecot). I'd better write down all the steps before I forget. Thanks Carsten for help with the CA stuff!I did this on Red Hat Enterprise Linux AS release 4 with dovecot-0.99.11-2.EL4.1 and openssl-0.9.7a-43.1.
- Become a Certification Authority by following steps in http://mia.ece.uic.edu/~papers/volans/settingupCA.html. The key step is generating a root certificate:
openssl req -x509 -newkey rsa -out cacert.pem -outform PEM -days 10000
It will generate a public/private key pair. The private key will stay on your server and will be used only for signing certificates. The public key (a.k.a. the CA certificate) is what you'll need to put on your phone so that it trusts certificates signed by you. Unlike most desktop mail clients K700i won't just warn you, it will refuse to work unless you do this. - Install the CA certificate on your phone. SE K700i seems to require the DER format of the certificate as well as .cer extension. If you use a different filename or format it will simply say "invalid certificate" or "ungültiges zertifikat" and refuse to proceed. Here's how you can convert the certificate to DER format:
openssl x509 -in cacert.pem -inform PEM -out cacert.cer -outform DER
Next step is to upload the certificate to your phone via Bluetooth (Connectivity -> Bluetooth -> My devices -> your-computer-name -> navigate to cacert.cer. The phone will give you a happy message that it accepted the certificate. If you go to Connectivity -> Internet settings -> Security -> Trusted cert., your certificate should be there among others. From now on your phone should accept certificates signed by you.
- Generate a certificate request as described in http://mia.ece.uic.edu/~papers/volans/settingupCA.html and issue the certificate as described on the same page. Suppose you call your certificate and private key wfcert.pem and wfkey.pem respectively.
- Edit /etc/dovecot.conf, put something like this in it:
... and restart your dovecot IMAP4 server.ssl_cert_file = /etc/ssl/certs/wfcert.pem ssl_key_file = /etc/ssl/private/wfkey.pem
- Configure the email account on your phone. Here are some important things that can go wrong
- When setting Encryption to TLS/SSL be sure to also set the Domain to whatever domain your certificate is issued for. If there is a mismatch here., the phone will give you a helpful message "TLS/SSL handshake failure".
- Mailbox should be set, believe it or not, to your login name.
|
Posted by Branimir Dolicki at 16:50 |
|
# - G - 2 comments - Add comment |
| Sunday, April 10, 2005 |
|---|
Universal Access o All Knowledge [www.itconversations.com]
Today I listened to one of the best programs so far from the excellent IT Conversations: Brewster Kahle, the author of Internet Archive speaks on his goal of archiving and making universaly available all content mankind has ever produced, including music and moving pictures. Absolutely fascinating! Oh yes, while Brewster Kahle speaks about ripping and storing all human knowledge, Clay Shirky speaks of categorizing it (or rather, false promises of doing it). Also a highly recommended podcast.|
Posted by Branimir Dolicki at 00:02 |
|
# - G - Add comment |
| Saturday, March 12, 2005 |
|---|
Building of Oktoberfest [www.municam.de]
These photos were taken from the same spot every day at the same time. It is interesting to watch how tents get built, the show starts and finishes and then tents get teared down.
|
Posted by Branimir Dolicki at 02:33 |
|
# - G - Add comment |
| Thursday, February 24, 2005 |
|---|
Ajax [adaptivepath.com]
The three most important things I care about when it comes to web pages (after their usefulness or whatever other reason I am using it in the first place) are speed, speed and speed. I've been looking at this really, really interesting new approach to web applications used by Google Suggests, Google Maps and Flickr and I think every web developer should take a hard look at it. It really promises to close the gap between web and desktop apps. It is interesting how playing with that stuff (which adaptivepath.com calls Ajax) changed my perspective on how web apps should behave: When I posted the link to it at del.icio.us I wasn't sure which tags to assign to it. A nice feature of del.icio.us is the ability to see which tags other people used for that same URL. Wouldn't it be nice if del.icio.us suggested tags as I type (based on other people's tags) similar to Google Suggests?|
Posted by Branimir Dolicki at 13:54 |
|
# - G - Add comment |
| Wednesday, February 9, 2005 |
|---|
Getting Hooked on Podcasting
Some months ago Lars mentioned Podcasting to me. The idea seemed appealing but as I never listened internet radio much it didn't seem like something I'd use. Later, Carsten drew my attention IT Conversations - a site where one can download recorded interviews and conference talks. My first reaction was that I would probably never use it as I prefer reading to listening.And then I got myself an iPod. I'm not sure why I bought it. I am a gadget geek, that's for sure. But I never liked Walkmans and things like that. I always hated getting entangled in wires while trying to punch my U-Bahn ticket or not hearing what's going on around me. I guess being a gadget geek and a design freak at the same time was enough to shell out over 419,90 € for the 40 GB version (it must be a 40 GB one, I don't want to have to make choices on what I take with me and what not).
At first, I almost felt guilty for spending that much money for a toy I'm not even sure I'd use. I already mentioned I don't like wires while commuting and the problem of listening to digital music at my home has already been solved.
And then I actually tried listening those recordings from IT Conversations on my iPod. A whole new world opened up in front of my, er... ears! The thesis #3 from Cluetrain Manifesto comes to mind: "Conversations among human beings sound human. They are conducted in a human voice.
Here are some of my favorite recordings:
- Steve Wozniak at Gnomedex 4.0 (part two) tells a fascinating story starting when he was a boy building crystal-set radios and playing pranks all the way to the making of Apple I and Apple II. You have to hear this. It is brilliant!
- Clayton Christensen - if you haven't read Christensen's book the Innovator's Dilemma you will probably order it after you hear this talk.
A totally cool feature of the IT Conversations site is personalized RSS feed: while surfing the site you can add items to your listening queue. Those items then get downloaded overnight by podcasting software such as iPodderX.
|
Posted by Branimir Dolicki at 01:43 |
|
# - G - Add comment |
| Sunday, February 6, 2005 |
|---|
My Piano just crashed!
I've been a proud owner of the Clavinova CLP-170 Digital Piano from Yamaha for over a year. I've been very happy with the instrument: the sound is extremely realistic, in fact I think it is much better than a low-end acoustic piano I would otherwise be able to afford. Now when I'm tired of dealing with computers and bugs and code I can sit and play some jazz. One of the nice things about this piano is that it has the middle sostenuto pedal which you usually find only in real Steinway-ish concert pianos. So I was experimenting with the sostenuto pedal in that I played a chord, and then improvised some music while keeping the pedal depressed. Like on a real piano, the Clavinova simulated the effect of dampers belonging to this particular chord's strings kept raised. So if I played the chromatic scale while keeping the pedal depressed I could hear how the notes from my chord were sustained while other notes were not. So, I was playing and playing, happy to be far from software and bugs until... my Piano hung!. It stop playing completely and the display showed,oh my God,
Isn't that great! I even know in which file the error occured so I can fix it. Thank you very much for helping me forget programming and bugs.
UnexpectedError
utg_asgn.c 00301
|
Posted by Branimir Dolicki at 18:24 |
|
# - G - 1 comment - Add comment |
| Wednesday, February 2, 2005 |
|---|
Hacking my cell phone [www.forum.nokia.com]
As soon as I learned that Nokia is shipping a Python interpreter for series 60 phones I could hardly wait to try it out. And you know what? It WORKS!I went to the above site and downloaded the interpreter. I dragged it onto my Bluetooth File Exchange and voila, I have a real Python interpreter in my Nokia 6600! Supplied are some neat examples such as a simple RSS reader. I was immediately able to read Slashdot headlines on my phone. The RSS reader, although simple, does have some important features such as downloading feeds in background and caching them on the phone. It's an application that I will actually use!
I could hardly wait to start some hacking myself. One of the interesting cell phone applications are location-based services. So I found this little script that returns basic GSM location info obtained through the gsm_location() API call.
Now, this script returns something like:
MCC: 262 MNC: 1 LAC: 35342 Cell id: 25867Which isn't particularly exciting. So, I surfed around and found this file, which gives some interesting information about all T-Mobile cells in Munich. With this information, when walking around the city I'll be able to see not only the useless cell id but also things like:
80331, Platzl 1, Mikrozelle über dem Eingang vom Hard Rock CafeOr:
80799, Heßstr. 22, 1800 MHz, Diakoniewerk München-Maxvorstadt, RückgebäudeIt is not really that useful, really, but it is fun for start.
So, the first step is to use this little Ruby script to parse the data from above mentioned 26201muenchen.clf. This script produces something like this:
# -*- coding: UTF-8 -*-
class CellInfo:
x={}
x[52926]='80331 An der Hauptfeuerwache 8, Feuerwehr'
x[9107]='80331 Kaufingerstr. 1-5, Kaufhof'
x[49295]='80331 Sonnenstr. 26, 1800 MHz'
[... and so on - 971 entries ...]
If you are a Python hacker please don't laugh at me for making that hash a class variable or for generating Python code with Ruby. I am still a complete idiot when it comes to Python. All I want right now is to get this working as fast as possible.
I saved the above script as cinfo.py and dragged to the phone. Then I changed a few line of the above gsm_location.py:
--- gsm_location.py Sun Jan 9 22:04:54 2005
+++ hacked_gsm_location.py Wed Feb 2 18:50:57 2005
@@ -3,6 +3,7 @@
import e32
import appuifw
import location
+import cinfo
class gsm_location :
def __init__(self) :
@@ -10,7 +11,8 @@
def gsm_location(self) :
(self.mcc, self.mnc, self.lac, self.cellid) = location.gsm_location()
- self.text = u"MCC: %s\nMNC: %s\nLAC: %s\nCell id: %s\n" % (self.mcc, self.mnc, self.lac, self.cellid)
+ self.cell_info = cinfo.CellInfo.x[self.cellid]
+ self.text = u"MCC: %s\nMNC: %s\nLAC: %s\nCell id: %s\nCell info: %s\n" % (self.mcc, self.mnc, self.lac, self.cellid, self.cell_info)
return self.text
def close(self) :
This will simply give me the cell information such as '80331 An der Hauptfeuerwache 8, Feuerwehr' in addition to the existing data.
I tried it and it works here at home! I can't wait to take a walk around Munich!
|
Posted by Branimir Dolicki at 19:28 |
|
# - G - Add comment |
| Thursday, December 9, 2004 |
|---|
POSIX locking
I just realized that the silly locking I described in my previous post on spam filtering doesn't really work because it is being ignored by most programs writing to my mailboxes. Here's the correct version of the rotmail script:
#!/bin/sh
# $Id: rotmail,v 1.3 2004/12/09 14:53:33 bdolicki Exp bdolicki $
# by bdolicki@branimir.com
# I execute this script on a regular basis to:
# 1/ Report manually classified spam to Vipul's razor and feed it into
# Bayesian learner
# 2/ Feed automatically classified spam into Bayesian learner
# 3/ Feed the ham from the inbox into Bayesian learner
# 4/ Archive and truncate the above three folders
# I assume that there are three different kinds of mail (see below):
# 1/ Legit mail (my inbox)
# 2/ Manually classified spam
# 3/ Automatically classified spam
##
# Some variables which can be customized. You definitely *want* to
# take a look a these!
##
# This is a program that does something like:
# cat file1 >> file 2; cat /dev/null > file1
# The reason I'm not doing it that way is that we need to do proper
# locking. In my case that means doing fcntl(2) (POSIX) locking.
ARCHIVING_PROGRAM=$HOME/bin/append-and-truncate.rb
# This is the directory where I keep mails that I actually care for and
# want the IMAP to access. It contains various mailing list folders,
# inbox-archive as well as folder with manually clasified mail so that I
# can do it with my IMAP client.
MAILDIR=$HOME/Mail
# This is the directory which I don't want to access with IMAP
SPAMDIR=$HOME/Spam
# We have six mail folders alltogether: Three folders mentioned above
# and an archive folder for each of those. Following variables control
# names of those folders.
# the inbox (usually in /var/spool/mail)
INBOX=/var/spool/mail/$USER
# manually classified spam: spam which managed to pass through the
# filter (a rare beast these days):
MANUAL_SPAM=$MAILDIR/sp
# automatically classified spam. In my case this contains all mails
# with SpamAssassin score > 5.
AUTOMATIC_SPAM=$SPAMDIR/spam
# Following three folders have suffix -archive. I append the contents
# of the above three folders to them.
INBOX_ARCHIVE=$MAILDIR/inbox-archive
MANUAL_SPAM_ARCHIVE=$SPAMDIR/sp-archive
AUTOMATIC_SPAM_ARCHIVE=$SPAMDIR/spam-archive
# This is where your GNU readlink utility (part of GNU Coreutils) sits:
PATH=$PATH:/opt/gnu/bin
##
# ----- You normally don't need to customize anything below this line -------
##
# Given two files this function will append the contents of the first
# file to the second file and truncate the first file. It uses the
# locking mechanism which is also used by sendmail, imapd ad mutt It
# will not do the truncation if appending failed.
# A small utility function
fileordie() {
if [ ! -e $1 ]
then
echo "$1: file not found. Exiting." 1>&2
exit 1
fi
}
# This is the beginning of the actual program
# If something is wrong with just one file we need -- exit. Just in case.
for FOLDERNAME in $INBOX $INBOX_ARCHIVE $MANUAL_SPAM $MANUAL_SPAM_ARCHIVE \
$AUTOMATIC_SPAM $AUTOMATIC_SPAM_ARCHIVE $ARCHIVING_PROGRAM
do
fileordie $FOLDERNAME
done
# Report manually classified spam. This will also invoke sa-learn
echo "reporting manual spams"
cat $MAILDIR/sp | formail -s spamassassin --report
$ARCHIVING_PROGRAM $MAILDIR/sp $SPAMDIR/sp-archive
echo "learning automatic spams with sa-learn"
sa-learn --spam --mbox --showdots $SPAMDIR/spam || exit 1
$ARCHIVING_PROGRAM $SPAMDIR/spam $SPAMDIR/spam-archive
echo "learning ham in $INBOX with sa-learn"
sa-learn --ham --mbox --showdots $INBOX || exit 1
$ARCHIVING_PROGRAM $INBOX $MAILDIR/inbox-archive
The key part is this little Ruby program: append-and-truncate.rb :
#!/usr/bin/ruby
# $Id: append-and-truncate.rb,v 1.1 2004/12/09 15:36:11 bdolicki Exp bdolicki $
# USAGE: append-and-truncate file1 file2
# An equivalent of:
# cat file1 >> file 2; cat /dev/null > file1
# but doing proper fcntl(2) locking.
#
# This is good because all programs I care about such as procmail, mutt
# and dovecot IMAP server use this kind of locking.
# I'll move this part into a library file eventually.
# fcntl() support is built into the File class. We need this module just
# to translate symbolic constants to their system-dependent numerical
# values:
require 'fcntl'
# Let's enrich the File class with a few more methods :-)
class File
def fcntl_read_lock
do_fcntl_lock Fcntl::F_RDLCK
end
def fcntl_write_lock
do_fcntl_lock Fcntl::F_WRLCK
end
def fcntl_unlock
do_fcntl_lock Fcntl::F_UNLCK
end
def do_fcntl_lock (type)
lock_info = [type, 0, 0, 0, 0].pack("ssqqi")
fcntl Fcntl::F_SETLKW, lock_info
end
end
# The actual program starts here.
if ARGV.length != 2
then
raise "I need exactly two files"
end
inbox = ARGV[0]
outbox = ARGV[1]
puts "Moving the contents of #{inbox} to #{outbox} ..."
File.open(inbox, "r+") do |infile|
File.open(outbox, "a") do |outfile|
outfile.fcntl_write_lock
infile.fcntl_write_lock
while line = infile.gets
outfile.puts line
end
infile.truncate(0)
infile.fcntl_unlock
outfile.fcntl_unlock
end
end
puts "... done"
|
Posted by Branimir Dolicki at 16:38 |
|
# - G - Add comment |
| Thursday, June 17, 2004 |
|---|
Enterprise Photo Gallery (EPG)
After having spent hours looking for decent photo gallery software I decided that it might be a better idea to spend a few minutes writing my own. Here you can see it in action: Zadar Gallery, Munich Gallery. Highlights:- Automatic Thumbnail Generation
- Thumbnail Caching
- HTML templating
- An elegant HTML template that comes with the box.
cd [ns_info pageroot]/[ns_conn url]
set list [glob *.jpg *.JPG *.jpeg *.JPEG \
*.png *.PNG *.gif *.GIF]
multirow create pics name
file mkdir thumbs
foreach pic $list {
if {![file exists thumbs/$pic] || \
[file mtime thumbs/$pic] < [file mtime $pic]} {
exec /usr/bin/convert -interlace NONE \
-geometry 300x300 $pic thumbs/$pic
}
multirow append pics $pic
}
Update (1 September 2004): A friend pointed out that I forgot to publish the HTML template - here it is:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head><title>Photo Gallery</title></head><body>
<multiple name="pics">
<a href="@pics.name@"><img border=0
src="thumbs/@pics.name@" alt="thumbnail"></a>
</multiple>
</body>
|
Posted by Branimir Dolicki at 20:23 |
|
# - G - Add comment |
| Sunday, May 30, 2004 |
|---|
Perspective [branimir.com]
When photographing buildings I try to hold the camera straight to avoid the ugly effect of converging verticals. Unfortunately, this isn't always possible: Either I can't go back far enough or I can't climb high enough to see the entire building without tilting the camera. Photoshop's Perspective option of the Crop tool can fix converging verticals. Here are my first attempts.|
Posted by Branimir Dolicki at 18:22 |
|
# - G - Add comment |
| Sunday, April 25, 2004 |
|---|
Enterprise Spam Filtering
Update: 9 December 2004: The program below is obsolete. Use this one.As I wrote before I don't have problems with spam anymore. I described how I'm using several mail filtering techniques to stop almost all the spam while not having to worry for legitimate mail to get lost.
Here's the last version of the very simple script that I execute once in a while to process both spam and "ham". I tidied it up so people can use it.
#!/bin/sh
# $Id: rotmail,v 1.2 2004/04/25 17:57:26 bdolicki Exp bdolicki $
# by bdolicki@branimir.com
# I execute this script on a regular basis to:
# 1/ Report manually classified spam to Vipul's razor and feed it into
# Bayesian learner
# 2/ Feed automatically classified spam into Bayesian learner
# 3/ Feed the ham from the inbox into Bayesian learner
# 4/ Archive and truncate the above three folders
# I assume that there are three different kinds of mail (see below):
# 1/ Legit mail (my inbox)
# 2/ Manually classified spam
# 3/ Automatically classified spam
##
# Some variables which can be customized
##
# This is the directory where I keep mails that I actually care for and
# want the IMAP to access. It contains various mailing list folders,
# inbox-archive as well as folder with manually clasified mail so that I
# can do it with my IMAP client.
MAILDIR=$HOME/Mail
# This is the directory which I don't want to access with IMAP
SPAMDIR=$HOME/Spam
# We have six mail folders alltogether: Three folders mentioned above
# and an archive folder for each of those. Following variables control
# names of those folders.
# the inbox (usually in /var/spool/mail)
INBOX=$MAIL
# manually classified spam: spam which managed to pass through the
# filter (a rare beast these days):
MANUAL_SPAM=$MAILDIR/sp
# automatically classified spam. In my case this contains all mails
# with SpamAssassin score > 5.
AUTOMATIC_SPAM=$SPAMDIR/spam
# Following three folders have suffix -archive. I append the contents
# of the above three folders to them.
INBOX_ARCHIVE=$MAILDIR/inbox-archive
MANUAL_SPAM_ARCHIVE=$SPAMDIR/sp-archive
AUTOMATIC_SPAM_ARCHIVE=$SPAMDIR/spam-archive
##
# ----- You normally don't need to customize anything below this line -------
##
# Given two files this function will append the contents of the first
# file to the second file and truncate the first file. It uses the
# locking mechanism which is also used by sendmail, imapd ad mutt It
# will not do the truncation if appending failed.
archive() {
SOURCE=`readlink -f $1`
TARGET=`readlink -f $2`
echo -n "$0: trying to lock file $TARGET..."
if lockfile -r10 -l3600 $TARGET.lock
then
echo "...success"
trap "rm -f $TARGET.lock" 1 2 3 13 15
echo -n "$0: trying to lock file $SOURCE..."
if lockfile -l3600 $SOURCE.lock
then
echo "...success"
trap "rm -f $SOURCE.lock" 1 2 3 13 15
echo -n "$0: appending $SOURCE to archive folder $TARGET"
echo -n " and truncating $SOURCE..."
cat $SOURCE >> $TARGET && cat /dev/null > $SOURCE
echo "...done"
rm -f $SOURCE.lock
else
echo "$0: couldn't lock the file $SOURCE. Exiting." 1>&2
exit 1
fi
rm -f $TARGET.lock
else
echo "$0: couldn't lock the file $TARGET. Exiting." 1>&2
exit 1
fi
}
# A small utility function
fileordie() {
if [ ! -e $1 ]
then
echo "$1: file not found. Exiting." 1>&2
exit 1
fi
}
# This is the beginning of the actual program
# If something is wrong with just one folder we exit. Just in case.
for FOLDERNAME in $INBOX $INBOX_ARCHIVE $MANUAL_SPAM $MANUAL_SPAM_ARCHIVE \
$AUTOMATIC_SPAM $AUTOMATIC_SPAM_ARCHIVE
do
fileordie $FOLDERNAME
done
# Report manually classified spam. This will also invoke sa-learn
echo "reporting manual spams"
cat $MAILDIR/sp | formail -s spamassassin --report || exit 1
archive $MAILDIR/sp $SPAMDIR/sp-archive
echo "learning automatic spams with sa-learn"
sa-learn --spam --mbox --showdots $SPAMDIR/spam || exit 1
archive $SPAMDIR/spam $SPAMDIR/spam-archive
echo "learning ham with sa-learn"
sa-learn --ham --mbox --showdots $MAIL || exit 1
archive $MAIL $MAILDIR/inbox-archive
|
Posted by Branimir Dolicki at 20:01 |
|
# - G - Add comment |
| Sunday, March 21, 2004 |
|---|
Some cool places in Copenhagen
I really enjoyed having a few beers in Bo-Bi Bar (Klareboderne 14, 1115 København) last Friday. Today I spent a very pleasant hour at Robert's Coffee (Larsbjørnstræde 17). The cafe is very cozy - there are two levels: upstairs is a nonsmoking area. Downstairs is very comfy featuring a fireplace. Before that I had a delicious falafel at Falafel House, Nygade 6.
Update: Last night I went to Indian Taj with Joel and Peter. It is an excellent Indian restaurant in
Jernbanegade 3-5, 1608 København V close to Rådhuspladsen. The food was great (they have an excellent selection of vegetarian dishes), and the atmosphere was very pleasant too.
|
Posted by Branimir Dolicki at 00:34 |
|
# - G - Add comment |
| Saturday, March 13, 2004 |
|---|
Blog and Beer [dalager.com]
As I write this I'm on a bloggers party in København so the most natural thing to do is to post a blog entry about it. Many cool people are here.
|
Posted by Branimir Dolicki at 21:52 |
|
# - G - 1 comment - Add comment |
| Friday, March 5, 2004 |
|---|
Disney Online and Hyperlinking
When you click on an external link while surfing Disney Online a popup comes up:The web site you are about to link to is not controlled by Disney Online and different terms of use and privacy policy will apply. By proceeding you agree and understand that Disney Online is not responsible for the site you are about to access.These guys have tough lawyers.
|
Posted by Branimir Dolicki at 08:46 |
|
# - G - Add comment |
| Tuesday, March 2, 2004 |
|---|
Zadar-Copenhagen [branimir.com]
I took both of these pictures in January this year. Yesterday they struck me as complementary...
|
Posted by Branimir Dolicki at 10:42 |
|
# - G - Add comment |
| Saturday, November 15, 2003 |
|---|
Spam Fighting
Update 2004-04-25: Enterprise Spam FilteringI've started to filter spam about half a year ago. Since then I've been tuning my setup. I think I've reached the point where I can say that Spam isn't a problem for me anymore. Here's how my setup looks like:
SpamAssassin version 2.60 invoked via this simple procmail rule:
The next rule will put all emails tagged by SpamAssassin into spam folder.:0fw | spamassassin
On my Linux box all I had to do is put above line in my $HOME/.procmailrc. Depending on your system you may need to invoke procmail using $HOME/.forward.:0: * ^X-Spam-Flag: YES $HOME/Mail/spam
I'm using pretty much default SpamAssassin configuration. The only two changes I did are described at the end of this post. The default configuration means among other things that the required_hits is set to 5. That means that all emails that have more than five SpamAssassin hits are considered spam. Typically, very innocent messages are large negative numbers.
DNS based IP address spam list sbl.spamhaus.org. I had some bad experience with some other blocklists that seem to shoot first and ask second. Can you believe: there are such blocklists that allow anyone to blocklist anyone else in the world without any check whatsoever. Of course I had problems with false positives. Innocent people were getting nasty bounces accusing them of spamming. Spamhaus is manually maintained and I haven't had any problems with it so far. Following lines in /etc/sendmail.cf did the trick:
R$* $: $&{client_addr}
Thanks Carsten for setting it up!
R$-.$-.$-.$- $: <?> $(ednsbl $4.$3.$2.$1.sbl.spamhaus.org. $: OK $)
R>OK $: OKSOFAR
R>$+<TMP> $: TMPOK
R>$+ $#error $@ 5.7.1 $: "550 Your mailserver spammed me, see http://www.abuse.net/sbl.phtml?IP="$&{client_addr}
Now we can enjoy looking at lines like this in /var/log/mail:
Nov 15 11:39:45 p15135922 sendmail[21367]: ruleset=check_relay, arg1=oon10.onetime-offers.net, arg2=127.0.0.2, relay=oo n10.onetime-offers.net [206.162.135.59] (may be forged), reject=550 5.7.1 Your mailserver spammed me, see http://www.ab use.net/sbl.phtml?IP=206.162.135.59
Bayesian filtering (see Paul Graham's Plan for Spam for intro on Bayesian filtering). I'm using the filter built into SpamAssassin. A nice thing about it is autolearning. Messages above certain treshold will be automatically fed into the Bayesian learning engine. I'm using the default setup that came with SpamAssassin 2.6. That is, I didn't need to do anything. It has in /usr/share/spamassassin/10_misc.cf
This means that messages with more than 12 SpamAssassin hits will automatically be learned by the Bayesian engine as spam, whereas messages with less than 0.1 hits will be learned as nonspam (also called ham in spam fighting lingo).bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_learn_threshold_spam 12.0
The best thing about Bayesian filtering is that really bad spams are actually helping you fight spam! Every spam with score above 12 will teach my Bayesian dog new tricks which can then be used also to detect spams that would otherwise not exceed the treshold.
This still leaves all the spams with score between 5 and 12 out of the Bayesian engine as well as all hams with score between 0.1 and 5. Currently I am feeding these manually into the Bayesian engine:
After enough spams have accummulated in the spam mailbox I run this script.
OK, so all emails that SpamAssassin thinks they are spams end up in $HOME/Mail/spam. What about false positives? How can I be sure that an important email didn't get misclassified as spam? Well I check the spam folder once a week. For most messages it is enough to quickly glance at the Subject line and sender. Very few need to actually be opened to be sure they are spams. But it doesn't make any sense, does it? If I have to check all the emails manually anyway what's the use of automatic spam filter? Well, the spam filter still has a huge benefit that it prevents distraction. Reading and answering email is a very different activity from filtering spam and "You have new mail" is finally a happy message again.
Still, as I get hundreds of spam messages every week it is really boring looking at all of these emails. Worse yet, after inspecting subject lines of first 50 or so messages I get inpatient and I speed up. That leaves an uneasy feeling that some legit emails might have slipped.
That's why the following final piece of my setup is absolutelly critical:
-
sorting messages in the spam folder by SpamAssassin score. That way I can look at those messages with lower score first. Those are much more likely to be legit mail falsely classified as spam. I achieved this by adding the following two lines into my $HOME/.spamassassin/user_prefs:
This adds the score assigned by SpamAssassin at the beginning of subject of every message that gets written to $HOME/Mail/spam. This way my spam folder looks like this when sorted by Subject in mutt:rewrite_subject 1 subject_tag _HITS_ |
So, I start looking at messages slowly and carefully, but by the time I reach those with score over 15 or 20 I know I can safely speed up. It is really impossible for a legit email to get a SpamAssassin report like this:1 + Nov 15 Herma Amstutz ( 110) 07.71 | Some facts you may find use 2 F Nov 14 To bdolicki@bra ( 72) 10.57 | 3 + Nov 13 Kenny Villanuev ( 205) 12.66 | Delete your Internet tracks 4 + Nov 15 Admin ( 68) 12.66 | Re: did you do it yet? 5 + Nov 14 Front office ( 70) 13.40 | bishop 6 + Nov 14 AmandaBear43@ne ( 108) 13.57 | Hiya, My name is Jennifer... 7 + Jun 11 Buford Riley ( 105) 14.19 | Add length to your Member 8 + May 05 Postmaster ( 71) 14.19 | Re: login info 9 + Nov 15 Michael Callao ( 76) 14.46 | ..,HGH Seal, Weight Loss, Fi 10 + Oct 06 Jaime Larkin ( 106) 14.69 | Hey My girl Bought me the pa 11 + Nov 14 Bradly Rhoades ( 240) 14.83 | Next day shipping on your me 12 + Nov 15 Chester Trevino ( 80) 14.94 | Did you ever know? 13 + Nov 14 Matthew Brewer ( 80) 14.94 | I have the cure. 14 + Nov 14 Willie Horn ( 242) 15.25 | Why waist your time at docto 15 + Nov 14 Antoine Dubois ( 130) 15.54 | Is V i a g r a Right For Y 16 + Nov 14 Carey Rose ( 86) 16.15 | You will add inches with thi 17 + Nov 13 Cliff C. Lay ( 114) 16.80 | Hi! 18 + Nov 14 Kelsey G. Camer ( 86) 17.10 | Hey I bought my Man the Patc 19 + Nov 14 Crystal Helms ( 92) 17.37 | Hi 20 + Nov 13 Dillon Osborn ( 91) 17.74 | The Penis Enlargement Patch
1.0 FROM_ENDS_IN_NUMS From: ends in numbers 4.1 SUBJ_VIAGRA Subject includes "viagra" 4.2 DATE_SPAMWARE_Y2K Date header uses unusual Y2K formatting 0.3 ORDER_NOW BODY: Encourages you to waste no time in ordering 4.1 VIAGRA BODY: Plugs Viagra 0.1 HTML_60_70 BODY: Message is 60% to 70% HTML 0.1 HTML_FONTCOLOR_BLUE BODY: HTML font color is blue 0.1 HTML_MESSAGE BODY: HTML included in message 0.3 HTML_FONT_BIG BODY: HTML has a big font 5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100% [score: 1.0000] 0.3 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 0.1 HTML_FONTCOLOR_RED BODY: HTML font color is red 0.6 MIME_HTML_NO_CHARSET RAW: Message text in HTML without charset 3.8 USERPASS URI: URL contains username and (optional) password 1.9 DATE_IN_FUTURE_03_06 Date: is 3 to 6 hours after Received: date 1.1 RCVD_IN_SORBS_HTTP RBL: SORBS: sender is open HTTP proxy server [24.128.81.88 listed in dnsbl.sorbs.net] 1.2 RCVD_IN_SORBS_MISC RBL: SORBS: sender is open proxy server [24.128.81.88 listed in dnsbl.sorbs.net] 0.1 RCVD_IN_SORBS RBL: SORBS: sender is listed in SORBS [24.128.81.88 listed in dnsbl.sorbs.net] 0.1 RCVD_IN_NJABL RBL: Received via a relay in dnsbl.njabl.org [24.128.81.88 listed in dnsbl.njabl.org] 1.2 RCVD_IN_NJABL_SPAM RBL: NJABL: sender is confirmed spam source [24.128.81.88 listed in dnsbl.njabl.org] 0.7 RCVD_IN_DSBL RBL: Received via a relay in list.dsbl.org [<http://dsbl.org/listing?ip=24.128.81.88>] 1.5 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net [Blocked - see <http://www.spamcop.net/bl.shtml?24.128.81.88>] 2.6 RCVD_IN_DYNABLOCK RBL: Sent directly from dynamic IP address [Dynamic/Residential IP range listed by] [easynet.nl DynaBlock - <http://dynablock.easynet.nl/errors.html>] 1.6 MISSING_MIMEOLE Message has X-MSMail-Priority, but no X-MimeOLE 1.0 FORGED_OUTLOOK_HTML Outlook can't send HTML message only 1.2 HTML_MIME_NO_HTML_TAG HTML-only message, but there is no HTML tag 1.0 FORGED_OUTLOOK_TAGS Outlook can't send HTML in this format 0.1 CLICK_BELOW Asks you to click below 1.1 MIME_HTML_ONLY_MULTI Multipart message only has text/html MIME parts 2.6 FORGED_MUA_OUTLOOK Forged mail pretending to be from MS Outlook 4.2 OBFUSCATING_COMMENT HTML comments which obfuscate textThis sorting by SpamAssassin score led me to even consider lowering the treshold from 5 to 4 or even 3. Spam filtering is not a binary choice anymore. I've noticed, for example, that legit emails with score between 4 and 5 are all messages from some lame mailing lists or (theoretically solicited) bulk email. In fact, I've noticed a strong correlation between uselesness of a message and its SpamAssassin score :-). I'm wondering whether I should instead of spam filtering simply sort all my mail by SpamAssassin score and look at those with lowest score first! After all, when was it that you have received a useful email that had only text/html MIME parts! :-)
-
|
Posted by Branimir Dolicki at 12:02 |
|
# - G - Add comment |
| Tuesday, October 21, 2003 |
|---|
Internet Keywords - Command Line Interface to the Web
When I want to look up Leo's German-English dictionary I just type "leo foo" in the Location field of my Mozilla. When I want to look up a RFC I just type "rfc 2070". This works thanks to Mozilla's delicious feature called Internet Keywords. Here's how it works: If I type into the Location field something that doesn't look like a Web address Mozilla visits a special URL (in my case http://branimir.com/kw?q=) appending to it whatever I have typed into the Location field. So, if I type "leo foo" Mozilla will request:http://branimir.com/kw?q=leo+fooNow, my little AOLserver script (source is available) will then redirect me to something like:
http://dict.leo.org/?search=foo&searchLoc=0&relink=on&spellToler=standard...If you look at the above source code you can see that Google is also supported: just type the search terms.
In order to take advantage of this feature I just had to edit my Mozilla preferences file prefs.js like this:
user_pref("keyword.URL", "http://branimir.com/kw?q=");You can use my server for this if you want. But if you are a hacker you'll probably want to customize the script so you'd be better off with your own.
user_pref("keyword.enabled", true);
This stuff is inspired by Lars Pind's Shortcuts. I liked the idea when I read it a while ago but syntax seemed awkward. With Internet Keywords I just type Command-L (Mac) or Ctrl-L (Linux, Windows) and I can immediatelly start to type my "command line".
|
Posted by Branimir Dolicki at 20:51 |
|
# - G - 2 comments - Add comment |
| Monday, October 13, 2003 |
|---|
PHP doesn't do Unicode!
I've just found out that PHP doesn't do Unicode. And I almost considered learning about that PHP thing. Oh well, it seems I will have to wait a few years.
|
Posted by Branimir Dolicki at 00:16 |
|
# - G - 1 comment - Add comment |
| Sunday, October 12, 2003 |
|---|
I added myself to the GeoURL
GeoURL is a database that maps URLs to geographic coordinates. After I found about it in Lars' blog I decided to enter my coordinates too.Apparently, clicking on the green thing below you will get all other mad bloggers near my home:
|
Posted by Branimir Dolicki at 23:15 |
|
# - G - Add comment |
| Tuesday, September 30, 2003 |
|---|
Did PowerPoint kill Columbia?
In his classic book Visual Explanations Edward R. Tufte shows how bad presentation of information had catastrophic consequence in making decision to launch the Space Shuttle Challenger on January 28, 1986.In his recent essay The Cognitive Style of PowerPoint (excellent read by the way, it is well worth the $7, buy a copy to everyone in your school or company who is PowerPoint-addict) Tufte analyzes the key slide in the Boeing report on Space Shuttle Columbia. The reports provided the rationale for NASA officials to curtail further research (such as photographing the Columbia with spy cameras) on the tiles during the flight.
Did
PowerPoint kill Columbia? Find out for yourself on Tufte's web site:
ET on Columbia Evidence—Analysis of Key Slide, also:
Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide.
|
Posted by Branimir Dolicki at 12:38 |
|
# - G - Add comment |
SPAM Filters That Fight Back!
I've been using SpamAssassin for some time now and I'm really pleased - the amount of spam I have to deal with manually has reduced to maybe 5% of the original. Especially interesting is filtering with Bayesian Classifier. Bayesian Classifier (explained at Paul Graham's Web site) learns from the past messages that you marked as spam or nonspam.Now, I find the last idea from Paul Graham Filters That Fight Back really cool. I mean, no matter how well my filters work they can't really do anything to prevent spam. Spam exists because there is (and I guess will always be) a tiny little fraction of people that actually read and respond to spam. No kidding. Such people really exist. They don't filter spam. So what can the vast majoriy of people who don't read spam and don't want spam actively do against spam? Paul Graham's idea is: have a program that will automatically follow every URL in the spam. If many did that, spammers' costs would grow linearly with amount of spam they send.
Long time ago I heard someone say that we should always call the 800 numbers advertised in the spam: call the number and go to lunch break. That way we would cost spammers money. The problem with that is that this process is hard (even impossible in some cases) to automate. The crawling idea can easily be automated.
|
Posted by Branimir Dolicki at 11:51 |
|
# - G - Add comment |
| Monday, September 8, 2003 |
|---|
Fax spammer
This morning I've got a fax spam. Apparently fax spamming is explicitly forbidden in Germany so I really want to kick those guys' asses. Fortunatelly there is this cool site: www.spammer-hammer.de which exists for exactly that purpose.SpammerHammer offers, among other things, a long template warning letter for fax-callback operators written in beautiful German legal language.
This particular spammer is already known to SpammerHammer and there are already people taking steps against him. I'm going to join them (I'm so excited! ;-).
The funniest part of this story is the telephone message that you hear when you call the listed number (which is there for the purpose of "unsubscribing"). It mentons the full address of the spammer! The mp3 of the message is here. The only problem is that the address is in Hungary and the address is in Hungarian. That is not a problem for SpammerHammer folks:
"Közleményszolgáltato es Kereskedelmi Korláltolt Feleössegü Társaság" ist der schier unausprechliche Name des großen Unbekannten, kurz Köz-Szol-Ker Kft., der angeblich in Karinthy F, út 4-6, 1111Budapest, Ungarn, residieren soll.
|
Posted by Branimir Dolicki at 11:28 |
|
# - G - Add comment |
| Sunday, September 7, 2003 |
|---|
Installed OpenACS for branimir.com
Branimir.com now runs OpenACS! I've installed version 4.6.3. It wasn't as smooth as I hoped but hey - it's working. It uses Apache as reverse proxy for virtual hosting and SSL and Postgres.BTW, I'm really impressed how db_source_sql_file works only thanks to the fact that db_get_pgbin is broken :-)
Here's what db_source_sql_file tries to do:
set fp [open "|[file join [db_get_pgbin] psql] $pghost $pgport $pguser -f $file_name [db_get_database] $pgpass" "r"]Now, this works correctly only because db_get_pgbin always returns empty string so that psql from the PATH gets executed. Here's what db_get_pgbin does:
set pool [lindex [nsv_get db_available_pools .] 0]
set driver [ns_config ns/db/pool/$pool Driver]
return [ns_config ns/db/driver/$driver pgbin]
What this seems to be trying to do is find psql binary in
a place like /usr/local/aolserver/bin. Fortunately the section name is misspelled in the last line (driver instead of drivers) so the proc returns empty string.
|
Posted by Branimir Dolicki at 04:51 |
|
# - G - Add comment |
You may request notification for branimir.com.
Here are some sites that badly need some Google PageRank:- Kindergruppe Gänseblümchen is a very good small Kindergarten in North Munich.
- Concerts in Zadar, Croatia
- International Choir Competition in Zadar






























































































