Protect against false positives in Spam Filtering with Whitelists

Angelegt von suung Mon, 04 Oct 2010 22:53:00 GMT

It is a huge problem for both sides: The companies trying to send information to customers and the recipients, trying to figure out wanted email from junk ordners.

It is  not the easiest thing to setup a mail server so that it avoids being in error marked positive as spam.

The popular german mailservice GMX has published a long list of tips, that administrators of foreign mailservers should follow.

The problem is: It's better to  be hard. There are two main principles:

a) blacklisting

b) intelligence

In a, there is a list of mailservers, that should not be trusted. Some companies provide these blacklists and they are used in most of our tools, embedded in Thunderbird or guarding in front of our freemail services.

In b, everything  is about  pattern recognition   such as hidden markov chains 

In  both cases, and usually they are combined, by error, be it a machine that decides wrongly or more often a user that clicks on 'junk' can influence decisions and create false positives.
And Mailservers could run by bad configuration into the trap. And you could use email adresses or domain names, containing words in spam keyword lists.

In the case you run a young business, this can really suck. You sent invites to people or even your visitors invite their friends - and the mail is going to junk.

There is one principle, that could help here, called whitelisting.  

A good whitelist is the opposite of a blacklist. If you are good, your name is on the list and you can send as much spam as you want.

I know two  projects, that aim to provide a commonly used email  whitelist:

Spamhaus Whitelist  and the larger DNSWL

Amazon filtert sarkastische Kommentare?

Angelegt von andi Thu, 01 Jul 2010 15:02:00 GMT

Wer sich schon daran gewoehnt hat - wenn auch vermutlich nicht gerade mit Freude, dass kritische oder hinterfragende Kommentare auf Internetplattformen einfach verschwinden, wird sich vielleicht eher nicht darueber freuen, dass diese jetzt automatisiert erkannt werden sollen.

Aus einer rein technischen Perspektive natuerlich hoch spannend.

Hier der Heise-Artikel: http://www.heise.de/newsticker/meldung/Filter-erfasst-sarkastische-Online-Kommentare-1031020.html

Und hier das PDF in dem Entwicklung und Sinn des Algorithmus beschrieben wird:

http://staff.science.uva.nl/~otsur/papers/sarcasmAmazonICWSM10.pdf

 

Rubygame - Library for writing games in Ruby

Angelegt von andi Mon, 31 May 2010 12:39:00 GMT

Stumbling over http://rubygame.org/ just after if finished my last blog-article, i first thought - oh know, is it another game to steal my precious time? (okay okay, rubywarrior is about precious ruby, not about my precious time, but…)

But, phew, it’s declaring itself as a library for writing games in ruby. Actually i don’t know if it focuses on any particular kind of games, but it surly is worth a look - maybe not only for writing games. From surfing through its rdoc i found a lot of methods like RubyGame::Events::MouseMoved or RubyGame::Events::KeyPressed.

If someone had a closer look, i’d love to be updated as well. I mean, there is useful stuff too that you can write as a game :)

Saving precious princess Ruby

Angelegt von andi Mon, 31 May 2010 11:28:00 GMT

Whoo, i actually did spent some hours this weekend playing rubywarrior, a game allowing you to write you’re own player-bot and send it through some kind of dungeons with sludges, wizards and archers longing to kill you.

Though i was doing what i do in my daily work - lurking around in my emacs, writing code, it felt quite a lot like this gaming spirit that i remember from the days when i used to play computers. Even some - usually when playing games quite reasonably - fear sneaked into my mind, telling me i could waste my time.

But actually I think, though i didn’t learn so much new ways of doing things, i think i learned a lot by simply combining ‘playing’ with ‘coding’, which actually is quite similar in its essence. But playing rubywarrior is less playing against my issue-tracker or a clients agenda, but more like playing against… erm, archers and sludges… well, okay, that could be considered quite similar as well… Anyhow - it has been a welcome change.

I will love to publish my - honestly clearly not completely well structured player, but not before some other players are done, allowing us to compare and learn from each other.

Sorry people, i don’t want to encourage you wasting your time - but probably it’s really not a waste. At least not if you play in your leisure :)

Oh, well - and i discovered this fork of rubywarrior - originally written by Ryan Bates, which we all do already now, e.g. from Railscasts.com - anyhow, looking at the forks github-page makes me belive there are new levels and functionality added. The voice in my head telling me not to waste my time, is still arguing. Anyhow, the fork can be found here.

Wish you a lot of fun!