HowTo: MailWasher with POPFile (Bayesian Filter)

gary · Last edited by gary on Sun Oct 19, 2003 11:00 am, edited 8 times in total

[Updated for version 0.20]

NOTE: THIS IS NOT FOR BEGINNERS! If you are squeamish about tinkering with your computer, making file modifications, or trying out new programs, this is not for you.

I've seen some interest expressed about getting a Bayesian filter running along with MailWasher in these forums. I've also received a few e-mail inquiries. There are a number of good Bayesian filters out there, but you need one that acts as a proxy, so that narrows the field somewhat. The one that I happen to use is POPFile, because it is quite good, and it is written in Perl, so can easily be modified. The problem with most of the filters is that they will ignore the "TOP" command, which MailWasher uses to retrieve a limited number of message lines. Bayesian filters, like MailWasher CFS, need more than 20 lines to be effective.

You can work around the TOP with POPFile with a command line switch:

"C:\Program Files\POPFile\perl.exe" popfile.pl -pop3_toptoo 1

(You may need to alter the path to the Perl executable.)

Alternately, in version 0.20 you may go to the "Advanced" tab in the user interface, and change the value of "pop3_toptoo" to "1", and save it.

This switch will cause POPFile to issue both a "TOP" and a "RETR" command when it encouters a "TOP" request from MailWasher. The "TOP" results are fed to MailWasher, and the "RETR" is used for the Bayesian filter. This slows down mail processing somewhat, so if you get a lot of e-mail you might want to use it only if you have a high speed internet connection.

There is one other hack that will help speed things up, and will cause the messages to show in your POPFile history. POPFile itself contains a hack for a utility called Fetchmail, which uses the command "TOP x 99999999" to retrieve mail, instead of "RETR". It turns out that MailWasher often uses "TOP x 9999" for some reason, so we just add another hack! (Hey, this is Perl, and we all know that TMTOWTDI, right? Wink

)

The file POP3.pm in the "proxy" subdirectory contains the key. Modify line 330 from:

Ikeb · Posted: Sat Jun 14, 2003 12:06 pm Post subject:

Thank you Gary!! Very Happy

I'll be trying this out ASAP when I have some time and my high speed link is back. Sad

Would you mind publishing these directions at your web site? I tend to lose track of which thread has what info. Embarassed

gary · Posted: Sat Jun 14, 2003 1:03 pm Post subject:

Sure, I'll publish it on my site. I'll drop a note in here when it's done.

Is there any interest in using POPFile to get to Yahoo or Hotmail? It requires using something like MrPostman, Izymail, or Web2POP, and daisy-chaining the proxies. It's a little more involved, but not terribly complicated.

TalonTSi · Corporal Joined: Mar 16, 2003 Posts: 55 Location: Canada

Works like a charm Gary, thanks! Very Happy

A couple of comments/clarifications for your documentation that I discovered while following your directions:

1. the POP3.pm filename in the proxy folder is case-sensitive (POPfile won't start if you accidentally rename it to pop3.pm)

2. there should be an extra space before the last "{" in your line modifications. Doesn't matter, except a copy/paste into a Find dialog won't find it.

3. your reference to line 338 was actually line 337 in my POP3.pm file (POPfile v0.19.0)

Thanks again for the great documentation! I've wanted to pair a Bayesian filter with Mailwasher for a long time. Tried K9, but it didn't work. This is exactly what I was looking for! Thumbs Up

gary · Posted: Sat Jun 14, 2003 4:01 pm Post subject:

TalonTSi · Corporal Joined: Mar 16, 2003 Posts: 55 Location: Canada

Guest · Guest

When I change my inbound settings in the new version of MWP as specified above, I get the same error I was having with the previous version - server intentionally closed - this happens on every mail check, just like it did before. I really want to use the MWP with POPfile - but looks like with ATTBI it is a no go. Maybe it will work when I am officially switched to Comcast at the end of the month.

gary · Posted: Sun Jun 15, 2003 2:32 am Post subject:

Are you internal to ATTBI's network, or external? I access it both ways, but I can't use POPFile externally, since a connection external to their network requires SSL.

Guest · Guest

Well I must be internal since I do not require SSL.

Guest · Posted: Sun Jun 15, 2003 12:55 pm Post subject:

Gary,

Excellent how-to and it made me head out, get POPfile and install it Smile

Just one thing that is not yet clear to me : the POP3.pm hacking in the proxy directory, does that replace the

"C:\Program Files\POPFile\perl.exe" popfile.pl -pop3_toptoo 1"

change, or should/can both be done?

Guest · Guest

Well it seems to be working today - I am not gettng the error message. I would guess that it was user error - perhaps I did not have POPFile started when I was checking mail.

However, I am not able to get MWP and POPFile working together. My email program does not support the X-Test-Classification header. So I am just using Subject modification into the buckets I want. Without MWP, it works fine. So I must be doing something wrong. I created an MWP filter that as follows "Subject" "contains " "[spam]" and put that as my first filter. But it is ignored. Any ideas?

gary · Posted: Sun Jun 15, 2003 3:18 pm Post subject:

Guest · Guest

My email program does not support using the X-Test-Classification header. If I use the Subject header in POPFile, POPFile marks it correctly and passes it on to my email program correctly and puts it in the correct folder. But I do not see MWP involved in this process - it will mark the email according to the filters already there - not to the newly created filters for POPFile. BTW, I did try the X-Test-Classification filter as you suggested, even though it won't work with my email program just to see - and same result - it is ignored by MWP.

Guest · Posted: Mon Jun 16, 2003 5:50 am Post subject:

I had the same problem - with the filtering that is - and solved it. You added a new filter for POPfile, but it's probably at the bottom of your filter list. Move it to the top and see if MWP uses it then.

Guest · Guest

Actually I did the POPFile filter to the top of my list - so that isn't the problem. I think I must be doing something else wrong but just can'f figure out what it is. And as I think about it, my setup as it is now should not work. I have POPFile running the background but it does not apparently attach tags until my email program is called. Thus MWP is left completely out of the loop - it is filtering email before POOPFile attaches the tags so the filters I have for POPFile would be ignored. What am I missing here?


	Home ˇ Topics ˇ Submit News ˇ Top 10 This entire site, Cops Themes, and Computer Cops are Š 2002 - 2004 Computer Cops, LLC. All rights reserved. You can syndicate our news using the file RSS 0.91, ultramode.txt, or RSS 1.0. Acceptable Use Policy. Use signifies your agreement. Engine Copyright Š 2002 by PHP-Nuke, GNU/GPL Licensed. ICRA Member. Paul Laudanski, Member of Computer Cops, LLC Server Load: 532 pages served in previous 5 minutes. Page Rendered: 5.442 seconds.