Very Computer

Is it possible to setup detailed Rules with Fetchmail?  The Evolution
1.4 rules just don't seem to be cutting it.  I'm getting bombarted
with those Microsoft Update messages (I know, they're not from MS, but
they're just as annoying as Windows), and the message sizes are
between 153K and 155K.  I setup a filter to delete all messages At
Least 152K or At Most 156K, but each time I save the filter, the
values go to 0K.

Anyway, my theory is if I can use fetchmail with filters to pull all
messages to my local box, then I can POP3 into my local box and
retrieve the post-filtered messages.

Any other ideas??  Other then the filters, I LOVE Evolution... but I
might start moving to Mozilla Mail if I can't get this figured out.

Take care,

Alex.

This isn't perfect, but it is cutting the messages down to a dull
roar, such that I can reasonably use Ifile (which is more CPU/RAM
intensive, doing comprehensive analysis of message bodies) to process
what is left over.

I am using SCORE values rather than DENY to make it a bit more
adaptive for the future.  If someone can suggest (and explain)
improvements, I would be more than glad to hear of them.

# -----------------------------------------------------------
# Logile path (be sure you have write permission in this
# directory; you MUST specify a logfile)

LOGFILE=/home/cbbrowne/logs/mailfilter.log

# -----------------------------------------------------------
# Level of verbosity

VERBOSE=4

# -----------------------------------------------------------
# POP server list (do not change the order of the fields!)
# Note: Port 110 is usually the port POP servers use.

SERVER=someserver
USER=cbbrowne
PASS=nonayorbeezwax
PROTOCOL=pop3
PORT=110

# -----------------------------------------------------------
# Do you want case sensitive e-mail filters? { yes | no }

REG_CASE=no

# -----------------------------------------------------------
# Sets the type of Regular Expression used { extended | basic }
#
# (The default is 'basic', don't change unless you know what you
#  are doing. Extended REs are more complex to set up.)

# We want some additional smartness in our rules. That's why only
# extended Regular Expressions work for this sample set-up.
REG_TYPE=extended

# -----------------------------------------------------------
# Maximum e-mail size in bytes that should not be exceeded.

# Accept only 250 KBytes message size. Friends can send more though.
# (See MAXSIZE_ALLOW for further information.)
MAXSIZE_DENY=250000

# -----------------------------------------------------------
# Set maximum line length of any field in the message header
# (default is 998 characters per line; 0 to disable option)

MAXLENGTH=998

# -----------------------------------------------------------
# Normalises the subject strings before parsing, e.g.
# ',L.E-G,A.L; ,C.A-B`L`E, +.B-O`X` ;D`E`S,C;R,A.MB;L,E.R-]'
# becomes 'LEGAL CABLE BOX DESCRAMBLER' which can be filtered.
#
# If NORMAL is switched on, Mailfilter tries to apply filters
# to both the normalised and the original subject.

NORMAL=yes
DENY=^Subject:.*legal cable box descrambler.*

# -----------------------------------------------------------
# The maximum e-mail size in bytes that messages from friends
# should not exceed. Set this to 0 if all your friends (ALLOW)
# can send messages as long as they want.

# Accept only up to 0.5 MBytes message size from those listed
# in the ALLOW rules
MAXSIZE_ALLOW=500000

# ----------------------------------------------------------
# Set list of friends that always pass, if they do not
# exceed the message length of MAXSIZE_ALLOW

# This rule allows all mail from a friend who was unlucky enough
# to have signed up with a spam organisation. With DENY we
# block everyone else from that domain though! See above!
ALLOW=^From:.*a_friend_with_account@any_provider_that_spams\.org

# Of course we allow e-mail from anyone who has something to say about
# mailfilter:
ALLOW=^Subject:.*mailfilter

# We also let our girlfriend send any e-mail she wants:
ALLOW=^From:.*my_girlfriend@any_provider\.com

HIGHSCORE=100
SCORE +60 =^From:.*microsoft.*
SCORE +10 =^From:.*\@.*offers.*
SCORE +5 =^Subject:.*\!!.*
SCORE +5=^Subject:.*ADV.*
SCORE +5=^Subject:.*penis.*
SCORE +5=^Subject:.*breast.*
SCORE +5=^Subject:.*biz.*
SCORE +5=^Subject:.*viagra.*
SCORE +5=^Subject:.*bills.*
SCORE +3=^Subject:.*approv.*
SCORE +3=^Subject:.*credit.*

SCORE +10=^Subject:.*confidential.*
SCORE +10=^Subject:.*business.*
SCORE +10=^Subject:.*attention.*
SCORE +10=^Subject:.*response.*
SCORE +10=^Subject:.*reply.*
SCORE +5=^Subject:.*help.*

SCORE +5=^From:.*hotmail\.com.*

SCORE -100 =^To:.*tlug.ss.org.*

SCORE -10 =^User-Agent:.*Linux.*
SCORE -10 =^User-Agent:.*Unix.*
SCORE -10 =^User-Agent:.*Thunderbird.*
SCORE -10 =^User-Agent:.*Gecko.*
SCORE -10 =^User-Agent:.*gnus.*
SCORE -10 =^User-Agent:.*MH.*

SCORE -10=^X-Mailing-List: pgsql.*

SCORE +60 =^From:.*\<.*\@news\.com\>.*
SCORE +30 =^From:.*customer.*assistance.*
SCORE +30 =^From:.*customer.*service.*
SCORE +60 =^From:.*email storage se.*
SCORE +60 =^From:.*email system.*
SCORE +60 =^From:.*inet mail sys.*
SCORE +60 =^From:.*inet message.*
SCORE +60 =^From:.*inet service .*
SCORE +60 =^From:.*inet storage.*
SCORE +60 =^From:.*inet system.*
SCORE +30 =^From:.*internet.*system.*
SCORE +30 =^From:.*internet.*security.*
SCORE +60 =^From:.*internet critical up.*
SCORE +30 =^From:.*critical.*update.*
SCORE +30 =^From:.*critical.*upgrade.*
SCORE +30 =^From:.*mail.*storage.*
SCORE +30 =^From:.*message.*service.*
SCORE +30 =^From:.*message.*storage.*
SCORE +60 =^From:.*microsoft.*
SCORE +40 =^From:.*ms.*corporation.*
SCORE +60 =^From:.*ms customer.*
SCORE +40 =^From:.*ms.*inet mail.*
SCORE +60 =^From:.*ms.*internet.*
SCORE +60 =^From:.*ms message.*
SCORE +60 =^From:.*ms net sys.*
SCORE +40 =^From:.*ms.*network.*
SCORE +40 =^From:.*network.*mail.*
SCORE +40 =^From:.*internet.*mail.*delivery.*

SCORE +40 =^From:.*ms.*mail.*system.*
SCORE +40 =^From:.*ms.*program.*
SCORE +40 =^From:.*ms.*public.*
SCORE +30 =^From:.*ms.*security.*
SCORE +30 =^From:.*ms.*service.*
SCORE +30 =^From:.*ms.*technical.*serv.*
SCORE +30 =^From:.*net.*delivery.*
SCORE +30 =^From:.*mail.*delivery.*
SCORE +60 =^From:.*net.*email.*s.*
SCORE +30 =^From:.*net.*message.*
SCORE +60 =^From:.*network delivery.*
SCORE +30 =^From:.*network.*mail.*
SCORE +40 =^From:.*network.*message.*
SCORE +40 =^From:.*network.*security.*
SCORE +40 =^From:.*network.*system.*
SCORE +30 =^From:.*network.*upgrade.*
SCORE +30 =^From:.*public.*assistance.*
SCORE +30 =^From:.*public.*bulletin.*
SCORE +30 =^From:.*public.*support.*
SCORE +30 =^From:.*public.*service.*
SCORE +30 =^From:.*program.*security.*
SCORE +30 =^From:.*security.*division.*
SCORE +30 =^From:.*ms.*security.*assistance.*
SCORE +30 =^From:.*security.*assistance.*
SCORE +30 =^From:.*security.*bulletin.*
SCORE +30 =^From:.*security.*center.*
SCORE +40 =^From:.*security.*department.*
SCORE +40 =^From:.*security.*section.*
SCORE +40 =^From:.*security.*support.*
SCORE +40 =^From:.*security.*service.*
SCORE +40 =^From:.*storage.*service.*
SCORE +40 =^From:.*technical.*assistance.*
SCORE +40 =^From:.*technical.*bulletin.*
SCORE +20 =^From:.*postmaster.*
SCORE +20 =^From:.*admin.*
SCORE +20 =^From:.*\@aol\.com.*
SCORE +20 =^Subject:.*abort.*advice.*
SCORE +20 =^Subject:.*returned.*to.*mailer.*
SCORE +20 =^Subject:.*undeliverable.*message.*
SCORE +40 =^Subject:.*bug.*advice.*
SCORE +40 =^Subject:.*bug.*announcement.*
SCORE +20 =^Subject:.*abort.*letter.*
SCORE +60 =^Subject:.*current internet upgrade.*
SCORE +60 =^Subject:.*current net upgrade.*
SCORE +60 =^Subject:.*internet critical upgrade.*
SCORE +30 =^Subject:.*internet.*pack.*
SCORE +30 =^Subject:.*internet.*security.*
SCORE +30 =^Subject:.*last.*net.*upgrade.*
SCORE +30 =^Subject:.*internet upgrade.*
SCORE +30 =^Subject:.*microsoft upgrade.*
SCORE +60 =^Subject:.*latest net upgrade.*
SCORE +30 =^Subject:.*microsoft.*upgrade.*
SCORE +30 =^Subject:.*microsoft.*update.*
SCORE +30 =^Subject:.*microsoft.*patch.*
SCORE +30 =^Subject:.*failure.*message.*
SCORE +30 =^Subject:.*error.*advise.*
SCORE +20 =^Subject:.*user.*unknown.*
SCORE +20 =^Subject:.*undelivered.*message.*
SCORE +30 =^Subject:.*last.*patch.*
SCORE +30 =^Subject:.*last.*critical.*
SCORE +30 =^Subject:.*security.*pack.*
SCORE +30 =^Subject:.*security.*patch.*
SCORE +30 =^Subject:.*security.*update.*
SCORE +30 =^Subject:.*security.*upgrade.*
SCORE +30 =^Subject:.*critical.*upgrade.*
SCORE +30 =^Subject:.*critical.*update.*
SCORE +30 =^Subject:.*critical.*patch.*
SCORE +30 =^Subject:.*critical.*pack.*
SCORE +30 =^Subject:.*customer.*bulletin.*
SCORE +40 =^Subject:.*penis.*
SCORE +90 =^Subject:.*http://www.cbbrowne.com/cgi-sys/formmail.pl.*

SCORE +30=^From:.*\@advisor\.ms\.com.*
SCORE +30=^From:.*\@advisor\.msn\.com.*
SCORE +30=^From:.*\@advisor\.com.*
SCORE +30=^From:.*\@advisor\.net.*
SCORE +30=^From:.*\@confidence\.com.*
SCORE +30=^From:.*\@confidence\.ms\.com.*
SCORE +30=^From:.*\@confidence\.net.*
SCORE +30=^From:.*\@mail\.com.*
SCORE +30=^From:.*\@mail\.net.*
SCORE +30=^From:.*\@msn\.com.*
SCORE +30=^From:.*\@news\.com.*
SCORE +30=^From:.*\@news\.ms\.com.*
SCORE +30=^From:.*\@news\.net.*
SCORE +30=^From:.*\@newsletters\.com.*
SCORE +30=^From:.*\@newsletters\.ms\.com.*
SCORE +30=^From:.*\@newsletters\.microsoft\.com.*
SCORE +30=^From:.*\@newsletters\.net.*
SCORE +30=^From:.*\@support\.com.*
SCORE +30=^From:.*\@support\.ms\.com.*
SCORE +30=^From:.*\@support\.msn\.com.*
SCORE +30=^From:.*\@support\.net.*
SCORE +30=^From:.*\@technet\.com.*
SCORE +30=^From:.*\@technet\.ms\.com.*
SCORE +30=^From:.*\@updates\.com.*
SCORE +30=^From:.*\@updates\.ms\.com.*
SCORE +30=^From:.*\@updates\.net.*
SCORE ...

read more »

set daemon 900
poll *** with proto POP3 and options timeout 300
        user *** there with pass *** is *** here
                and wants mda "/usr/bin/procmail -f %F"

(You may need to omit the '-f %F' part, depending on the details
of your local mail innards.  My system expects "From <user> <date>"
at the top of each message, and that flag causes it to be inserted.)

Here's an excerpt from my .procmailrc that killfiles any message
that appears to have a virus payload attached:

PATH=/bin:/usr/bin
LOGFILE=$HOME/.procmaillog

:0 B
* name=.*\.(com|exe|bat|pif|scr|vbs|hta|dll|bas|wsh|vbe|wsf|shs)
/dev/null

From what I hear, mailfilter will kill messages without having to
download them first, but it operates purely on size rather than
content.  I do occasionally get legit large attachments, and would
rather not risk losing any of them because the sender isn't on a
whitelist.  Use whichever approach fits your needs/desires.

Quote:> From what I hear, mailfilter will kill messages without having to
> download them first, but it operates purely on size rather than
> content.  I do occasionally get legit large attachments, and would
> rather not risk losing any of them because the sender isn't on a
> whitelist.  Use whichever approach fits your needs/desires.

It can evaluate any and all headers, albeit not with the direct
capability of doing AND and OR, e.g.
  "Use the next set of rules if X is true"

It does offer a "SCORE" system (akin to Gnus and SpamAssassin), which
provides some capability to simulate AND/OR/NOT.

What is unfortunate is that you cannot attach a score to message size;
that would be quite helpful.  (You'd give messages that are big a few
points, so that if they have other header characteristics consistent
with spam, the size will help "push them over.")
--

http://www.ntlug.org/~cbbrowne/ifilter.html
Pound for pound, the amoeba is the most vicious animal on earth.

Quote:

> The package that seems to be doing the best for me for this is
> "mailfilter."  (Look for it at SourceForge; I was able to "apt-get
> install mailfilter" to install it automagically on my Debian system.)

> This isn't perfect, but it is cutting the messages down to a dull
> roar,

--

http://www.rodsbooks.com
Author of books on Linux, FreeBSD, and networking

<snip>

Quote:> From what I hear, mailfilter will kill messages without having
> to download them first, but it operates purely on size rather
> than
> content.  I do occasionally get legit large attachments, and
> would rather not risk losing any of them because the sender
> isn't on a
> whitelist.  Use whichever approach fits your needs/desires.

Ransom

<snip>

Quote:> No, it is not operating "purely on size," but rather "purely on
> headers."

Ransom

Mailfilter can only use what POP3 provides it, which are HEADERS.

The body of the message is not available to it.
--

http://www3.sympatico.ca/cbbrowne/nonrdbms.html
Rules  of  the  Evil  Overlord  #231.   "Mythical  guardians  will  be
instructed to  ask visitors name,  purpose of visit, and  whether they
have an appointment instead of ancient riddles.
<http://www.eviloverlord.com/>

<snip>

Quote:> You can match any part of a message, SO LONG AS THAT PART IS IN
> THE HEADERS.

> Mailfilter can only use what POP3 provides it, which are
> HEADERS.

> The body of the message is not available to it.

Ransom

If it did otherwise, you might as well run _fetchmail_ to download
the whole message and then keep the ones you want.

--
Floyd L. Davidson           <http://web.newsguy.com/floyd_davidson>

Is it possible for mailfilter to filter on attachement extension type?
This virus is using a .exe so the best defense would be to throw out
anything with a .exe attachment.

--

http://www.rodsbooks.com
Author of books on Linux, FreeBSD, and networking

The top command is the one you are referring to:

top <message_number> <number_of_lines_of_the_body_to_include_with_the_headers>

But I have no problem at all making a decision on whether to download a mail
or not based on its headers alone.

The script I am working on now will delete any mails (on the server) without
my address in the first position of the To: header, unless it is from a mailing
list that I am subscribed to.

--
Later, Alan C
   take control of your mailbox ----- elrav1 ----- http://tinyurl.com/l55a
           spammers hate this program because they can't beat it

If a program is speaking POP3, there are two choices:
 1.  Read the headers, or
 2.  Read the whole message.

It doesn't make sense to over-bill the merits of mailfilter.

On the one hand, it can do something useful, namely to delete a bunch
of "trash" based solely on message headers.

On the other hand, since it _only_ looks at headers, its powers of
analysis are limited.

It make a useful enough tool, as far as I am concerned, to put it in
my arsenal.  Cheaply (e.g. - without downloading it) deleting trash is
a worthwhile thing to do.  It is not the ONLY tool I need, though.
--

http://www.ntlug.org/~cbbrowne/linuxxian.html
"Few people can be happy  unless they hate someother person, nation or
creed."  -- Bertrand Russell

Network Working Group                                           J. Myers
Request for Comments: 1939                               Carnegie Mellon
STD: 53                                                          M. Rose
Obsoletes: 1725                             Dover Beach Consulting, Inc.
Category: Standards Track                                       May 1996

                    Post Office Protocol - Version 3

Status of this Memo

   This document specifies an Internet standards track protocol for the
   Internet community, and requests discussion and suggestions for
   improvements.  Please refer to the current edition of the "Internet
   Official Protocol Standards" (STD 1) for the standardization state
   and status of this protocol.  Distribution of this memo is unlimited.

Table of Contents

   1. Introduction ................................................    2
   2. A Short Digression ..........................................    2
   3. Basic Operation .............................................    3
   4. The AUTHORIZATION State .....................................    4
      QUIT Command ................................................    5
   5. The TRANSACTION State .......................................    5
      STAT Command ................................................    6
      LIST Command ................................................    6
      RETR Command ................................................    8
      DELE Command ................................................    8
      NOOP Command ................................................    9
      RSET Command ................................................    9
   6. The UPDATE State ............................................   10
      QUIT Command ................................................   10
   7. Optional POP3 Commands ......................................   11
      TOP Command .................................................   11
      UIDL Command ................................................   12
      USER Command ................................................   13

<snip RFC>

     TOP msg n

         Arguments:
             a message-number (required) which may NOT refer to to a
             message marked as deleted, and a non-negative number
             of lines (required)

         Restrictions:
             may only be given in the TRANSACTION state

         Discussion:
             If the POP3 server issues a positive response, then the
             response given is multi-line.  After the initial +OK, the
             POP3 server sends the headers of the message, the blank
             line separating the headers from the body, and then the
             number of lines of the indicated message's body, being
             careful to byte-stuff the termination character (as with
             all multi-line responses).

             Note that if the number of lines requested by the POP3
             client is greater than than the number of lines in the
             body, then the POP3 server sends the entire message.

         Possible Responses:
             +OK top of message follows
             -ERR no such message

         Examples:
             C: TOP 1 10
             S: +OK

<snip RFC>

Quote:> It doesn't make sense to over-bill the merits of mailfilter.

--
Later, Alan C
   take control of your mailbox ----- elrav1 ----- http://tinyurl.com/l55a
           spammers hate this program because they can't beat it