Filtering CDR open

[Eric Murray]

I've set up a new CDR that filters posts to it's subscribers- only posts
from subscribers to other CDRs or known remailers (or subscriber's alternate
addresses) will go to lne.com CDR subscribers.

Here's the list welcome message, which explains how it works in more
detail.  We're not connected to all the CDRs yet but as far as I can tell
we're not missing any posts.


-------------------------------------------------------------------------




Welcome to the lne.com Cypherpunks Distributed Remailer
This is welcome message v1.0, dated 2/22/01

Cypherpunks is a list for discussing cryptography and its impact on
society.  It is not a moderated list.  The list operators are not responsible
for the list contents.  See the following pages for cypherpunks info:


Background/general explanation:

http://world.std.com/~franl/crypto/cypherpunks.html
(read the two Manifestos-- Eric Hughes' and Tim May's-- for good
background material).

http://einstein.ssz.com/cdr

Physical meetings:

http://www.cryptorights.org/cypherpunks/meetingpunks.html

Cypherpunks list archves:

http://www.inet-one.com/cypherpunks
http://cypherpunks.venona.com
 
The Cypherpunks list is run in a distributed fashion.  There are a number of
Cypherpunks Distributed Remailers which handle distribution of the list.
Different CDRs have different policies.  http://einstein.ssz.com/cdr/index.html
explains the CDR system.

To unsubscribe from this CDR, send the words "unsubscribe cypherpunks"
(without the quotes) in the body of email to majordomo at lne.com.
If you are subscribed to another CDR, you will have to send your
unsubscribe to it.

The lne.com CDR is an attempt to reduce the spam that has invaded the
cypherpunks list without blocking or delaying non-spam posts.  The way
we do that is to 1) drop posts which are not from a subscriber to any
of the CDRs, and which are not from a known anonymous remailer  2) make
an attempt to recognize spam from the contents of the mail message and
mark spam as such.

These filtering methods affect only cypherpunks mail going to subscribers
of cypherpunks at lne.com- for example, if someone sends mail to the CDR
cypherpunks at cyberpass.com, the mail gets passed to all the CDR nodes
inluding lne for distribution to subscribers.  If the sender isn't a
subscriber to any of the CDRs, the lne.com CDR node will not send the
message to its subscribers.  It will forward the message to the other
CDRs however.

This also means that as a subscriber to the lne CDR, you may see replies
from other CDR suibscribers to posts that you didn't see, because
the were from non-subscribers.

Posts sent directly to the lne CDR need to be from a subscriber to any of
the CDRs (or a known remailer) in order to be accepted and passed on to
the other CDRs.  I'd prefer to have it send everything along, but that'd
require a whole lot of majordomo hacking and it's just not worth it.

Some cypherpunks posters post from addresses that are different
from the one that they're subscribed with, causing their post to be
dropped by lne.com.  When this happens, I'll forward their post to
cypherpunks at lne.com (headers intact) and add their alternate address
to the list of posters.   I will reserve the right to also delete an
address I've added to the list if that address starts sending spam-
this is to defeat an attack where the spammer sends a non-spam message
to the list to get his address added, and then starts spamming the list.
I doubt that I'll have to do this, but I'm mentioning it now in case it
does happen.

The spam-recognition software in use at lne.com is described at
http://www.lne.com/ericm/spammaster.  In short, it looks for key phrases
in the mail body (like "Dear Friend" and "This is not spam") and assigns
points to the different phrases.  The spam point total is put in the
"X-spam" header added at the end of the mail headers.  The mail is still
sent to the list.  The rating is there for lne.com CDR subscribers to
use or not as they please.  The spammaster config file that the lne CDR
uses is available at http://www.lne.com/cpunk/spamfilter and there is
more explanation of the software at http://www.lne.com/ericm/spammaster.

Other anti-spam measures:  lne.com has a policy of blocking mail from
sites that it deems are spam havens.  It may also use the MAPS database(s)
to block mail from spammers.  These measures will affect the posting of
mail directly to the lne CDR but will not affect the relaying of mail from
the other CDRs.  We do not get a feed from the CDR ssz, which is in the
MAPS RSS database because it's an open relay.  While we don't use the
RSS currently, our alternate MX host does.  If lne can't accept mail,
then it goes to the MX host, which rejects mail from SSZ.  Since this
creates an ugly mail loop and neither our MX or SSZ are likely to change
their policies, we don't get a feed from SSZ.  This shouldn't affect CDR
distribution though, as we'll get posts that originate from SSZ through
the other CDRs.

Lne.com runs the input to its CDR list through demime
(http://scifi.squawk.com/demime.html) which deletes MIME attachments
from mail.  Demime leaves a note in the attachments place, so that
recipients know that there was some cruft there.

Lne.com does not archive cypherpunks messages.

The Lne.com CDR strips out the "CDR:" tags in the Subject of messages
that some other CDRs put in.


If you have any questions that aren't answered by this message, send
me email.

Eric Murray

ericm at lne.com


Appendix: technical details.

Here's the deep technical stuff, for those who're interested.
Mail from other CDRs is sent to an address at lne.com, where it's processed
by procmail.  The procmail script checks for bounced mail and for dups, then
sends the mail to the other CDRs, excepting the one that sent it.
It also sends the mail to cypherpunks at lne.com, which is the list, run
by majordomo.  The list alias processes the mail first through a small
script which cleans up the headers (i.e. removing CDR: and Re: Re: from
Subject lines) and then passes it to demime.  Demime strips out any
MIME attachments, leaving a single line in their place to note what
was there.  Then the message is passed to majordomo itself.

The majordomo config is pretty much standard.  The allowed-posters
list is seperate from the subscribers list, so we can use a list of
all CDR subscribers, remailers, etc.  Mail sent directly to the lne CDR
goes to cypherpunks first, but the local address that runs through
procmail is subscribed, so the mail also gets forwarded to the other CDRs.

We generate the list of CDR subscribers by requesting a 'who cypherpunks'
from each one, once a day.  Some CDRs don't allow that command, so
for those we ask that they send the list via a cron job.  The list is
processed along with a locall-maintained list of remailers and alternate
addresses to make the list that's used for determing if a post is from
a subscriber.

The only modifications to Majordomo was a small one to allow wildcards
in addresses, so we can allow posts from *@remailer.somewhere without
having to add each different remailer user's address (for remailers
that allow different user ids to show in the From: line).





-- 
  Eric Murray           Consulting Security Architect         SecureDesign LLC
  http://www.securedesignllc.com                            PGP keyid:E03F65E5