Filtering CDR open

[Declan McCullagh]

FYI I'm subscribed to cypherpunks from one address; I post to it 
from a second; my From: line is a third. I sometimes use a fourth address
to post.

I'm not sure how you're going to handle that, but you might want to consider
semi-automated ways of extracting addresses from "known good" posts,
to the extent that any of mine fit that category.

-Declan



On Thu, Feb 22, 2001 at 10:13:34AM -0800, Eric Murray wrote:
> I've set up a new CDR that filters posts to it's subscribers- only posts
> from subscribers to other CDRs or known remailers (or subscriber's alternate
> addresses) will go to lne.com CDR subscribers.
> 
> Here's the list welcome message, which explains how it works in more
> detail.  We're not connected to all the CDRs yet but as far as I can tell
> we're not missing any posts.
> 
> 
> -------------------------------------------------------------------------
> 
> 
> 
> 
> Welcome to the lne.com Cypherpunks Distributed Remailer
> This is welcome message v1.0, dated 2/22/01
> 
> Cypherpunks is a list for discussing cryptography and its impact on
> society.  It is not a moderated list.  The list operators are not responsible
> for the list contents.  See the following pages for cypherpunks info:
> 
> 
> Background/general explanation:
> 
> http://world.std.com/~franl/crypto/cypherpunks.html
> (read the two Manifestos-- Eric Hughes' and Tim May's-- for good
> background material).
> 
> http://einstein.ssz.com/cdr
> 
> Physical meetings:
> 
> http://www.cryptorights.org/cypherpunks/meetingpunks.html
> 
> Cypherpunks list archves:
> 
> http://www.inet-one.com/cypherpunks
> http://cypherpunks.venona.com
>  
> The Cypherpunks list is run in a distributed fashion.  There are a number of
> Cypherpunks Distributed Remailers which handle distribution of the list.
> Different CDRs have different policies.  http://einstein.ssz.com/cdr/index.html
> explains the CDR system.
> 
> To unsubscribe from this CDR, send the words "unsubscribe cypherpunks"
> (without the quotes) in the body of email to majordomo at lne.com.
> If you are subscribed to another CDR, you will have to send your
> unsubscribe to it.
> 
> The lne.com CDR is an attempt to reduce the spam that has invaded the
> cypherpunks list without blocking or delaying non-spam posts.  The way
> we do that is to 1) drop posts which are not from a subscriber to any
> of the CDRs, and which are not from a known anonymous remailer  2) make
> an attempt to recognize spam from the contents of the mail message and
> mark spam as such.
> 
> These filtering methods affect only cypherpunks mail going to subscribers
> of cypherpunks at lne.com- for example, if someone sends mail to the CDR
> cypherpunks at cyberpass.com, the mail gets passed to all the CDR nodes
> inluding lne for distribution to subscribers.  If the sender isn't a
> subscriber to any of the CDRs, the lne.com CDR node will not send the
> message to its subscribers.  It will forward the message to the other
> CDRs however.
> 
> This also means that as a subscriber to the lne CDR, you may see replies
> from other CDR suibscribers to posts that you didn't see, because
> the were from non-subscribers.
> 
> Posts sent directly to the lne CDR need to be from a subscriber to any of
> the CDRs (or a known remailer) in order to be accepted and passed on to
> the other CDRs.  I'd prefer to have it send everything along, but that'd
> require a whole lot of majordomo hacking and it's just not worth it.
> 
> Some cypherpunks posters post from addresses that are different
> from the one that they're subscribed with, causing their post to be
> dropped by lne.com.  When this happens, I'll forward their post to
> cypherpunks at lne.com (headers intact) and add their alternate address
> to the list of posters.   I will reserve the right to also delete an
> address I've added to the list if that address starts sending spam-
> this is to defeat an attack where the spammer sends a non-spam message
> to the list to get his address added, and then starts spamming the list.
> I doubt that I'll have to do this, but I'm mentioning it now in case it
> does happen.
> 
> The spam-recognition software in use at lne.com is described at
> http://www.lne.com/ericm/spammaster.  In short, it looks for key phrases
> in the mail body (like "Dear Friend" and "This is not spam") and assigns
> points to the different phrases.  The spam point total is put in the
> "X-spam" header added at the end of the mail headers.  The mail is still
> sent to the list.  The rating is there for lne.com CDR subscribers to
> use or not as they please.  The spammaster config file that the lne CDR
> uses is available at http://www.lne.com/cpunk/spamfilter and there is
> more explanation of the software at http://www.lne.com/ericm/spammaster.
> 
> Other anti-spam measures:  lne.com has a policy of blocking mail from
> sites that it deems are spam havens.  It may also use the MAPS database(s)
> to block mail from spammers.  These measures will affect the posting of
> mail directly to the lne CDR but will not affect the relaying of mail from
> the other CDRs.  We do not get a feed from the CDR ssz, which is in the
> MAPS RSS database because it's an open relay.  While we don't use the
> RSS currently, our alternate MX host does.  If lne can't accept mail,
> then it goes to the MX host, which rejects mail from SSZ.  Since this
> creates an ugly mail loop and neither our MX or SSZ are likely to change
> their policies, we don't get a feed from SSZ.  This shouldn't affect CDR
> distribution though, as we'll get posts that originate from SSZ through
> the other CDRs.
> 
> Lne.com runs the input to its CDR list through demime
> (http://scifi.squawk.com/demime.html) which deletes MIME attachments
> from mail.  Demime leaves a note in the attachments place, so that
> recipients know that there was some cruft there.
> 
> Lne.com does not archive cypherpunks messages.
> 
> The Lne.com CDR strips out the "CDR:" tags in the Subject of messages
> that some other CDRs put in.
> 
> 
> If you have any questions that aren't answered by this message, send
> me email.
> 
> Eric Murray
> 
> ericm at lne.com
> 
> 
> Appendix: technical details.
> 
> Here's the deep technical stuff, for those who're interested.
> Mail from other CDRs is sent to an address at lne.com, where it's processed
> by procmail.  The procmail script checks for bounced mail and for dups, then
> sends the mail to the other CDRs, excepting the one that sent it.
> It also sends the mail to cypherpunks at lne.com, which is the list, run
> by majordomo.  The list alias processes the mail first through a small
> script which cleans up the headers (i.e. removing CDR: and Re: Re: from
> Subject lines) and then passes it to demime.  Demime strips out any
> MIME attachments, leaving a single line in their place to note what
> was there.  Then the message is passed to majordomo itself.
> 
> The majordomo config is pretty much standard.  The allowed-posters
> list is seperate from the subscribers list, so we can use a list of
> all CDR subscribers, remailers, etc.  Mail sent directly to the lne CDR
> goes to cypherpunks first, but the local address that runs through
> procmail is subscribed, so the mail also gets forwarded to the other CDRs.
> 
> We generate the list of CDR subscribers by requesting a 'who cypherpunks'
> from each one, once a day.  Some CDRs don't allow that command, so
> for those we ask that they send the list via a cron job.  The list is
> processed along with a locall-maintained list of remailers and alternate
> addresses to make the list that's used for determing if a post is from
> a subscriber.
> 
> The only modifications to Majordomo was a small one to allow wildcards
> in addresses, so we can allow posts from *@remailer.somewhere without
> having to add each different remailer user's address (for remailers
> that allow different user ids to show in the From: line).
> 
> 
> 
> 
> 
> -- 
>   Eric Murray           Consulting Security Architect         SecureDesign LLC
>   http://www.securedesignllc.com                            PGP keyid:E03F65E5