net of 11 more spam hosts User-Agent: Mutt/1.5.6+20040907i on Thu, Mar 03, 2005 at 12:05:48PM -0500, Aaron Sherman (ajs@ajs.com) wrote:
On Wed, 2005-03-02 at 23:52, Karsten M. Self wrote:
on Wed, Mar 02, 2005 at 06:16:38PM -0500, Aaron Sherman (ajs@ajs.com) wrote:
Ok, I won't speak for Nick, but here's why I get all flustered about this kind of thing: I have this nervous tick that forces me to imagine how every "victory" in the war against X (drugs, spam, terrorism, you name it) will, in turn, be used against me in future.
[...]
I believe in a _balance_ of powers, a weighting on _merits_, and an avoidance of _extremes_. Large low-entropy pools are inherently dangerous.
Interesting. I would not have thought to phrase it that way, but it's a fair point.
Thank you.
_Those_ are the particulars of _this_ case.
Yep, and I'm glad they've been booted, and I was over-reacting as a result of a mis-reading. However, I defend the reaction in that, had the root servers (or some subset) taken such unilateral action, I think it would have been wildly inappropriate and would have set a frightening precedent.
The root servers situation is interesting and unclear. I haven't been referring to it. That said: if someone thumbs their nose at the world sufficiently long and hard, they shouldn't be too suprised to find the world thumbing back. Unconstrained, this does lead to breakdowns and opportunities for abuse (Joe-jobs, reputation trashing). I can't deny feeling a sense of karmic justice, however.
I know you may have come away from the previous discussion with the sense that I have a desire to defend send-safe, but I assure you that nothing could be farther from the truth. My feeling on the MCI policy issue is that, worst-case, they should have amended their rules, refunded send-safe's money and terminated the contract long ago.
s/refunded.*money// ...particulary in the face of an AUP denying network abuse, spamming, etc. Send-safe were likely skating under the rules and were taking an overt business risk in doing so.
I do want to point out, though, that this means financial hardship for MCI,
...my heart bleeds....
which implies that some other company is going to be able to step in to this market and make money by supporting the send-safes of the world... get ready for the next phase of spam growth: the large, international corporation that owns several ISPs and is entirely willing to mix large pools of legitimate users, servers and networks in with their spam services.
Welcome to 1999, Aaron. Large ISPs have been doing this for years. Its one of the direct motives behind SPEWS. While emission-specific blocklists (SBL/XBL, SpamCop, etc.) track spam by source: SPEWS maintains a list of known spam sources and spam friendly hosts so that e-mail can be rejected from these problem sites. http://www.spews.org/ Track NANAE for a week and you'll see a litany of sob stories: "my IP is blocked by SPEWS, please delist me". No, your _hosting provider_ is listed by SPEWS. _They_ need to clean up their act.
You're going to NEED a real reputation system sometime soon, or spam WILL become even more unmanageable than it is today.
If you think IPV4 is bad, try IPV6. That's among the reasons I'm strongly promoting ASN/CIDR tracking: IPV6 aggregates that much more. Still, it's going to be problematic. There's the corrolary aspect that ever more traffic is going to be IP based. Currently it's email, web services, chat, and the like. Moving forward, it's going to be phone, IM, SMS, and more. What do you think the mean time between rings for, say, a high-score socioeconomic class areacode (202, 212, 415, 650, 312, 818) will be where long distance is free and Choicepoint household data floats freely on the Dark Net? And you _don't_ have realtime content/context data to filter in realtime audio. As for reputation systems: yeah, look at Senderbase / Iron Mountain. I see big things for them, particularly if they've got _any_ brains.
What if the complaints start originating from the MPAA or the US government or the IMF or spammers?
I don't particularly care where a complaint originates from. I _do_ care that the complaint is legitimate and valid.
Also fair. I wonder though who writes the spec on those two terms?
How about we judge based on community outrage ;-)
We're forcing them to evolve or die... what happens when that evolution involves buying a very large ISP? Do we shut off half the world or come up with a better plan?
Do you connect your sewer mains to your spigots or your drains, Aaron? That's really a no-brainer.
So yes: if a large portion of the Net turns black, well, it turns black more ways than one.
Your points about ASN/CIDR identification are good, but this last statement bothers me. Plus, IPs are a poor metric. As we all know they are ephemeral.
I've had this discussion in the past week. IP ownership is ephemeral, particularly in dialup / dynamic access. ASN assignment is markedly less so, though they also change over time. What's markedly _less_ ephemeral are private blocklists. Remember, the public DNSBLs are just the visible tip of the iceberg. Private, site-specific lists, and dark lists, passed among friends and associates, but not made generally public, are legion. I used one such, with 65K+ domains, as part of my web content filtering for a youth center tech lab. Getting on and off public DNSBLs is _relatively_ sane. The lists have to be accountable to their users, and not violate either stated policy or useful effects too greatly, or they lose following. Within a private site, both usage patterns and propriety mean that once on, getting off a list is at best iffy. The result is that network abuse leads to a continuous erosion of useful IPs. Rotating abusive activity through a large block merely accelerates that process *and* provides a useful behavioral pattern identifying black-hat providers.
We *NEED* a reputation system that's based on strong encryption.
Encryption without authentication _and_ reputation data is useless. The response time of such a system may be infeasible. And systems with weak authentication may work well: honeypots, web content, and other abuse monitors identifying specific IPs of interest. A watch that extends to a sensible range (say, a /24 initially, broader based on other heuristics). Local management of such lists means that defenses are continuously adapted to conditions. Hardwired whitelists for criticial contacts to avoid friendly fire losses. Hard-fail conditions to provide feedback to those who are inadvertantly blocked. It's not an ideal system. It's probably a 95% solution. And it doesn't rely on encryption at all. Sure: encryption and PKI are useful. I use and advocate their use. Uptake remains limited. And in large-volume processing, it's expensive. That said: opportunistic authentication _should_ be used and favored where it exists.
We have all of the tools (SMTP/TLS + CAs + DNS), we just need a single protocol that at least a few large players agree to use that connects them. Once that's in place, we could END spam.
Yeah, sure. You want to split the profits? Since when do you trust CAs to identify nonspammers? What's this protocol you're talking about? How does it extend past email to _other_ forms of abuse? I see a handwave and "Profit!". I don't see a workable system yet. Care to expand?
Sure, there would be trade in stolen keys, but a decent system would adapt (via the same mechanisms as DNSBLs) and repair quickly enough that using a key would result in a return on investment insufficient to warrant the effort in stealing it.
Y'know, key repudiation _remains_ one of the weak points of current PKI. And the number of SSL sites lacking valid, authoritative, or other than non-self-signed keys is large. Plus the whole level of intermediation through browsers leads to _many_ user education issues.
New players who want to establish a rep would have to be "introduced" or suffer most of their mail being black-holed until a sufficient reputation had been established (perhaps years)
How long does it take you to make friends? How long does it take you to _really_ trust someone? If there's such a reputation system, I'd like it to offer two features: - Love at first sight: the ability to decide, quickly, that yes, you pretty much do trust someone rather highly. - Disowning: Ok, so you've changed your mind (or have sufficient experience that you really ought to) about this relationship. Can you get a divorce? When I designed the K5 moderation and mojo systems, I built in two features: - Moderation should quickly reflect initial experiences. The first moderation counts most. Subsequent mods _can_ change the overall score, but it takes a lot of 'em to do much shifting. After a time, the message moderation tends to stabilize. - Mojo should weight recent activity. If someone's been long-term trustworthy, but suddenly gets all wiggly, three-year-old rep _really_ shouldn't count that much.
but let's face it: if you're deploying your own mail server then one of the following is going to be true:
* Your up-stream ISP approves and will vouch for you as a customer who is under certain contractual obligations (not a guarantee of validity, but a starting point) * You know someone who trusts you and is willing to sign your key (risking their own reputation). * You are sending to a fair number of specific individuals who will white-list you, slowly building your reputation.
Under the scheme I envision, your relationship _largely_ is of concern to those with whom you're doing business. Basically: behave yourself the first time you come around. Eventually, you'll be considered a member of the family. Or at least given a neighborly greeting. Fortunately, most spammers currently exhibit such flagrant disregard for _any_ social niceties that you can write them off quickly.
Built properly, such a system would account for differing "opinions" (e.g. many small businessmen in Russia seem to trust each other, even though the rest of the world thinks they're evil... that's fine, they have their view, we have ours). This is easily accomplished by doing roughly what DNSBLs do: centering each origin of trust on a domain name, and allowing any domain name to originate a new root of trust at any time.
Sounds complicated and my brain's not wrapping around this well ATM. There is the fundamental problem of trust transitivity. OTOH, if you use a weighting system in which you simply _correlate_ others' opinions with your own, you don't have to worry about agreement, you only need concern yourself with predictive value. That chap with whom you have _no_ common tastes is a perfect predictor: corrolation is -1. The useless data is the person with whom you're aligned half the time, corrolation 0. Authentication here is useful in that you're reasonably assured that the reputation is the one you think it is. Beyond that, you don't really need to trust it.
Nice things about this system are: the SPF problem goes away. You don't care who initially started the chain, only who delivered it to your front door. They have the power to vouch for bad-guys, but it will nuke their rep. Your own relays would have a level of trust that puts your central delivery host into a slave mode, so no reputation updates would be performed (preventing your central MTA from harming your relay's reputation when it passes you spam, and also reducing overhead).
Right. I think I get at this somewhat more trivially with a proportionate rejection scheme: - Accept the first mail from a new source. - Rank that mail for spamminess. - Rinse, wash, repeat for some reasonable sample, over some aggregated netblock. You're going to find you're getting some proportion of spam vs. ham for that block. - Toss in curves for behavior: is the peer doing a dictionary attack on your space? Do they ignore temporary rejects, or not follow traditional retry intervals? Score goes down. - Based on the ratio of spam/ham coming from the site, at whatever granularity gives you a good statistical sample, say 10-30 datapoints per defined time period, where a week to a month should be more than sufficient, issue your own randomly assigned accept / reject responses. - Season to taste.
This might be rocket science, but it's not HARD rocket science.
To the contrary, it's cooking with spam.
Peace.
--
Karsten M. Self