On Wed, Jul 07, 2004 at 10:28:01AM -0400, Tyler Durden wrote:
Well, I don't actually believe it's all recorded. As I've attempted to explain previously, "they" almost certainly have risk models in place. When several variables twinkle enough (eg, origination area, IP address, presence of crypto...) some rule fires and then diverts a copy into the WASP'S Nest. There's probably some kind of key word search that either diverts the copy into storage or into the short list for an analyst to peek it.
How much plain text can ~10^9 online monkeys daily enter into their keyboard? A ~10^3 average ballpark gives you a TByte/day (minus the redundancy), which is currently a 1U worth of SATA RAID/day, or 3 years worth of world's entire traffic in a 10^3 node cluster, which is on the low side these days. Hard drive storage density goes up exponentially, and probably faster than people can go online (the old world has saturated) -- it isn't a problem, given that population increase doesn't occur at these growth rates. You don't have to delete anything, ever. Given what Google manages with some 10^4..10^5 nodes, this problem set looks puny in comparison. Keeping the data on a cluster gives you the local crunch to do some very nontrivial data mining, especially if you narrow the scope down sufficiently to be able to lock the data in memory and crunch it there. Fax OCR/telex is just as easy, speech recognition doable, given the budget. We don't know whether they are actually doing it (I *think* these people are too conservative to be doing clusters right now, so they're probably doing storage hierarchies with tape libraries -- but then they as well could be MIB types years ahead of the mainstream), the point it is that they could, given the documented amount of hired talent and official budget. -- Eugen* Leitl <a href="http://leitl.org">leitl</a> ______________________________________________________________ ICBM: 48.07078, 11.61144 http://www.leitl.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE http://moleculardevices.org http://nanomachines.net [demime 1.01d removed an attachment of type application/pgp-signature]