Another question: How much traffic they are monitoring with these definitions? All visible? Almost all except the u$a? I suspect the above will require quite large hardware. I suppose this is a matter of importance for the dear NSA. To paraphrase a Susan Sontag quote [1] ``Most people in this society who aren't actively terrorists are, at best, reformed or potential terrorists.'' [1] http://thinkexist.com/quotation/most_people_in_this_society_who_aren-t_activ... On Fri, Jul 04, 2014 at 04:38:20PM +0200, Nathan Andrew Fain wrote:
Based on the xkeyscore rules does anyone have some idea of the technology being utilized?
Looking at the mapreduce::plugin definition I get the impression Hadoop is in use. Hadoop provides a stream interface for Map Reduce functions letting one utilize any program or language of their choosing [1-example]. Can with more knowledge of distributed data technologies confirm this?
1. http://cs.smith.edu/dftwiki/index.php/Hadoop_Tutorial_2.2_--_Running_C++_Pro... see also slide 5: http://cecs.wright.edu/~tkprasad/courses/cs707/ProgrammingHadoop.pdf <quote> cat input | grep | sort | unique -c | cat > output Input | Map | Shuffle & Sort | Reduce | Output </quote>