On Fri, Jul 4, 2014, at 04:38 PM, Nathan Andrew Fain wrote:
Based on the xkeyscore rules does anyone have some idea of the technology being utilized?
Looking at the mapreduce::plugin definition I get the impression Hadoop is in use. Hadoop provides a stream interface for Map Reduce functions letting one utilize any program or language of their choosing [1-example]. Can with more knowledge of distributed data technologies confirm this?
It's been known for a while that the NSA are using Hadoop (June 9, 2013)[1]: "The NSA's advances have come in the form of programs developed on the West Coast—a central one was known by the quirky name Hadoop—that enable intelligence agencies to cheaply amplify computing power, U.S. and industry officials said." Also, from the Hadoop 2014 speaker lineup [2]: "Joey Echeverria is Cloudera`s Chief Architect for Public Sector where he coordinates with Cloudera`s Customers and Partners as well as Cloudera`s Product, Engineering, and Field teams to speed up the time it takes to move Hadoop applications to production. Previously Joey was a Principal Solutions Architect where he worked directly with customers to deploy production Hadoop clusters and solve a diverse range of business and technical problems. Joey joined Cloudera from the NSA where he worked on data mining, network security, and clustered data processing using Hadoop." Alfie [1] http://online.wsj.com/news/articles/SB10001424127887323495604578535290627442964?mg=reno64-wsj&url=http%3A%2F%2Fonline.wsj.com%2Farticle%2FSB10001424127887323495604578535290627442964.html [2] http://hadoopsummit.org/san-jose/speakers/ -- Alfie John alfiej@fastmail.fm