Interesting aspects of the metadata analysis [was Re: NYT: N.S.A. Gathers Data on Social Connections of U.S. Citizens]

coderman coderman at gmail.com
Mon Sep 30 10:31:58 PDT 2013


there are some interesting tidbits in this release:

"The agency had asked for the new power as early as 1999, the
documents show, but had been initially rebuffed because it was not
permitted under rules of the Foreign Intelligence Surveillance Court
that were intended to protect the privacy of Americans....
A 2009 draft of an N.S.A. inspector general’s report suggests that
contact chaining and analysis may have been done on Americans’
communications data under the Bush administration’s program of
wiretapping without warrants"

i find it interesting that this request occurred the same time that
core collection on US backbone links at Sprint was under development.

Sprint provided the perfect initial test as their ATM infrastructure,
and IPoATM data networks, were the most challenging environment for
deep packet inspection. (Solve it for Sprint, it can be applied
anywhere)



"...which began after the Sept. 11 attacks to detect terrorist
activities and skirted the existing laws governing electronic
surveillance."

it began earlier than this!



"A new policy that year [2008], detailed in “Defense Supplemental
Procedures Governing Communications Metadata Analysis,” authorized by
Defense Secretary Robert M. Gates and Attorney General Michael B.
Mukasey, said that since the Supreme Court had ruled that metadata was
not constitutionally protected, N.S.A. analysts could use such
information “without regard to the nationality or location of the
communicants,” according to an internal N.S.A. description of the
policy...
After that decision, which was previously reported by The Guardian,
the N.S.A. performed the social network graphing in a pilot project
for 1 ½ years “to great benefit,” according to the 2011 memo. It was
put in place in November 2010 in “Sigint Management Directive 424”
(sigint refers to signals intelligence)."

so this is when we see the floodgates really open, and the data deluge
turned up to 11...



"The N.S.A. documents show that one of the main tools used for
chaining phone numbers and e-mail addresses has the code name Mainway.
It is a repository into which vast amounts of data flow daily from the
agency’s fiber-optic cables, corporate partners and foreign computer
networks that have been hacked.

The documents show that significant amounts of information from the
United States go into Mainway. An internal N.S.A. bulletin, for
example, noted that in 2011 Mainway was taking in 700 million phone
records per day. In August 2011, it began receiving an additional 1.1
billion cellphone records daily from an unnamed American service
provider...

... the agency is pouring money and manpower into creating a metadata
repository capable of taking in 20 billion “record events” daily and
making them available to N.S.A. analysts within 60 minutes.

The spending includes support for the “Enterprise Knowledge System,”
which has a $394 million multiyear budget and is designed to “rapidly
discover and correlate complex relationships and patterns across
diverse data sources on a massive scale,” ...

A top-secret document titled “Better Person Centric Analysis”
describes how the agency looks for 94 “entity types,” including phone
numbers, e-mail addresses and IP addresses. In addition, the N.S.A.
correlates 164 “relationship types” to build social networks and what
the agency calls “community of interest” profiles, using queries like
“travelsWith, hasFather, sentForumMessage, employs.”

A 2009 PowerPoint presentation provided more examples of data sources
available in the “enrichment” process, including location-based
services like GPS and TomTom, online social networks, billing records
and bank codes for transactions in the United States and overseas."

and this is interesting in both scale and technical detail on the
metadata collected and how it is utilized in social network analysis.



keep 'em coming!




More information about the cypherpunks mailing list