Who's in Big Brother's Database?

[Eugen Leitl]

http://www.nybooks.com/articles/23231

Volume 56, Number 17 B7 November 5, 2009

Who's in Big Brother's Database?

By James Bamford

The Secret Sentry: The Untold History of the National Security Agency

by Matthew M. Aid

Bloomsbury, 423 pp., $30.00

On a remote edge of Utah's dry and arid high desert, where temperatures often
zoom past 100 degrees, hard-hatted construction workers with top-secret
clearances are preparing to build what may become America's equivalent of
Jorge Luis Borges's "Library of Babel," a place where the collection of
information is both infinite and at the same time monstrous, where the entire
world's knowledge is stored, but not a single word is understood. At a
million square feet, the mammoth $2 billion structure will be one-third
larger than the US Capitol and will use the same amount of energy as every
house in Salt Lake City combined.

Unlike Borges's "labyrinth of letters," this library expects few visitors.
It's being built by the ultra-secret National Security Agencybwhich is
primarily responsible for "signals intelligence," the collection and analysis
of various forms of communicationbto house trillions of phone calls, e-mail
messages, and data trails: Web searches, parking receipts, bookstore visits,
and other digital "pocket litter." Lacking adequate space and power at its
city-sized Fort Meade, Maryland, headquarters, the NSA is also completing
work on another data archive, this one in San Antonio, Texas, which will be
nearly the size of the Alamodome.

Just how much information will be stored in these windowless cybertemples? A
clue comes from a recent report prepared by the MITRE Corporation, a Pentagon
think tank. "As the sensors associated with the various surveillance missions
improve," says the report, referring to a variety of technical collection
methods, "the data volumes are increasing with a projection that sensor data
volume could potentially increase to the level of Yottabytes (1024 Bytes) by
2015."[1] Roughly equal to about a septillion
(1,000,000,000,000,000,000,000,000) pages of text, numbers beyond Yottabytes
haven't yet been named. Once vacuumed up and stored in these near-infinite
"libraries," the data are then analyzed by powerful infoweapons,
supercomputers running complex algorithmic programs, to determine who among
us may bebor may one day becomeba terrorist. In the NSA's world of automated
surveillance on steroids, every bit has a history and every keystroke tells a
story.

In the near decade since September 11, the tectonic plates beneath the
American intelligence community have undergone a seismic shift, knocking the
director of the CIA from the top of the organizational chart and replacing
him with the new director of national intelligence, a desk-bound espiocrat
with a large staff but little else. Not only surviving the earthquake but
emerging as the most powerful chief the spy world has ever known was the
director of the NSA. He is in charge of an organization three times the size
of the CIA and empowered in 2008 by Congress to spy on Americans to an
unprecedented degree, despite public criticism of the Bush administration's
use of the agency to conduct warrantless domestic surveillance as part of the
"war on terror." The legislation also largely freed him of the nettlesome
Foreign Intelligence Surveillance Court (FISA). And in another significant
move, he was recently named to head the new Cyber Command, which also places
him in charge of the nation's growing force of cyber warriors.

Wasting no time, the agency has launched a building boom, doubling the size
of its headquarters, expanding its listening posts, and constructing enormous
data factories. One clue to the possible purpose of the highly secret
megacenters comes from the agency's British partner, Government
Communications Headquarters. Last year, the British government proposed the
creation of an enormous government-run central database to store details on
every phone call, e-mail, and Internet search made in the United Kingdom.
Click a "send" key or push an "answer" button and the details of the
communication end up, perhaps forever, in the government's data warehouse to
be scrutinized and analyzed.

But when the plans were released by the UK government, there was an immediate
outcry from both the press and the public, leading to the scrapping of the
"big brother database," as it was called. In its place, however, the
government came up with a new plan. Instead of one vast, centralized
database, the telecom companies and Internet service providers would be
required to maintain records of all details about people's phone, e-mail, and
Web-browsing habits for a year and to permit the government access to them
when asked. That has led again to public anger and to a protest by the London
Internet Exchange, which represents more than 330 telecommunications firms.
"We view...the volume of data the government now proposes [we] should collect
and retain will be unprecedented, as is the overall level of intrusion into
the privacy of citizenry," the group said in August.[2]

Unlike the British government, which, to its great credit, allowed public
debate on the idea of a central data bank, the NSA obtained the full
cooperation of much of the American telecom industry in utmost secrecy after
September 11. For example, the agency built secret rooms in AT&T's major
switching facilities where duplicate copies of all data are diverted,
screened for key names and words by computers, and then transmitted on to the
agency for analysis. Thus, these new centers in Utah, Texas, and possibly
elsewhere will likely become the centralized repositories for the data
intercepted by the NSA in America's version of the "big brother database"
rejected by the British.

Matthew M. Aid has been after the NSA's secrets for a very long time. As a
sergeant and Russian linguist in the NSA's Air Force branch, he was arrested
and convicted in a court-martial, thrown into prison, and slapped with a bad
conduct discharge for impersonating an officer and making off with a stash of
NSA documents stamped Top Secret Codeword. He now prefers to obtain the NSA's
secrets legally, through the front door of the National Archives. The result
is The Secret Sentry: The Untold History of the National Security Agency , a
footnote-heavy history told largely through declassified but heavily redacted
NSA reports that have been slowly trickling out of the agency over the years.
They are most informative in the World War II period but quickly taper off in
substance during the cold war.

Aid begins his study on the eve of Pearl Harbor, a time when the entire
American cryptologic force could fit into a small, half-empty community
theater. But by war's end, it would take a football stadium to seat the
37,000 military and civilian "crippies." On August 14, 1945, as the ink dried
on Japan's instruments of surrender, the linguists and codebreakers manning
the thirty-seven key listening posts around the world were reading more than
three hundred diplomatic code and cipher systems belonging to sixty
countries. "The American signals intelligence empire stood at the zenith of
its power and prestige," notes Aid. But within days, the cryptanalysts put
away their well-sharpened pencils and the intercept operators hung up their
earphones. By the end of December 1945, America's crypto world had shrunk to
7,500 men and women.

Despite the drastic layoffs, the small cadre of US and British codebreakers
excelled against the new "main enemy," as Russia became known. The joint
US-British effort deciphered tens of thousands of Russian army and navy
messages during the mid-to-late 1940s. But on October 29, 1948, as President
Truman was about to deliver a campaign speech in New York, the party was
over. In what became known within the crypto world as "Black Friday," the
Russian government and military flipped a switch and instantly converted to
new, virtually unbreakable encryption systems and from vulnerable radio
signals to buried cables. In the war between spies and machines, the spies
won. The Soviets had managed to recruit William Weisband, a forty-year-old
Russian linguist working for the US Army, who informed them of key
cryptologic weaknesses the Americans were successfully exploiting. It was a
blow from which the codebreakers would never recover. NSA historians called
it "perhaps the most significant intelligence loss in US history."

In the 1970s, when some modest gains were made in penetrating the Russian
systems, history would repeat itself and another American turncoat, this time
Ronald Pelton, would again give away the US secrets. Since then, it has
largely been a codemaker's market not only with regard to high-level Russian
ciphers, but also those of other key countries, such as China and North
Korea. On the other hand, the NSA has made significant progress against less
cryptologically sophisticated countries and, from them, gained insight into
plans and intentions of countries about which the US has greater concerns.
Thus, when a Chinese diplomat at the United Nations discusses some new
African venture with a colleague from Sudan, the eavesdroppers at the NSA may
be deaf to the Chinese communications links but they may be able to get that
same information by exploiting weaknesses in Sudan's communications and
cipher systems when the diplomat reports the meeting to Khartoum. But even
third-world cryptography can be daunting. During the entire war in Vietnam,
writes Aid, the agency was never able to break the high-level encryption
systems of either the North Vietnamese or the Vietcong. It is a revelation
that leads him to conclude "that everything we thought we knew about the role
of NSA in the Vietnam War needs to be reconsidered."

Because the book is structured chronologically, it is somewhat difficult to
decipher the agency's overall record. But one sees troubling trends. One
weakness that seems to recur is that the agency, set up in the wake of World
War II to prevent another surprise attack, is itself frequently surprised by
attacks and other serious threats. In the 1950s, as over 100,000 heavily
armed North Korean troops surged across the 38th parallel into South Korea,
the codebreakers were among the last to know. "The North Korean target was
ignored," says a declassified NSA report quoted by Aid. "North Korea got lost
in the shuffle and nobody told us that they we encamped in Cuba. Nor did it
detect the move of five complete medium-range and intermediate-range missile
regiments from their Russian bases to Cuba. And it had no knowledge that
Russian ballistic missiles were on Cuban soil, being positioned in launchers.
"Soviet communications security was almost perfect," according to an NSA
historian.

The first clues that something unusual was happening had come in mid-July
1962, when NSA analysts noticed record numbers of Soviet cargo and passenger
ships heading for Cuba. Analysis of their unencrypted shipping manifests led
the NSA to suspect that the ships were delivering weapons. But the
nuclear-armed ballistic missiles were not detected until mid-October, a month
after their arrival, and not by the NSA; it was the CIA, acting on
information from its sources in Cuba and Florida, that ordered the U-2
reconnaisance flight that photographed them at launch sites on the island.
"The crisis," Aid concludes, "was in fact anything but an intelligence
success story." This is a view shared by the agency itself in a candid
internal history, which noted that the harrowing events "marked the most
significant failure of SIGINT [signals intelligence] to warn national leaders
since World War II."

More recently, the NSA was unaware of India's impending nuclear test in 1998,
the 1993 attack on the World Trade Center, the attack on the USS Cole in
2000, and the 1998 bombing of two of America's East African embassies. The
agency first learned of the September 11 attacks on $300 television sets
tuned to CNN, not its billion-dollar eavesdropping satellites tuned to
al-Qaeda.

Then there is the pattern by which the NSA was actually right about a
warning, but those in power chose to ignore it. During the Korean War, the
AFSA picked up numerous indications from low-level unencrypted Chinese
intercepts that the Chinese were shifting hundreds of thousands of combat
troops to Manchuria by rail, an obvious signal that China might enter the
war. But those in charge of Army intelligence simply refused to believe it;
it didn't fit in with thetely by policymakers in Washington," and as the UN
troops began crossing the divide, Chinese troops crossed the Yalu River into
North Korea. Even when intercepts indicated that the Chinese were well
entrenched in the North, officials in Washington and Seoul remained in a
state of disbelief, until both South Korean and US forces there were attacked
by the Chinese forces.

The pattern was repeated in Vietnam when NSA reporting warned on January 25,
1968, that a major coordinated attack would occur "in the near future in
several areas of South Vietnam." But neither the White House, the CIA, nor
General William Westmoreland at US military headquarters in Saigon believed
it, until over 100,000 North Vietnamese and Vietcong troops launched their
Tet offensive in the South five days later on January 30. "The [NSA] reports
failed to shake the commands in Washington and Saigon from their perception,"
says an NSA history. Tragically, Aid notes, at the end of the war, all of the
heroic Vietnamese cryptologic personnel who greatly helped the NSA were left
behind. "Many," the NSA report reveals, "undoubtedly perished." It added,
"Their story is yet untold." Then again in 1973, as in Korea and Vietnam, the
NSA warned that Egypt and Syria were planning "a major offensive" against
Israel. But, as Aid quotes an official NSA history, the CIA refused to
believe that an attack was imminent "because [they thought] the Arabs
wouldn't be 'stupid enough' to attack Israel." They were, they did, and they
won.

Everything seemed to go right for the NSA during the Soviet invasion of
Afghanistan, which the agency had accurately forecast. "NSA predicted on
December 22 [1979], three full days before the first Soviet troops crossed
the SovietbAfghan border, that the Russians would invade Afghanistan within
the next seventy-two hours," writes Aid, adding, "Afghanistan may have been
the 'high water mark' for NSA."

The agency also recorded the words of the Russian fighter pilot and his
ground controllers as he shot down Korean Airlines Flight 007 in 1983.
Although the agencyhite House's selective release of the most salacious of
the NSA material concerning the shootdown set off a firestorm of criticism
inside NSA," writes Aid. It was not the first time, nor would it be the last,
that the NSA's product was used for political purposes.

The most troubling pattern, however, is that the NSA, through gross
incompetence, bad intelligence, or deliberate deception through the selective
release of information, has helped to push the US into tragic wars. A prime
example took place in 1964 when the Johnson administration claimed that two
US Navy destroyers in the Gulf of Tonkin, one on an eavesdropping mission for
the NSA, were twice attacked by North Vietnamese torpedo boats. Those attacks
were then used to justify the escalation of American involvement in the
Vietnam War. But Aid cites a top-secret NSA analysis of the incident,
completed in 2000, which concluded that the second attack, the one used to
justify the war, never took place. Instead, NSA officials deliberately
withheld 90 percent of the intelligence on the attacks and told the White
House only what it wanted to hear. According to the analysis, only
intelligence "that supported the claim that the communists had attacked the
two destroyers was given to administration officials."

Not having learned its lesson, in the lead-up to the war in Iraq the NSA
again told the administration only what it wanted to hear, despite the
clearly ambiguous nature of the evidence. For years beforehand, the agency's
coverage of Iraq was disastrous. In the late 1990s, the Iraqis began shifting
much of their high-level military communications from radio to buried fiber
optic networks, and at the same time, Saddam Hussein banned the use of cell
phones. That left only occasional low-level troop communications. According
to a later review, Aid writes, NSA had "virtually no useful signals
intelligence on a target that was one of the United States' top intelligence
priorities." And the little intelligence it did have pointed away from Iraq
possessing weapons of mass destruction. "We lookede's rudimentary
communications system, its leaders were forced to communicate only by
satellite phones, which were very susceptible to NSA monitoring.

Other NSA tactical teams, Aid explains, collaborated on the ground with
Special Forces units, including in the mountains of Tora Bora. But it was a
new type of war, one the NSA was not prepared for, and both Osama bin Laden
and Taliban leader Mullah Omar easily slipped through its electronic net.
Eight years later, despite billions of dollars spent by the agency and dozens
of tapes released by bin Laden, the NSA is no closer to capturing him or
Mullah Omar than it was at Tora Bora in 2001.

Disappointingly, the weakest section of the book, mostly summaries of old
news clips, deals with what may be the most important subject: the NSA's
warrantless eavesdropping and its targeting of American communications. There
is no discussion, for example, of the agency's huge data-mining centers,
mentioned above, currently being built in Utah and Texas, or to what extent
the agency, which has long been confined to foreign and international
communications, is now engaged in domestic eavesdropping.

It is a key question and we have no precise answer. By installing its
intercept rooms in such locations as AT&T's main switching station in
downtown San Francisco, the agency has physical access to domestic as well as
international communications. Thus it is possible that the agency scans all
the e-mail of both and it may also eavesdrop on the telephone calls of both
for targets on its ever-growing watch lists. According to a recent Justice
Department report, "As of December 31, 2008, the consolidated terrorist
watchlist contained more than 1.1 million known or suspected terrorist
identities."[3]

Aid's history becomes thin as it gets closer to the present day and the
archival documents dwindle, especially since he has no substantial
first-person, on-the-record interviews. Beyond a brief mention, he also
leaves other important aspects of the NSA's history unaddressed, including
the tumultuous years in the mid-1970sd hijackers, knowing they had been sent
by bin Laden, while they were in the US preparing for the attacks. The
terrorists even chose as their command center a motel in Laurel, Maryland,
almost within eyesight of the director's office. Yet the agency never once
sought an easy-to-obtain FISA warrant to pinpoint their locations, or even
informed the CIA or FBI of their presence.

But pulling the plug, or even allowing the lights to dim, seems unlikely
given President Obama's hawkish policies in Afghanistan. However, if the war
there turns out to be the train wreck many predict, then Obama may decide to
take a much closer look at the spy world's most lavish spender. It is a
prospect that has some in the Library of Babel very nervous. "It was a great
ride while it lasted," said one.  Notes

[1]The MITRE Corporation, "Data Analysis Challenges" (December 2008), p. 13.

[2]David Leppard, "Internet Firms Resist Ministers' Plan to Spy on Every
E-mail," The Sunday Times , August 2, 2009.

[3]"The Federal Bureau of Investigation's Terrorist Watchlist Nomination
Practices," US Department of Justice, Office of the Inspector General, Audit
Division, Audit Report 09-25, May 2009.