Banford: Who's in Big Brother's Database?

[R.A. Hettinga]

<http://www.nybooks.com/articles/23231>

The New York Review of Books


Volume 56, Number 17 7 November 5, 2009
Who's in Big Brother's Database?
By James Bamford

The Secret Sentry: The Untold History of the National Security Agency
by Matthew M. Aid
Bloomsbury, 423 pp., $30.00

On a remote edge of Utah's dry and arid high desert, where
temperatures often zoom past 100 degrees, hard-hatted construction
workers with top-secret clearances are preparing to build what may
become America's equivalent of Jorge Luis Borges's "Library of Babel,"
a place where the collection of information is both infinite and at
the same time monstrous, where the entire world's knowledge is stored,
but not a single word is understood. At a million square feet, the
mammoth $2 billion structure will be one-third larger than the US
Capitol and will use the same amount of energy as every house in Salt
Lake City combined.

Unlike Borges's "labyrinth of letters," this library expects few
visitors. It's being built by the ultra-secret National Security Agency
which is primarily responsible for "signals intelligence," the
collection and analysis of various forms of communicationto house
trillions of phone calls, e-mail messages, and data trails: Web
searches, parking receipts, bookstore visits, and other digital
"pocket litter." Lacking adequate space and power at its city-sized
Fort Meade, Maryland, headquarters, the NSA is also completing work on
another data archive, this one in San Antonio, Texas, which will be
nearly the size of the Alamodome.

Just how much information will be stored in these windowless
cybertemples? A clue comes from a recent report prepared by the MITRE
Corporation, a Pentagon think tank. "As the sensors associated with
the various surveillance missions improve," says the report, referring
to a variety of technical collection methods, "the data volumes are
increasing with a projection that sensor data volume could potentially
increase to the level of Yottabytes (1024 Bytes) by 2015."[1] Roughly
equal to about a septillion (1,000,000,000,000,000,000,000,000) pages
of text, numbers beyond Yottabytes haven't yet been named. Once
vacuumed up and stored in these near-infinite "libraries," the data
are then analyzed by powerful infoweapons, supercomputers running
complex algorithmic programs, to determine who among us may beor may
one day becomea terrorist. In the NSA's world of automated
surveillance on steroids, every bit has a history and every keystroke
tells a story.


In the near decade since September 11, the tectonic plates beneath the
American intelligence community have undergone a seismic shift,
knocking the director of the CIA from the top of the organizational
chart and replacing him with the new director of national
intelligence, a desk-bound espiocrat with a large staff but little
else. Not only surviving the earthquake but emerging as the most
powerful chief the spy world has ever known was the director of the
NSA. He is in charge of an organization three times the size of the
CIA and empowered in 2008 by Congress to spy on Americans to an
unprecedented degree, despite public criticism of the Bush
administration's use of the agency to conduct warrantless domestic
surveillance as part of the "war on terror." The legislation also
largely freed him of the nettlesome Foreign Intelligence Surveillance
Court (FISA). And in another significant move, he was recently named
to head the new Cyber Command, which also places him in charge of the
nation's growing force of cyber warriors.

Wasting no time, the agency has launched a building boom, doubling the
size of its headquarters, expanding its listening posts, and
constructing enormous data factories. One clue to the possible purpose
of the highly secret megacenters comes from the agency's British
partner, Government Communications Headquarters. Last year, the
British government proposed the creation of an enormous government-run
central database to store details on every phone call, e-mail, and
Internet search made in the United Kingdom. Click a "send" key or push
an "answer" button and the details of the communication end up,
perhaps forever, in the government's data warehouse to be scrutinized
and analyzed.

But when the plans were released by the UK government, there was an
immediate outcry from both the press and the public, leading to the
scrapping of the "big brother database," as it was called. In its
place, however, the government came up with a new plan. Instead of one
vast, centralized database, the telecom companies and Internet service
providers would be required to maintain records of all details about
people's phone, e-mail, and Web-browsing habits for a year and to
permit the government access to them when asked. That has led again to
public anger and to a protest by the London Internet Exchange, which
represents more than 330 telecommunications firms. "We view...the
volume of data the government now proposes [we] should collect and
retain will be unprecedented, as is the overall level of intrusion
into the privacy of citizenry," the group said in August.[2]

Unlike the British government, which, to its great credit, allowed
public debate on the idea of a central data bank, the NSA obtained the
full cooperation of much of the American telecom industry in utmost
secrecy after September 11. For example, the agency built secret rooms
in AT&T's major switching facilities where duplicate copies of all
data are diverted, screened for key names and words by computers, and
then transmitted on to the agency for analysis. Thus, these new
centers in Utah, Texas, and possibly elsewhere will likely become the
centralized repositories for the data intercepted by the NSA in
America's version of the "big brother database" rejected by the British.

Matthew M. Aid has been after the NSA's secrets for a very long time.
As a sergeant and Russian linguist in the NSA's Air Force branch, he
was arrested and convicted in a court-martial, thrown into prison, and
slapped with a bad conduct discharge for impersonating an officer and
making off with a stash of NSA documents stamped Top Secret Codeword.
He now prefers to obtain the NSA's secrets legally, through the front
door of the National Archives. The result is The Secret Sentry: The
Untold History of the National Security Agency , a footnote-heavy
history told largely through declassified but heavily redacted NSA
reports that have been slowly trickling out of the agency over the
years. They are most informative in the World War II period but
quickly taper off in substance during the cold war.

Aid begins his study on the eve of Pearl Harbor, a time when the
entire American cryptologic force could fit into a small, half-empty
community theater. But by war's end, it would take a football stadium
to seat the 37,000 military and civilian "crippies." On August 14,
1945, as the ink dried on Japan's instruments of surrender, the
linguists and codebreakers manning the thirty-seven key listening
posts around the world were reading more than three hundred diplomatic
code and cipher systems belonging to sixty countries. "The American
signals intelligence empire stood at the zenith of its power and
prestige," notes Aid. But within days, the cryptanalysts put away
their well-sharpened pencils and the intercept operators hung up their
earphones. By the end of December 1945, America's crypto world had
shrunk to 7,500 men and women.

Despite the drastic layoffs, the small cadre of US and British
codebreakers excelled against the new "main enemy," as Russia became
known. The joint US-British effort deciphered tens of thousands of
Russian army and navy messages during the mid-to-late 1940s. But on
October 29, 1948, as President Truman was about to deliver a campaign
speech in New York, the party was over. In what became known within
the crypto world as "Black Friday," the Russian government and
military flipped a switch and instantly converted to new, virtually
unbreakable encryption systems and from vulnerable radio signals to
buried cables. In the war between spies and machines, the spies won.
The Soviets had managed to recruit William Weisband, a forty-year-old
Russian linguist working for the US Army, who informed them of key
cryptologic weaknesses the Americans were successfully exploiting. It
was a blow from which the codebreakers would never recover. NSA
historians called it "perhaps the most significant intelligence loss
in US history."

In the 1970s, when some modest gains were made in penetrating the
Russian systems, history would repeat itself and another American
turncoat, this time Ronald Pelton, would again give away the US
secrets. Since then, it has largely been a codemaker's market not only
with regard to high-level Russian ciphers, but also those of other key
countries, such as China and North Korea. On the other hand, the NSA
has made significant progress against less cryptologically
sophisticated countries and, from them, gained insight into plans and
intentions of countries about which the US has greater concerns. Thus,
when a Chinese diplomat at the United Nations discusses some new
African venture with a colleague from Sudan, the eavesdroppers at the
NSA may be deaf to the Chinese communications links but they may be
able to get that same information by exploiting weaknesses in Sudan's
communications and cipher systems when the diplomat reports the
meeting to Khartoum. But even third-world cryptography can be
daunting. During the entire war in Vietnam, writes Aid, the agency was
never able to break the high-level encryption systems of either the
North Vietnamese or the Vietcong. It is a revelation that leads him to
conclude "that everything we thought we knew about the role of NSA in
the Vietnam War needs to be reconsidered."

Because the book is structured chronologically, it is somewhat
difficult to decipher the agency's overall record. But one sees
troubling trends. One weakness that seems to recur is that the agency,
set up in the wake of World War II to prevent another surprise attack,
is itself frequently surprised by attacks and other serious threats.
In the 1950s, as over 100,000 heavily armed North Korean troops surged
across the 38th parallel into South Korea, the codebreakers were among
the last to know. "The North Korean target was ignored," says a
declassified NSA report quoted by Aid. "North Korea got lost in the
shuffle and nobody told us that they were interested in what was going
on north of the 38th parallel," exclaimed one intelligence officer. At
the time, astonishingly, the Armed Forces Security Agency (AFSA), the
NSA's predecessor, didn't even have a Korean-language dictionary.
Unfortunately for General Douglas MacArthur, the codebreakers were
able to read the communications of Spain's ambassador to Tokyo and
other diplomats, who noted that in their discussions with the general,
he made clear his secret hope for all-out war with China and Russia,
including the use of nuclear weapons if necessary. In a rare instance
of secret NSA intercepts playing a major part in US politics, once the
messages were shown to President Truman, MacArthur's career abruptly
ended.

Another major surprise came in the 1960s when the Soviet Union was
able to move large numbers of personnel, large amounts of equipment,
and many ballistic missiles to Cuba without the NSA hearing a peep.
Still unable to break into the high-level Soviet cipher systems, the
agency was unaware that the 51st Rocket Division had packed up and was
encamped in Cuba. Nor did it detect the move of five complete medium-
range and intermediate-range missile regiments from their Russian
bases to Cuba. And it had no knowledge that Russian ballistic missiles
were on Cuban soil, being positioned in launchers. "Soviet
communications security was almost perfect," according to an NSA
historian.

The first clues that something unusual was happening had come in mid-
July 1962, when NSA analysts noticed record numbers of Soviet cargo
and passenger ships heading for Cuba. Analysis of their unencrypted
shipping manifests led the NSA to suspect that the ships were
delivering weapons. But the nuclear-armed ballistic missiles were not
detected until mid-October, a month after their arrival, and not by
the NSA; it was the CIA, acting on information from its sources in
Cuba and Florida, that ordered the U-2 reconnaisance flight that
photographed them at launch sites on the island. "The crisis," Aid
concludes, "was in fact anything but an intelligence success story."
This is a view shared by the agency itself in a candid internal
history, which noted that the harrowing events "marked the most
significant failure of SIGINT [signals intelligence] to warn national
leaders since World War II."

More recently, the NSA was unaware of India's impending nuclear test
in 1998, the 1993 attack on the World Trade Center, the attack on the
USS Cole in 2000, and the 1998 bombing of two of America's East
African embassies. The agency first learned of the September 11
attacks on $300 television sets tuned to CNN, not its billion-dollar
eavesdropping satellites tuned to al-Qaeda.

Then there is the pattern by which the NSA was actually right about a
warning, but those in power chose to ignore it. During the Korean War,
the AFSA picked up numerous indications from low-level unencrypted
Chinese intercepts that the Chinese were shifting hundreds of
thousands of combat troops to Manchuria by rail, an obvious signal
that China might enter the war. But those in charge of Army
intelligence simply refused to believe it; it didn't fit in with their
plans.

Then, by reading the dispatches between India's well-connected
ambassador to Beijing and his Foreign Office, it became clear that
China would intervene if UN forces crossed the 38th parallel into
North Korea. But again, says Aid, the warning "was either discounted
or ignored completely by policymakers in Washington," and as the UN
troops began crossing the divide, Chinese troops crossed the Yalu
River into North Korea. Even when intercepts indicated that the
Chinese were well entrenched in the North, officials in Washington and
Seoul remained in a state of disbelief, until both South Korean and US
forces there were attacked by the Chinese forces.

The pattern was repeated in Vietnam when NSA reporting warned on
January 25, 1968, that a major coordinated attack would occur "in the
near future in several areas of South Vietnam." But neither the White
House, the CIA, nor General William Westmoreland at US military
headquarters in Saigon believed it, until over 100,000 North
Vietnamese and Vietcong troops launched their Tet offensive in the
South five days later on January 30. "The [NSA] reports failed to
shake the commands in Washington and Saigon from their perception,"
says an NSA history. Tragically, Aid notes, at the end of the war, all
of the heroic Vietnamese cryptologic personnel who greatly helped the
NSA were left behind. "Many," the NSA report reveals, "undoubtedly
perished." It added, "Their story is yet untold." Then again in 1973,
as in Korea and Vietnam, the NSA warned that Egypt and Syria were
planning "a major offensive" against Israel. But, as Aid quotes an
official NSA history, the CIA refused to believe that an attack was
imminent "because [they thought] the Arabs wouldn't be 'stupid enough'
to attack Israel." They were, they did, and they won.

Everything seemed to go right for the NSA during the Soviet invasion
of Afghanistan, which the agency had accurately forecast. "NSA
predicted on December 22 [1979], three full days before the first
Soviet troops crossed the SovietAfghan border, that the Russians
would invade Afghanistan within the next seventy-two hours," writes
Aid, adding, "Afghanistan may have been the 'high water mark' for NSA."

The agency also recorded the words of the Russian fighter pilot and
his ground controllers as he shot down Korean Airlines Flight 007 in
1983. Although the agency knew that the Russians had accidently
mistaken the plane for a potentially hostile US military aircraft, the
Reagan administration nevertheless deliberately spun the intercepts to
make it seem that the fighter pilot knew all along that it was a
passenger jet, infuriating NSA officials. "The White House's selective
release of the most salacious of the NSA material concerning the
shootdown set off a firestorm of criticism inside NSA," writes Aid. It
was not the first time, nor would it be the last, that the NSA's
product was used for political purposes.

The most troubling pattern, however, is that the NSA, through gross
incompetence, bad intelligence, or deliberate deception through the
selective release of information, has helped to push the US into
tragic wars. A prime example took place in 1964 when the Johnson
administration claimed that two US Navy destroyers in the Gulf of
Tonkin, one on an eavesdropping mission for the NSA, were twice
attacked by North Vietnamese torpedo boats. Those attacks were then
used to justify the escalation of American involvement in the Vietnam
War. But Aid cites a top-secret NSA analysis of the incident,
completed in 2000, which concluded that the second attack, the one
used to justify the war, never took place. Instead, NSA officials
deliberately withheld 90 percent of the intelligence on the attacks
and told the White House only what it wanted to hear. According to the
analysis, only intelligence "that supported the claim that the
communists had attacked the two destroyers was given to administration
officials."

Not having learned its lesson, in the lead-up to the war in Iraq the
NSA again told the administration only what it wanted to hear, despite
the clearly ambiguous nature of the evidence. For years beforehand,
the agency's coverage of Iraq was disastrous. In the late 1990s, the
Iraqis began shifting much of their high-level military communications
from radio to buried fiber optic networks, and at the same time,
Saddam Hussein banned the use of cell phones. That left only
occasional low-level troop communications. According to a later
review, Aid writes, NSA had "virtually no useful signals intelligence
on a target that was one of the United States' top intelligence
priorities." And the little intelligence it did have pointed away from
Iraq possessing weapons of mass destruction. "We looked long and hard
for any signs," said one retired NSA official. "We just never found a
'smoking gun' that Saddam was trying to build nukes or anything else."
That, however, did not prevent the NSA director, Lieutenant Gen.
Michael V. Hayden, from stamping his approval on the CIA's 2002
National Intelligence Estimate arguing that Iraq's WMDs posed a grave
danger, which helped prepare the way for the devastating war.

While much of the terrain Aid covers has been explored before, the
most original areas in The Secret Sentry deal with the ground wars in
Afghanistan and Iraq, where the NSA was forced to marry, largely
unsuccessfully, its super-high-tech strategic capabilities in space
with its tactical forces on the ground. Before the September 11
attacks, the agency's coverage of Afghanistan was even worse than that
of Iraq. At the start of the war, the NSA's principal listening post
for the region did not have a single linguist proficient in Pashto or
Dari, Afghanistan's two principal languages. Agency recruiters
descended on Fremont, California, home of the country's largest
population of Afghan expatriates, to build up a cadre of translators
only to have most candidates rejected by the agency's overparanoid
security experts. On the plus side, because of the collapse of the
Taliban regime's rudimentary communications system, its leaders were
forced to communicate only by satellite phones, which were very
susceptible to NSA monitoring.
Other NSA tactical teams, Aid explains, collaborated on the ground
with Special Forces units, including in the mountains of Tora Bora.
But it was a new type of war, one the NSA was not prepared for, and
both Osama bin Laden and Taliban leader Mullah Omar easily slipped
through its electronic net. Eight years later, despite billions of
dollars spent by the agency and dozens of tapes released by bin Laden,
the NSA is no closer to capturing him or Mullah Omar than it was at
Tora Bora in 2001.

Disappointingly, the weakest section of the book, mostly summaries of
old news clips, deals with what may be the most important subject: the
NSA's warrantless eavesdropping and its targeting of American
communications. There is no discussion, for example, of the agency's
huge data-mining centers, mentioned above, currently being built in
Utah and Texas, or to what extent the agency, which has long been
confined to foreign and international communications, is now engaged
in domestic eavesdropping.

It is a key question and we have no precise answer. By installing its
intercept rooms in such locations as AT&T's main switching station in
downtown San Francisco, the agency has physical access to domestic as
well as international communications. Thus it is possible that the
agency scans all the e-mail of both and it may also eavesdrop on the
telephone calls of both for targets on its ever-growing watch lists.
According to a recent Justice Department report, "As of December 31,
2008, the consolidated terrorist watchlist contained more than 1.1
million known or suspected terrorist identities."[3]

Aid's history becomes thin as it gets closer to the present day and
the archival documents dwindle, especially since he has no substantial
first-person, on-the-record interviews. Beyond a brief mention, he
also leaves other important aspects of the NSA's history unaddressed,
including the tumultuous years in the mid-1970s when it was
investigated by the Senate's Church Committee for decades of illegal
spying; Trailblazer, the nearly decade-long failure to modernize the
agency; and the NSA's increasingly important role in cyberwarfare and
its implications in future wars.

Where does all this leave us? Aid concludes that the biggest problem
facing the agency is not the fact that it's drowning in untranslated,
indecipherable, and mostly unusable data, problems that the troubled
new modernization plan, Turbulence, is supposed to eventually fix.
"These problems may, in fact, be the tip of the iceberg," he writes.
Instead, what the agency needs most, Aid says, is more power. But the
type of power to which he is referring is the kind that comes from
electrical substations, not statutes. "As strange as it may sound," he
writes, "one of the most urgent problems facing NSA is a severe
shortage of electrical power." With supercomputers measured by the
acre and estimated $70 million annual electricity bills for its
headquarters, the agency has begun browning out, which is the reason
for locating its new data centers in Utah and Texas. And as it pleads
for more money to construct newer and bigger power generators, Aid
notes, Congress is balking.

The issue is critical because at the NSA, electrical power is
political power. In its top-secret world, the coin of the realm is the
kilowatt. More electrical power ensures bigger data centers. Bigger
data centers, in turn, generate a need for more access to phone calls
and e-mail and, conversely, less privacy. The more data that comes in,
the more reports flow out. And the more reports that flow out, the
more political power for the agency.
Rather than give the NSA more money for more powerelectrical and
politicalsome have instead suggested just pulling the plug. "NSA can
point to things they have obtained that have been useful," Aid quotes
former senior State Department official Herbert Levin, a longtime
customer of the agency, "but whether they're worth the billions that
are spent, is a genuine question in my mind."

Based on the NSA's history of often being on the wrong end of a
surprise and a tendency to mistakenly get the country into, rather
than out of, wars, it seems to have a rather disastrous cost-benefit
ratio. Were it a corporation, it would likely have gone belly-up years
ago. The September 11 attacks are a case in point. For more than a
year and a half the NSA was eavesdropping on two of the lead
hijackers, knowing they had been sent by bin Laden, while they were in
the US preparing for the attacks. The terrorists even chose as their
command center a motel in Laurel, Maryland, almost within eyesight of
the director's office. Yet the agency never once sought an easy-to-
obtain FISA warrant to pinpoint their locations, or even informed the
CIA or FBI of their presence.

But pulling the plug, or even allowing the lights to dim, seems
unlikely given President Obama's hawkish policies in Afghanistan.
However, if the war there turns out to be the train wreck many
predict, then Obama may decide to take a much closer look at the spy
world's most lavish spender. It is a prospect that has some in the
Library of Babel very nervous. "It was a great ride while it lasted,"
said one.

Notes
[1]The MITRE Corporation, "Data Analysis Challenges" (December 2008),
p. 13.
[2]David Leppard, "Internet Firms Resist Ministers' Plan to Spy on
Every E-mail," The Sunday Times , August 2, 2009.
[3]"The Federal Bureau of Investigation's Terrorist Watchlist
Nomination Practices," US Department of Justice, Office of the
Inspector General, Audit Division, Audit Report 09-25, May 2009.