Big Brother takes a controlling interest in chips

[Eugen Leitl]

(oldie but goodie -- maybe a dupe)

http://www.guardian.co.uk/technology/2006/jun/29/guardianweeklytechnologysection5

Big Brother takes a controlling interest in chips

A chilling novel details how everyday technologies could gradually lead to a
far more invasive society than even Orwell dreamed of

    Wendy Grossman

    The Guardian, Thursday 29 June 2006

Science fiction writers love to ask "What if?" What if a super-intelligent
alien race had planted a pair of devices to boost our development at key
moments? What if new technologies such as television and electronics become
pervasive? George Orwell's answer to the latter, in 1948's 1984, was to show
the apparently perfect, controlled society they could enable.

But one thing usually missing is the "how?" At the Computers, Freedom, and
Privacy (CFP) conference, held in Washington, DC, last month, Vernor Vinge, a
retired computer scientist and the author of Rainbows End, provided a
compelling explanation of how developing technology and powerful interests
could create a society far more invasive and controlled than anything Orwell
dreamed of.

The scenario he describes is the background he researched for Rainbows End.
Set in 2025, the characters are surrounded by logical extensions of today's
developing technology. Wearable computing is commonplace. Tagging and
ubiquitous networked sensors mean you can look at the landscape with your
choice of overlay and detail. People send each other silent messages and
Google for information within conversations with participants who may be
physically present or might be remote projections. One character's projection
is hijacked and becomes the front for three people. The owner of another
remote intelligence is unknown. Several continents' top intelligence
operatives try to solve a smart biological attack that infects a test
population with the willingness to obey orders.

Assumptions

Vinge makes two opening assumptions: no grand physical disaster occurs, and
today's computing and communications trends continue.

He added a third trend: "The great conspiracy against human freedom." As
novelist Doris Lessing has observed, barons on opposite sides of the river
don't need to be in cahoots if their interests coincide. In our case,
defence, homeland security, financial crime enforcement, police, tax
collectors and intellectual property rights holders offer reasons to want to
control the hardware we use. Then there are geeks, who can be tempted to
forget the consequences if the technology is cool enough. Vinge quotes the
most famous line from the comic strip Pogo: "We have met the enemy, and he is
us."

Vinge's technology to satisfy these groups' dreams is the Secure Hardware
Environment (She), which dedicates some bandwidth and a small portion of
every semiconductor for regulatory use. Deployment is progressive, as
standards are implemented. Built into new chips, She will spread inevitably
through its predecessors' obsolescence.

This part is terribly plausible. It sounds much like the Trusted Computing
Platform, implemented in Intel chips and built into machines from Dell,
Fujitsu-Siemens and others. Most people don't realise their new computer
contains a chip designed to block the operation of any software not certified
by the group. Now enhance that and build it into RFID chips, networked
embedded systems, shrink and distribute as "smart dust". All are current
trends or works in progress.

Geeks are willing to fight Trusted Computing on the grounds that it could be
used to block open-source software or to enforce draconian digital rights
management. But what if accepting it meant less visible security, less
bureaucracy, even slight profit? She automatically sends taxes, enables much
less noticeable surveillance and gets you through security checkpoints with
no waiting. There's less crime, because legislative reality can be enforced
on physical reality. Fewer false convictions. Make regulation automatic, and
it seems to go away. New laws can be downloaded as a regulatory upgrade.

"She," Vinge concluded at the conference, "fits the trajectory that economics
and technical progress are following. The infrastructure for such control
will probably arrive in any case." He also calls his scenario optimistic.

This is, however, one of the paths to Technological Singularity, which Vinge
wrote up in a 1993 essay. It's the moment superhuman intelligence is
achieved, either through systems we build or human augmentation. The
Singularity changes everything; nothing after it is predictable. In the world
of Rainbows End, he thinks, they are either on the verge of Singularity or it
is happening. This caused its own singularity in science fiction. Charlie
Stross, whose novel Accelerando attempts to depict living through the
Singularity, has called it "the turd in the punchbowl of near-future SF - you
may politely pretend it isn't there, but everyone has to deal with it."

Author Ken MacLeod places the Singularity in the context of post-2001
hopelessness. "When human beings feel they can't change the future, they
begin to imagine that maybe superhuman beings can: gods, angels, aliens - and
now artificial intelligences (AI). The idea of the Singularity is just a
sophisticated version of this ancient ... superstition, that human history is
or soon will be made by something other and better than human beings."

Way to believe

Vinge doesn't dispute the notion that humans look for a way to believe things
will be better. Year 2000 software remediation is an example: "Prudent
apprehension caused an awful lot of money to be spent ... and was one reason
there were no significant problems."

Most people assume the cause of the Singularity, if it happens, will be the
effect of Moore's Law (that the number of transistors that can fit in a given
space on a semiconductor doubles every 18 months). It's also usually
connected to AI. In his essay, however, Vinge suggested four or five paths to
the Singularity, only one of which was unitary AI.

But here is where Vinge's thinking gets optimistic (unlike the CFP
conference, which saw surveillant databases everywhere). The road to
technological Gaia is full of frictional costs that could stop it.

"It's not going to work very well," he says, "but it will be attempted, both
by the state and by civil special interest petitioners." The drug laws
provide a perfect illustration: "The ideal job to have in government is
something everybody is convinced is essential to be done successfully that
cannot be done successfully and the government is the only entity that can do
it. Every time you fail, you say, 'The problem is much larger than we
imagined, give us some more money.'"

"The leaders of most powerful countries are coming to realise that the most
important natural resources are not factories or the size of armies. Economic
power is in the size of the population that is well-educated, creative and
generally happy enough to be optimistic enough to want to do something
creative."

"The illusion of freedom becomes a strange thing when a government is dealing
with ... thousands of people who are as bright as the smartest people running
the government. Together, they outclass the people running the show. The
turning point is the notion that to provide this illusion of freedom for such
a group would wind up being more like real freedom than anything in human
history." Or, as he thinks Pogo might say for the 21st century, "We have met
what's going to save our ass and it is us."

7 If you'd like to comment on any aspect of Technology Guardian, send your
emails to tech at guardian.co.uk