[FoRK] Why We Are Losing The War on Terrorism

[J.Andrew Rogers]

On May 3, 2004, at 9:14 AM, Contempt for Meatheads forwarded:
>We desperately need adult supervision and high quality minds in the
>intelligence business!  I am growing more convince that the security
>clearance process, the government hiring/promotion process, and
>information silos are overwhelming our ability to get even a
>marginally adequate level of intelligence needed to fight terrorism. 
>Wow, this is depressing.
>
>My confident belief (100%):  we will continue to lose the war on
>terrorism until we fix our intelligence system.


I think this analysis is correct, but also a bit too shallow to be
really insightful.  While there are some significant institutional
problems and byzantine self-defeating regulations, these things are
masking a much bigger technical problem that desperately needs to be
tackled from their perspective.

The volume of data they collect has reached the point where good
analysis is no longer tractable in a theoretical algorithmic sense with
the best tools they currently have at their disposal, particularly when
you have a data space as broad and diffuse as "terrorism" to sift.
Institutional procedures and problems aggravate this, but the
underlying issues are deeper.

One of the ways I keep track of what the US DoD is up to is by analysis
of the open research programs, contracts, and grants that they publish.
 By threading the many, many programs together over time, you can see
how fast different technologies are progressing and you can chain
inferences to make an intelligent estimate as to when specific
capabilities (which may require the intersection of multiple research
tracks) could theoretically be available to the DoD.  Furthermore, the
program managers have a habit of mildly editorializing their program
descriptions in response to some of the proposals they have received
and the success of the proposals they have actually funded, which also
gives some added insight.

One thing that I have noticed for several years is that the advanced
data mining and automated intelligence analysis research programs have
been essentially stalled for many years now despite aggressive
marketing and a large number of agencies willing to liberally fund
proposals.  And the editorializing of the program managers on this
research track makes it clear that they are quite frustrated both with
the lack of progress in this area and with the fact that research
proposals keep trying to beat the same dead horse over and over.
Furthermore, while most programs have a shelf-life after which they are
either closed (both on good progress or no progress), these particular
programs keep getting extended and re-funded over and over, sometimes
under a different name but always with roughly the same parameters.

As long as this program track is stuck in neutral, the intelligence
agencies will have serious problems that will be all but
insurmountable.  The US intelligence service is a victim of its own
ability to acquire data.  This isn't a problem that they can simply
throw money at in the sense that it requires pretty substantial
algorithm breakthroughs to even be tractable for high-quality analysis.
 To date, private research organizations have clearly been unable to
solve this problem in any meaningful way, and there is substantial
evidence of this fact.  In the mean time, they are left using narrow
brittle algorithms to sift and analyze the data, with holes you could
drive a truck (bomb) through.

Someone who fully understood the theoretical limitations and likely
implementation parameters of the current state-of-the-art could likely
defeat the automated analysis.  Fortunately for the intelligence
agencies, few people have those skills and they get by on a pretty
broken system hampered further by institutional problems.

j. andrew rogers

_______________________________________________
FoRK mailing list
http://xent.com/mailman/listinfo/fork

----- End forwarded message -----
--
Eugen* Leitl <a href="http://leitl.org">leitl</a>
______________________________________________________________
ICBM: 48.07078, 11.61144            http://www.leitl.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
http://moleculardevices.org         http://nanomachines.net

[demime 1.01d removed an attachment of type application/pgp-signature]