[drone-list] US DoD Directive on Autonomy
gfoster at entersection.org
Mon Nov 26 07:26:21 PST 2012
Secrecy News (Nov 26) - "Autonomy in Weapon Systems":
One note. Mention is made in this document about the importance of
"verification and validation" (V&V) of systems. Earlier this month,
DARPA's High-Assurance Cyber Military Systems (HACMS) program awarded a
4.5 year $18M contract to an international consortium "to develop a
complete, formally proven architecture to protect the control and
communication systems of an aerial vehicle from compromise by faults and
Part of that award went to National ICT Australia [ @NICTA ] which toots
its own horn for formally verifying 7,500 lines of C code. That tells you
something about the challenge of "formal verification" which
mathematically proves a system will behave as specified - a much higher
bar than "rigorous" testing.
That also tells you something about the complexity of systems which have
thus far achieved formal verification. For comparison, the Linux
kernel---which sits at the heart of many of the workstations used by the
US military's remote pilots---surpassed 15 million lines of code in
January, leading the creator of Linux, Linus Torvalds, to lament that he
is "afraid of the day" when there will be an error that "cannot be
Considering DARPA knows the path to formal verification of larger systems
must be cleared - demand it. The challenge of formal verification presents
a strong argument for delaying deployment of lethal autonomous systems.
Gregory Foster || gfoster at entersection.org
@gregoryfoster <> http://entersection.com/
drone-list mailing list
drone-list at lists.stanford.edu
Should you need to change your subscription options, please go to:
If you would like to receive a daily digest, click "yes" (once you click above) next to "would you like to receive list mail batched in a daily digest?"
You will need the user name and password you receive from the list moderator in monthly reminders.
Should you need immediate assistance, please contact the list moderator.
----- End forwarded message -----
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
More information about the cypherpunks-legacy