CCTV biometric surveillance software fails German reliability test

[Martin Virtel]

German federal police enrolled 200 commuters to test if they could use face
recognition software to pick out suspects from a CCTV feed at a train
station under real-world circumstances. The three systems tested (produced
by Cognitec, Bosch and Cross Match) failed to recognize 8 out of 10 people
they should have, even when they were fed images of people standing still on
an escalator, one of the favourite settings for this kind of biometrics.
The key factor was the bad lighting conditions in the morning and afternoon,
when most of the test suspects passed the cameras. (The test suspects were
also fitted with RFID tags so they could be reliably identified by the test
setting). Under the right conditions, the systems failed to recognize 4 out
of 10 people, at a rate of 0.1 per cent of false alarms, which the
researchers thought acceptable for practical police work.

The final report [German, link below] recommends against using the systems
for identification purposes. They would only be useful under constant
lighting conditions, and either openly seeking cooperation of the persons
being checked by the biometrics software, or making them cooperate
involuntarily, by using what the report calls "eye-catchers", like changing
billboards or marquees. The report states that three-dimensional face
recognition, currently being developed, could probably do better.

Although the report points out that the systems tested are basically not
usable yet, there is still a major flaw in the design: The researchers
thought 23 false alarms per day would be acceptable. If you have 23 false
alarms a day, and only one or two real suspects (probably hiding their faces
behind a newspaper) crossing the cameras per week, I think you would stop to
trust the system very soon.

The final report (28 pages, german) is available here:
http://www.bka.de/kriminalwissenschaften/fotofahndung/pdf/fotofahndung_abschlussbericht.pdf

Martin Virtel, Redakteur Forschen & Entwickeln Fon: +49/40/319 90 469
Financial Times Deutschland GmbH & Co KG, Stubbenhuk 3, 20459 Hamburg;
Amtsgericht Hamburg HRA 92810 http://www.ftd.de/forschung virtel.martin at ftd.de

------------------------------