"a skilled backdoor-writer can defeat skilled auditors"?

[James A. Donald]

On 2014-06-04 08:35, rysiek wrote:
> Hi there,
>
> in a different thread, Cam posted a link containing this gem:
>
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>
> In short several very skilled security auditors examined a small Python
> program — about 100 lines of code — into which three bugs had been inserted by
> the authors. There was an “easy,” “medium,” and “hard” backdoor. There were
> three or four teams of auditors.
>
> 1. One auditor found the “easy” and the “medium” ones in about 70 minutes, and
> then spent the rest of the day failing to find any other bugs.
>
> 2. One team of two auditors found the “easy” bug in about five hours, and
> spent the rest of the day failing to find any other bugs.
>
> 3. One auditor found the “easy” bug in about four hours, and then stopped.
>
> 4. One auditor either found no bugs or else was on a team with the third
> auditor — the report is unclear.
>
> See Chapter 7 of Yee’s report for these details.
>
> I should emphasize that that I personally consider these people to be
> extremely skilled. One possible conclusion that could be drawn from this
> experience is that a skilled backdoor-writer can defeat skilled auditors. This
> hypothesis holds that only accidental bugs can be reliably detected by
> auditors, not deliberately hidden bugs.
>
> Anyway, as far as I understand the bugs you folks left in were accidental bugs
> that you then deliberately didn’t-fix, rather than bugs that you intentionally
> made hard-to-spot.
>
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> https://blog.spideroak.com/20140220090004-responsibly-bringing-new-cryptography-product-market#footnote1
>
> I have no problem believing it is thus, but can't help wondering if there are
> any ways to mitigate it.
>


The underhanded C contest produced stuff that was pretty easy to detect. 
  Maybe Python supports more subtle bugs, or maybe the auditors sucked.