Closed source more secure than open source

[Joseph Ashwood]

----- Original Message -----
From: "Anonymous" <nobody at remailer.privacy.at>

> Ross Anderson's paper at
> http://www.ftp.cl.cam.ac.uk/ftp/users/rja14/toulouse.pdf
> has been mostly discussed for what it says about the TCPA.  But the
> first part of the paper is equally interesting.

Ross Andseron's approximate statements:
Closed Source:
> "the system's failure rate has just
> dropped by a factor of L, just as we would expect."

Open Source:
bugs remain equally easy to find.

Anonymous's Statements:
>For most programs, source code will be of
> no benefit to external testers, because they don't know how to program.

> Therefore the rate at which (external) testers find bugs does not vary
> by a factor of L between the open and closed source methodologies,
> as assumed in the model.  In fact the rates will be approximately equal.

> The result is that once a product has gone into beta testing and then into
> field installations, the rate of finding bugs by authorized testers will
> be low, decreased by a factor of L, regardless of open or closed source.

I disagree, actually I agree and disagree with both, due in part to the
magnitudes involved. It is certainly true that once Beta testing (or some
semblance of it) begins there will be users that cannot make use of source
code, but what Anonymous fails to realize is that there will be beta testers
that can make use of the source code.

Additionally there are certain tendencies in the open and closed source
communities that Anonymous and Anderson have not addressed in their models.
The most important tendencies are that in closed source beta testing is
generally handed off to a separate division and the original author does
little if any testing, and in open source the authors have a much stronger
connection with the testing, with the authors' duty extending through the
entire testing cycle. These tendencies lead to two very different positions
than generally realized.

First, closed source testing, beginning in the late Alpha testing stage, is
generally done without any assistance from source code, by _anyone_, this
significantly hampers the testing. This has led to observed situations where
QA engineers sign off on products that don't even function, let alone have
close to 0 bugs. With the software engineers believing that because the code
was signed off, it must be bug-free. This is a rather substantial problem.
To address this problem one must actually correct the number of testers for
the ones that are effectively doing nothing. So while L is the extra
difficulty in finding bugs without source code, it is magnified by something
approximating (testers)/(testers not doing anything). It's worth noting that
(testers) > (testers not doing anything) causing the result K =
L*(testers)/(testers not doing anything), to tend towards infinite values.

In open source we have very much the opposite situation. The authors are
involved in all stages of testing, giving another value. This value is used
to adjust L as before, but the quantities involved are substantially
different. It must be observed, as was done by Anonymous, that there are
testers that have no concept what source code is, and certainly no idea how
to read it, call these harassers. In addition though there are also testers
who read source code, and even the authors themselves are doing testing,
call these coders. So in this case K = L*(harassers)/(harassers+coders).
Where it's worth noting that K will now tend towards 0.

It is also very much the case that different projects have different
quantities of testers. In fact as the number of beta testers grows, the
MTBD(iscovery) of a bug must not increase, and will almost certainly
decrease. In this case each project must be treated separately, since
obviously WindowsXP will have more people testing it (thanks to bug
reporting features) than QFighter3
(http://sourceforge.net/projects/qfighter3/ the lest active development on
sourceforge). This certainly leads to problems in comparison. It is also
worth noting that it is likely that actual difficulty in locating bugs is
probably related to the maximum of (K/testers) and the (testers root of K).
Meaning that WindowsXP is likely to have a higher ratio of bugs uncovered in
a given time period T than QFighter3. However due to the complexity of the
comparisons, QFighter3 is likely to have fewer bugs than WindowsXP, simply
because WindowsXP is several orders of magnitude more complex.

So while the belief that source code makes bug hunting easier on everyone,
is certainly not purely the case (Anonymous's observation), it is also not
the case that the tasks are equivalent (Anonymous's claim), with the
multiplier in closed source approaching infinite, and open source towards 0.
Additionally the quantity of testers appears to have more of an impact on
bug-finding than the discussion of open or closed source. However as always
complexity plays an enormous role in the number of bugs available to find,
anybody with a few days programming experience can write a bug free Hello
World program, but it takes significantly more effort to write something the
complexity of Windows or Linux bug-free. That is where open source receives
the biggest boost, where Microsoft has a limited number of software
engineers, and the Linux development process which has fewer testers but
many more developers. This will reduce the number of bugs simply due to the
effort put into the process by developers.

Which is better? That depends entirely on your situation.
                    Joe