The failure of NVP and the likelihood of correlated security exploits
In 1986, John Knight & Nancy Leveson conducted an experiment to empirically test the assumption of independence in N version programming. What they found was that the hypothesis of independence of failures in N-version programs could be rejected at a 99% confidence level. While their results caused quite a stir in the software community, see their A reply to the critics for a flavour, what’s of interest to me is what they found when they took a closer look at the software faults.
…approximately one half of the total software faults found involved two or more programs. This is surprisingly high and implies that either programmers make a large number of similar faults or, alternatively, that the common faults are more likely to remain after debugging and testing.
Knight, Leveson 1986
To summarise the experimental results Knight and Leveson found that while some common faults could be laid at the feet of what you might call difficult computations, most correlated failures occurred when the faulty paths had common input-domains.
We conclude that this occurs when the faulty paths have common input-domains. Correlated failures occur when the partial functions computed by the paths are identically wrong. The actual mistakes made, however, need not be similar or logically-related.
Brilliant, Knight & Leveson 1990
Now let’s consider system security at the point in a system where input-driven computation occurs, here a secure system should ideally parse an input for validity, and reject invalid inputs. Exploitations, unexpected input-driven computations due to maliciously crafted inputs, usually occur at this point as well, and rely on manipulating both latent design faults and regular features to subvert this parsing of inputs security function.
What the LK results imply is that a security exploits on the input domain will also correlate, that is an exploit of one system that utilises a specific set of inputs will likely work on another ‘like’ system, even though what actually breaks inside each will not be the same logical part, and even though the two systems appear to be designed quite differently. To put it simply, the existence of input domain correlated failures in software systems strongly implies the same correlation exists for security exploits regardless of design differences.
From a practical perspective this means that one should not naively rely on arguments of dissimilarity when considering the applicability of security exploits.