Archives For Complexity

Complexity, what is, how do we deal with it, and how does it contribute to risk.

Mars code: JPL and risk based design

Linguistic security, and the second great crisis of computing

Distributed systems need to communicate, or talk, through some sort of communications channel in order to achieve coordinated behaviour which introduces the need for components to firstly recognise the difference between valid and invalid messages and secondly to have a common set of expectation of behaviour. And fairly obviously these two problems of coordination have safety and security implications of course.

The problem is that up to now security has been framed in the context of code, but this approach fails to realise that recognition and context are essentially language problems, which brings us firstly to the work of Chomsky on languages and next to Turing on computation. As it turns out above a certain level of expressive power of a language in the Chomsky hierarchy figuring out whether an input is valid runs into the halting problem of Turing. For such expressively powerful languages the question, ‘is it valid?’ is simply undecidable, no matter how hard you try. This is an important point, it’s not just hard or even really really hard to do but actually undecidable so…don’t go there.

Enter the study of linguistic security to address the vulnerabilities introduced by the to date unrecognised expressive power of the languages we communicate with.

Continue Reading…

20140122-072236.jpg

The failure of NVP and the likelihood of correlated security exploits

In 1986, John Knight & Nancy Leveson conducted an experiment to empirically test the assumption of independence in N version programming. What they found was that the hypothesis of independence of failures in N-version programs could be rejected at a 99% confidence level. While their results caused quite a stir in the software community, see their A reply to the critics for a flavour, what’s of interest to me is what they found when they took a closer look at the software faults.

…approximately one half of the total software faults found involved two or more programs. This is surprisingly high and implies that either programmers make a large number of similar faults or, alternatively, that the common faults are more likely to remain after debugging and testing.

Knight, Leveson 1986

Continue Reading…

The kettle of doom

20/12/2013 — 1 Comment

My thanks to Charlie Stross for alerting us all to the unfortunate incident of the Russian kettle, bugged with malware intended to find unsecured Wi-fi networks and co-opt them into a zombie bot net (1).

Now Charlie’s take on this revolves around the security/privacy implications for the ‘Internet of Things’ movement, making everything smart and web savvy may sound really cool, but not if your toaster ends up spying on you, a creepy little fore-taste of the panopticon future.

Continue Reading…

Toyota ECM (Image source: Barr testimony presentation)

Economy of mechanism and fail safe defaults

I’ve just finished reading the testimony of Phil Koopman and Michael Barr given for the Toyota un-commanded acceleration lawsuit. Toyota settled after they were found guilty of acting with reckless disregard, but before the jury came back with their decision on punitive damages, and I’m not surprised.

Continue Reading…

Singularity (Image source:  Tecnoscience)

Or ‘On the breakdown of Bayesian techniques in the presence of knowledge singularities’

One of the abiding problems of safety critical ‘first of’ systems is that you face, as David Collingridge observed, a double bind dilemma:

  1. Initially an information problem because ‘real’ safety issues (hazards) and their risk cannot be easily identified or quantified until the system is deployed, but 
  2. By the time the system is deployed you now face a power (inertia) problem, that is control or change is difficult once the system is deployed or delivered. Eliminating a hazard is usually very difficult and we can only mitigate them in some fashion.

    Continue Reading…

BMW HUD concept (Image source: BMW) Those who cannot remember the past of human factors are doomed to repeat it…

With apologies to the philosopher George Santayana, I’ll make the point that the BMW Head Up Display technology is in fact not the unalloyed blessing premised by BMW in their marketing material.

Continue Reading…

New Battery boxes (Image source: Boeing)

The end of the matter…well almost

Continue Reading…

No, not the alternative name for this blog. :)

I’ve just given the post Pitch ladders and unusual attitude a solid rewrite adding some new material and looking a little more deeply at some of the underlying safety myths.

X-Ray of JAL Battery (Image Source: NTSB)

A bit more on Boeing’s battery woes…

The NTSB has released more pictures of the JAL battery, and there are some interesting conclusions that can be drawn from the evidence to date.

Continue Reading…