So what’s Highly Optimised Tolerance theory?
Highly Optimised Tolerance or HOT theory was developed as part of the study of complex biological and engineered systems, the theory emphasises that such systems are typified by highly structured, unique and self-dissimilar internal architectures that have evolved an complex optimally robust design within a set of resource constraints.
The proponents of HOT (Carlson & Doyle inter alia) argue that this complexity both delivers robustness but conversely also makes HOT systems vulnerable to minor perturbations which can have catastrophic consequences. As a result of this vulnerability HOT system cumulative frequency distributions of severity will exhibit heavy tail distributions (1). This heavy tail distribution is due to the inherent trade-off between addressing small losses for common events (i.e. robustness for a specified fault hypothesis), at the expense of large losses when subject to rare perturbations (when the specified fault hypothesis is violated).
Logically if HOT theory holds true then we should be able to see a change in the distribution of the severity of adverse events as the design moves from the, ‘just make it work’ stage to the ‘optimise for robustness’ stage. This is something we can actually test through observation of real world systems and accident statistics drawn from the US commercial aviation fleet provides us with an opportunity to do so. Note that by it’s nature this data includes a range of air carriers and aircraft types so we are really looking at the performance of the total US air transport system.
Zipf plotting the cumulative distributions of US air accidents in decadal cohorts allows us to determine whether the power laws relation posited by the HOT theory exists and to what degree this changes over time. From Figure 1. we can see away see a striking difference between the 1962-1972 data plot of Fig 1. and subsequent decades wit the 1962-1972 data plot decays rapidly at the 100+ mark while the other plots exhibit tails extending to the right (3).This shift to the right in the distribution tails can be explained by the growth in carrying capacity of aircraft across the period of the study which thereby establishes a greater asymptote for each of the curves (4).
What’s interesting is that the 62-72 data appears to have the strongest cutoff with the other decades exhibiting weaker ones. In order to provide an insight into overall improvement of robustness we can also plot the total number of accidents per decade (Fig. 2). As Fig. 2 shows the total number of accidents per decade has trended down significantly from the 1962 to 2006 (Fig. 2) (5).
Accident severity cannot behave as a pure power law as there is maximum upper size to all such events so, as the scaling factor approaches this cutoff value, the size of accidents must taper off. From just looking at the data it does appear that a truncated power law with an exponential cut-off exists and that this is true across the decadal cohorts. To rigorously check whether there is a difference between the distributions would require a formal estimation of the scaling factor, then defining where the scaling region finishes followed by checking the actual goodness of fit and, finally, comparing alternative distributions (such as Weibull, Log Normal) to see if they’re a better fit (6).
Some tentative conclusions
As the proponents of the theory (Carlson, Doyle 2002) point out the fundamental point is that a HOT system doesn’t exhibit Gaussian or Exponential distributions, to that extent the data as presented confirms this assertion. The question of whether aviation transport systems are inherently HOT or have evolved over time to become so appears to be contradicted by the data, although a proper statistical analysis should be performed to confirm this.
1. Although a power law relation is stated by HOT’s proponents, this is not seen as being a critical element of the HOT theory (Carlson, Doyle 2002), rather they emphasise that the tails should not be exponential or gaussian.
2. Severity of each accident is expressed by the total number of fatalities.
3. The 1962-1972 period saw the capacities in mixed class layouts reach 200+ (727) then climb past 400+ (747) in the early seventies.
4. Based on the worst credible aviation accident posited. For example the collision of two fully laden commercial aircraft, as occurred at Tenerife.
5. This neatly illustrates the problem we have with proving safety as the robustness of systems increases. As robustness increases, the amount of empirical data actually decreases.
6. For those of steely will Clauset et al (Clauset, Shalizi, Newman 2007) provide an excellent discussion of the techniques required.
Carlson, J.M., Doyle, J., Complexity and Robustness, Proc. of the National Academy of Sciences, 19 February, 2002, vol. 99 suppl. 1 pg 2545.
Clauset, A., Shalizi, C.R., Newman, M. E. J., Power-law distributions in empirical data, 2007, arXiv:0706.1062v1, URL http://arxiv.org/abs/0706.1062v1.