Why probability is not corroboration
The IEC’s 61508 standard on functional safety assigns a series of Safety Integrity Levels (SIL) that correlate to the achievement of specific hazardous failure rates. Unfortunately this definition of SILs, that ties SILs to a probabilistic metric of failure, contains a fatal flaw.
The root of the problem is that we are not dealing with a question of probability of failure as traditionally defined by reliability, i.e. trying to establish the likelihood of random component failures expressed over some duration, instead the question is whether our design is safe (i.e contains no errors) or not. We are here making an expert judgment as to the correctness of our hypothesis about the safety of the design we are considering. So why can’t we then associate a probability to our degree of corroboration, and in turn let this metric represent the probability of its failure in service? The answer to this question is that such an assignment violates the doctrine that the degree of acceptability cannot be a probability as proposed by Popper (1959).
An example illustrates the point. One can always report the results of testing of a theory in the form of a statement of the degree of corroboration of the theory. But such an appraisal can never take the form of a statement of probability, because this simply does not identify the severity of those tests and in what fashion the theory passed this test. (Popper 1959). The reason for this is that the empirical content of a theory establishes it’s testability and degree of corroboration. The IEC’s tying the degree of corroboration provided by SIL defined assurance activities to a desired failure rate is is an unwitting attempt to make just such an unwarranted assignment.
Fundamentally if we seek to corroborate the correctness of our design, such efforts of themselves cannot then be parlayed into a probabilistic measure of the systems subsequent reliability. You cannot, unfortunately, get there from here.
Popper, K., The Logic of Scientific Discovery (1959).