How do we assure safety when we modify a system?
While the safety community has developed a comprehensive suite of analyses and management techniques for system developments the number of those available to ensure the safe modifications of systems are somewhat less prolific.
Which is odd when one considers that most systems spend the majority of their life in operation rather than development…
The major problem that emerges when one looks at accidents whose causes originated in modifications is that of ‘linear thinking’, that is thinking about the desired consequences of the change and not about the potential side effects.
The results of overlooking such effects can be seen in the Flixborough accident, in which 28 people died, and the plant was leveled, because of an ill-considered, poorly executed modification.
To minimise the potential effects of such thinking a modification can be thought of as a miniature development program which should includes the same general safety activities as any new system development.
In summary one needs to carefully assess the impact of the modification upon safety, ensure that the modified design remains within the established system design standards and finally make sure that the modification is actually carried out correctly.
Assess the safety of the modification (Perform a hazard analysis)
The objective of a modification hazard analysis is to determine the hazards associated with the change, assess the associated risk and predict the safety impact on the existing system.
The first step is to identify the boundary of the change. This is also one of the most critical because a poorly thought out and incomplete definition of the change’s interface (i.e. the boundary of the change) increases the likelihood that non-obvious side effects will be overlooked.
One technique I have found useful is to draw a context diagram to identify those components of the system and environment with which the change will interface with directly and interact with indirectly.
One should also consider partitioning the interfaces identified into (for example) electrical/functional, mechanical/physical, supplied services, software, process and human interface categories to ensure that no potential interaction is overlooked.
Having established the boundary of the change, identified the affected interfaces a review of the systems documented hazards and their controls (sometimes termed the safety baseline) should be conducted to identify the understood safety risk associated with the system.
This is where having an up to date Preliminary Hazard Analysis (PHA) report or other inventory of the systems hazards can prove invaluable.
If an existing hazard analysis is not available then an initial hazard analysis may need to be performed to identify the system hazards, assess their risk catalogue the hazard controls in place and establish the safety baseline.
There may also be supporting design analyses of critical components or system attributes (for example weight or balance in a vehicle would be considered critical) that may need to be revised in light of the change.
Finally we need to assess the impact of the change upon the system’s safety. I’ve found the following checklist of questions useful in identifying such impacts:
- Could this increase the likelihood of an existing hazard occurring?
- Could this adversely affect an existing hazard controls effectiveness?
- Will this introduce a new hazard?
- Will this adversely affecting safety-critical components?
- Will this cause a noncompliance with an existing safety design standard?
- Will this change the criticality of a component (reducing redundancy)?
- Will the above issues (1), (2), (3), (4), (5) and (6) increase the likelihood of a hazardous state occurring?
- Could there be a hazardous interaction with another modification?
- What is the likelihood of the design of the modification being wrong? And then what?
- Do we understand what we are modifying, does the design match the product?
This process of considering the impact of a change is not as easy as one might imagine.
For example the engineers who designed the Airbus A320 software modification that was intended to prevent thrust reversers from deploying in flight, after the Lauda Air 004 disaster, would have found it incredible to consider that their software modification would be a causal factor in the subsequent Warsaw accident a few years later.
Ensure compliance to design standards
Design standards are applied to catch unknown/unknown hazards in the system. Where we make a modification that is non compliant with an existing design standard that can increase the likelihood of an unidentified hazard resulting in an accident.
So one of the easiest ways to reduce the risk of any modification is to simply identify the applicable design standard and insist that it be complied with.
Formally manage the modification process
The cornerstone of any effective modification management process is formal change management that addresses both the technical (i.e. design) and operational (i.e maintenance and operational use) impacts of any change.
The change management process should also require appropriately authorised and competent persons to approve the design including the identification and incorporation of specialist engineering discipline advice where necessary.
The design of the modification should be objective in nature, that is there should be a specific design output such as a field modification instruction, service bulletin or modification drawing that has been formally released, can be revision controlled and whose authority is understood.
Before starting any modification process there should be assurance that the physical systems configuration matches that of the design. In older systems this can be a significant challenge and may in the end require an audit of the pre-modification configuration to be conducted.
The modification process should verify the integrity of critical aspects of the design before running up the plant, for example safety pressure test before pressurising or the testing of safety functions that may have been affected before commencing operation.
Finally we should confirm that the modification design as intended has been implemented in the plant by:
- Carrying out inspections and audits of the system, to check the system against the design and ensure they correlate,
- Ensure that the system that is going into service is the same configuration as that which was tested (very important for software intensive systems), and
- Enforce a waiver/deviation process to approve minor ‘delta’s against the design during the modification process.
None of the above are earth shattering or ground breaking concepts of course, but from my experience the above is is the minimum quality standard for the process of safely modifying plant.