Safety researchers love using metaphors as a framework to describe how accidents happen, which they call accident models.
One of the earliest models, dating back to 1931, is Herbert W. Heinrich’s domino model of accident causation:
About seven years later, in 1997, Jens Rasmussen proposed the dynamic safety model. This model doesn’t have an evocative a name as “domino” or “Swiss cheese”. I like to call it the “boundary” model, because everyone talks about it in terms of drifting towards a safety boundary:
I haven’t encountered a good metaphor that captures the role of multiple contributing factors in incidents. I’m going to propose one and call it the Gamma knife model of incidents.
Gamma knife is a system that surgeons use for treating brain tumors by focusing multiple beams of gamma radiation on a small volume inside of the brain.
Each individual beam is of low enough intensity that it doesn’t affect brain tissue. It is only when multiple beams intersect at one point that the combined intensity of the radiation has an impact.
Every day inside of your system, there are things that are happening (or not happening(!)) that could potentially enable an incident. You can think of each of these as a low-level beam of gamma radiation going off in a random direction. Somebody pushes a change to production, zap! Somebody makes a configuration change with a typo, zap! Somebody goes on vacation, zap! There’s an on-call shift change, zap! A particular service hasn’t been deployed in weeks, zap!
Most of these zaps are harmless, they have no observable impact on the health of the overall system. Sometimes, though, many of these zaps will happen to go off at the same time and all point to the same location. When that happens, boom, you have an incident on your hands.
Alas, there’s no way to get rid of all of those little beams of radiation that go off. You can eliminate some of them, but in the process, you’ll invariably create new ones. There are some you can’t avoid, and there are many that you don’t even see, unless you know how to look for them. One of the reasons I am interested in otherwise harmless operational surprises is that they can reveal the existence of previously unknown beams.