Printed on demandShips worldwide14-day returns

Single Point of Failure: Meaning, Examples and the Bus Factor

A single point of failure (SPOF) is one part of a system that stops the whole system when it fails. If your app runs on one database server and that server dies, the app dies too. The usual fix is redundancy: two or more of the part, so one can fail while the other keeps working. And in many teams, the most dangerous single point of failure is a person.

What a single point of failure means

Wikipedia defines it as “a part of a system that would stop the entire system from working if it were to fail.” The test is simple. Pick a component and ask: if this disappears right now, does everything stop? If the answer is yes, it is a SPOF.

A chain is a good model. It does not matter how strong most links are. One weak link, with nothing next to it, decides when the chain breaks.

Common examples

Single point of failureWhat happens when it failsUsual fix
One database serverEvery request that needs data failsA replica with automatic failover
One load balancerTraffic cannot reach any of the healthy servers behind itA pair of load balancers sharing a virtual IP
One DNS providerUsers cannot resolve your domainSecondary DNS with another provider
One power supply or network linkThe server or rack goes offlineRedundant power and a second uplink
One cloud regionA regional outage takes the whole service downDeployments in two regions
One personNobody else can fix productionDocumentation, pairing and on-call rotation

Redundancy has its own traps. Two web servers behind one load balancer only move the SPOF to the load balancer. Three replicas on the same physical host still share that host. You remove a single point of failure only when the backup does not share the thing that failed.

A single point of failure in Kubernetes

A Deployment with one replica is a SPOF by definition. If that pod crashes or its node goes down, the service is gone until a new pod starts:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 1

Raising replicas to 3 is the first step. It is not enough on its own: the scheduler may place all three pods on the same node. A topology spread constraint asks Kubernetes to spread them across nodes:

spec:
  replicas: 3
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: api

Now one node can fail and two pods keep serving. The database behind them may still be a SPOF, which is the point: you remove them one at a time.

The human single point of failure: the bus factor

The bus factor is the number of people who would have to disappear suddenly before a project stalls. The name comes from the grim question “what if they get hit by a bus?”. Kinder versions say “lottery factor”: what if they win the lottery and never come back?

Wikipedia cites a June 1994 Python mailing list post by Michael McLay, asking what would happen “if Guido van Rossum were to be hit by a bus”, as an early example of the question. The variant “truck number” appears in Organizational Patterns of Agile Software Development by Coplien and Harrison (2004).

A bus factor of one means one person is a single point of failure. Everyone knows who it is: the developer who wrote the billing system, the sysadmin who knows why that cron job runs at 03:17, the only person with the registrar password. They cannot take a holiday without a phone in their pocket.

Git history gives a rough first look at who knows a part of the code. This lists authors by number of commits in one directory:

git shortlog -sn HEAD -- src/billing/

If one name has almost all the commits, you have found a candidate. Commit counts are not knowledge, but they are a good place to start the conversation.

How to raise the bus factor

  • Write the runbook while the expert is still around, and have someone else follow it.
  • Pair on the scary parts of the system, not only on new features.
  • Rotate on-call so more than one person has fixed production.
  • Store credentials in a shared vault, not in one person’s head or laptop.
  • Review code across teams, so at least two people have read every critical file.

Why the shirt is funny

Our Single Point of Failure shirt has a red arrow pointing up, at the person wearing it. It is a proud confession: I am the part that brings everything down. It is also a gentle hint to the team that someone should write some documentation.

It fits the colleague who is always on call because “only they know”. It sits in the Sysadmin & IT Support collection, next to another classic of production support, Have you tried turning it off and on again.

Single Point of Failure T-Shirt, front

Single Point of Failure T-Shirt · 24 €
The arrow points at the wearer.

Sysadmin Hero T-Shirt, front

Sysadmin Hero T-Shirt · 24 €
For the person who keeps production alive.

Have You Tried Turning It Off and On Again T-Shirt, front

Have You Tried Turning It Off and On Again T-Shirt · 24 €
What the SPOF says on every call.

Unisex, S to 5XL, printed on demand. Check the size guide. More ideas in our gifts for DevOps engineers and gifts for sysadmins.

FAQ

What is an example of a single point of failure?

A single database server with no replica. If it fails, every part of the application that needs data fails with it.

How do you remove a single point of failure?

Add redundancy that does not share the same weakness: a second server on different hardware, a second region, a second person who knows the system.

What is the difference between a SPOF and the bus factor?

A SPOF is any component whose failure stops the system. The bus factor counts people. A bus factor of one is a human single point of failure.