Engineering Practice
Draw a Shared-Dependency Map Before Debugging Ten Identical Failures
When many systems fail the same way at the same time, a shared-dependency map turns correlation into a concrete debugging plan instead of ten parallel investigations.
- Debugging
- Operations
- Systems Thinking
- Root Cause Analysis
- Reliability
When ten things fail the same way at the same time, opening ten terminals is usually the wrong first move. The useful question is not which victim to inspect first; it is which dependencies all ten victims share. A small dependency map turns that question into something you can test, cross out, and hand to another operator without relying on intuition.
A dependency map is more useful than a list of suspects
The basic heuristic is simple: simultaneous, uniform failures usually point upward toward something shared. The map is what makes that heuristic operational.
Start with the affected things on the left. They might be hosts, jobs, services, disks, access points, or user sessions. Then draw the layers each one depends on to do the specific thing that is failing. Do not diagram the entire environment. A map that includes every possible dependency becomes architecture documentation, not a debugging tool.
For a service that cannot authenticate, the useful path might be application process → local network → resolver → identity service → certificate or credential source. For a set of machines that all lose storage, the path might be host → controller → link → enclosure → power. The names change; the shape does not.
The first pass should answer one question: where do the paths converge? Those convergence points are your high-value suspects because one failure there can explain many victims without requiring coincidence.
Map only the path that can produce the observed symptom
A dependency is relevant only if its failure can plausibly create the symptom you actually have. That constraint keeps the map small.
If ten hosts can still resolve names but cannot reach one application, DNS probably does not belong near the top of the suspect list. If all ten lose every route beyond a shared gateway, an application database is not interesting yet. The map should follow the broken capability, not everything the systems happen to use.
I use three questions for every node I add:
- Is it shared by the affected victims?
- Can it produce this exact failure mode?
- Can I test it independently or compare it with a healthy control?
If the answer to all three is no, it does not deserve attention on the first pass. This is where the map beats brainstorming. Brainstorming rewards imagination; a dependency map rewards explanatory power.
Timeline correlation tells you which shared layer moved
Topology explains what could affect everything. Time helps explain what did.
Put the first observed failure time next to the victims, then annotate the shared path with events that happened around that window: a restart, configuration rollout, lease change, certificate transition, power event, link flap, dependency outage, or scheduled maintenance. The point is not to assume the most recent change caused the problem. It is to turn “what changed?” into a bounded comparison against the shared layers already on the map.
Clock quality matters here. A log timestamp from one host, an alert timestamp from another, and the time an operator remembers seeing the symptom are not automatically equivalent. Use the best common time source you have and note uncertainty when clocks are questionable.
A shared layer that changed at the same moment the victims failed deserves an early test. A shared layer that was stable before, during, and after the event becomes less interesting. Neither conclusion is absolute, but both are stronger than choosing a component because its name appeared in an error message.
Healthy controls remove whole branches from suspicion
The fastest map is not the one with the most failing examples. It is the one with a useful healthy comparison.
Look for something that performs the same operation but differs at exactly one dependency boundary. If nine clients behind one switch fail while a tenth client behind another switch succeeds, that healthy client is not an annoyance; it is a cut line through the dependency graph. If two jobs use the same service but only the jobs from one runner pool fail, the shared application may be innocent while the runner-side path becomes interesting.
A good control answers: what is the smallest dependency difference between healthy and broken?
That often lets you cross out several nodes at once. If both healthy and failing systems use the same resolver, identity service, and database, those shared services are less likely to explain the split. The branch where their paths diverge becomes more valuable.
Controls are especially useful when every individual victim looks healthy in isolation. The problem may live between components rather than inside one of them. Comparing paths exposes that boundary.
Use the worksheet to turn the map into a test sequence
I would keep the exercise small enough to fit on one screen or sheet of paper. For each incident, fill out this sequence:
- Name the victims. Which systems are actually affected, and which similar systems are not?
- Normalize the symptom. Confirm that “same failure” really means the same layer, error class, and start time.
- Draw the shared path. Add only dependencies required for the failing capability.
- Mark convergence points. Circle dependencies whose failure could explain several victims at once.
- Add the timeline. Note relevant changes or state transitions near the first failure.
- Find a healthy control. Identify the smallest path difference between working and broken cases.
- Test from narrowest shared boundary outward. Prefer checks that can eliminate several hypotheses at once.
- Record what the test disproved. Cross nodes out instead of repeatedly rediscovering that they are healthy.
That last step is easy to skip and surprisingly important. Debugging gets expensive when every handoff resets the investigation to “here are ten things that are broken.” A marked-up map preserves negative evidence: what was tested, when, and what that test actually ruled out.
The map is a companion to the heuristic, not a rewrite of it
The existing lesson that identical failures often share a cause is the recognition step. This exercise starts after that recognition and asks what to do next. It does not assume there is exactly one root cause, and it does not say every simultaneous failure is shared infrastructure. Correlated changes can create multiple failures at once, and one upstream problem can expose several downstream defects.
The map still helps because it makes those possibilities visible. If the victims stop converging on one node, you may have multiple fault domains. If a healthy control shares everything you thought was causal, your model is wrong and needs another layer. That is useful progress.
The goal is not to draw a beautiful diagram. It is to spend the first few minutes reducing the search space before you spend the next hour debugging individual victims. When failures arrive as a crowd, map what they share, compare what stayed healthy, and make the dependency graph earn its keep. The broader reasoning starts with the shared-cause heuristic; this is the worksheet I would use to turn that heuristic into an investigation.