A loud failure is almost a stroke of luck. Something stops working, someone notices within the hour, it gets dealt with. It’s annoying, it costs a morning, and it ends there.
The failures that really cost are the other kind. The ones where the system keeps telling you everything went fine.
I found one recently, in something I run myself. A submission that had to go out every evening, with a legal deadline, and that showed as sent every time. On the panel it was green, with its neat confirmation line. On the receiving end it was being rejected, every evening, in silence.
It had been going on for two weeks.
Nobody was doing anything wrong. The piece that sent confirmed it had sent, and that was true: it had sent. The piece that received recorded a rejection, and that was true as well. The two simply weren’t talking, and nobody had ever put those two facts on the same line.
I only found it because I’d started crossing them automatically, for an unrelated reason.
This strikes me as the most useful thing I’ve learned in months, and it generalizes further than it looks. A system that checks itself isn’t checked. All it can tell you is whether it did what it believed it was supposed to do, which is a different question from the one you care about.
The question you care about is whether the effect arrived. And whoever acted never has that answer. Somebody else does.
Since then I’ve got into the habit of asking, for anything that matters, what the second source is. The send confirmation against the receipt of whoever was supposed to collect. The number of people declared against the number registered. The published page against what you actually see opening the site from outside. The document the form accepted against the one the office would accept.
Every time the two sources turn out to be the same source, there’s a blind spot.
It isn’t a technology problem, which is why I’m writing about it here. It holds identically for work done by people. A report written by whoever carried out the work describes the carrying out, not the result. A client who doesn’t complain isn’t a satisfied client, they’re a client who didn’t complain. Revenue booked isn’t revenue until you see it in the bank.
These are all things we know. The point is that nobody verifies them, because verifying costs attention and in that moment there’s no signal to justify it.
And the signal isn’t coming. That’s the definition of the problem.
So the question to ask isn’t whether something works. That one always gets a reassuring answer, which is exactly why we ask it.
How would I know, and who would tell me if it stopped?