The team at a mid-sized logistics company, let us call them the routing group, had spent five months on a model that predicted which deliveries would arrive late. Offline, it looked excellent. The launch plan was tidy: switch on predictions for dispatchers, watch accuracy, celebrate. Two weeks before release, their lead did something that felt almost rude. She cancelled the planned kickoff and called a different meeting, with one instruction: “Assume the launch has failed. Tell me why.” This is an account of what that meeting produced, and it is a good model for any AI launch.
Setting the scene: strong numbers, thin evidence
The routing group had a validation score everyone liked. What they did not have was evidence about the conditions in which dispatchers would meet the model. Validation data came from last year’s records, tidied and complete. Live data would arrive messy, late and occasionally missing. The lead knew that an offline score describes the past, and that optimism about a model usually attaches itself to the number it already has.
So she framed the meeting as a pre-mortem. The room was told to write, in the past tense, a short story of the failure. Past tense matters. “The model might drift” sounds hypothetical; “the model drifted and nobody noticed for a month” sounds like something you would have to explain.
What the room wrote down
Eleven stories came out of ten minutes. After clustering, three themes held most of the weight.
Theme one: the input changed under the model
An engineer wrote that a depot began recording delivery windows in a new format, the feature pipeline quietly turned the values into blanks, and predictions degraded while every dashboard stayed green. The group had no check that compared incoming data with the shape the model was trained on. That is a cheap thing to build and an expensive thing to be missing.
Theme two: people used the output in ways nobody planned
A dispatcher wrote that the late-delivery flag started to be treated as a verdict rather than a probability. Drivers were reassigned on the strength of a flag that was wrong a fair share of the time. Nobody had told dispatchers what the flag meant, or how often it would miss. For readers who want a plain introduction to how predictions get used in business choices, the earlier piece on how machine predictions feed everyday business decisions covers the basics.
Theme three: no one could turn it off
The most uncomfortable story was the shortest: “We could not tell if it was working, and rolling back meant a week of work.” The group had built the launch as a one-way door. An afternoon of work would make it a door that opens both ways.
Turning stories into changes, with evidence attached
The lead refused to leave with a list of worries. Each theme got an action that produced evidence, not reassurance.
- For the changed inputs: a comparison between live feature values and the training distribution, run every day, with an alert on large gaps.
- For the misuse: a one-page note for dispatchers explaining how to read the flag, where its limits are, and what to do when it feels wrong, plus a feedback button recording each disagreement.
- For the missing exit: a switch that returns dispatchers to the old rule-based estimate within minutes, tested once before launch.
Notice what is absent. There was no plan to make the model better. The pre-mortem found that the likeliest failures lived around the model, in data plumbing, human habits and release mechanics. That is usually where they live, because the people closest to the model have already stress-tested it a hundred times, while the surrounding pipes, handoffs and habits have been tested by no one.
A lighter-touch way to bring in outside perspective
Midway through, the lead pointed at a habit from software security. Over on DevSecOps Now, a DevSecOps site, the blog concentrates on securing CI/CD pipelines, a discipline that treats every release path as something that can fail or be misused. She told the group to borrow that attitude for model releases: any path a model change takes to production deserves the same question about what could go wrong on the way.
What happened after launch
The launch was not flawless, and a pre-mortem does not promise that. In the first week the daily comparison flagged a shift in one depot’s data, and the team found a formatting change within hours instead of weeks. A dispatcher used the feedback button to report a pattern of misses on holiday weekends, which became the next training task. Nobody needed the rollback switch, but two people said later that knowing it existed made the launch feel less tense.
Evidence from the first month was modest and honest: a few recurring failure modes found early, and no incident bigger than a confusing alert. That is what a successful pre-mortem looks like from the outside. It is dull, and the dullness is the result.
Lessons worth copying
- Hold the meeting two weeks before release, while schedules can still move.
- Write failures in the past tense and keep them short.
- Cluster the stories, then choose the few that combine likelihood and damage.
- Attach an evidence-producing action to each, not a promise to be careful.
- Look at the plumbing around the model before the model itself.
A team working on factory automation faces the same pattern, which the post on applying machine learning to production lines touches on from a different angle: the model is rarely the weakest link; the process around it is.
The routing group’s lead summed it up at the retrospective. “Our optimism was a feature of the plan, not a flaw in the people. We just needed a meeting where it was not allowed to speak first.”