When an AI system starts behaving strangely, the first hour decides the next three days. A team that sorts its statements into confirmed facts, assumptions and plain guesses usually finds the cause quickly. A team that treats every theory as a fact retrains the wrong model. This guide gives a six-step routine for debugging a misbehaving AI model that you can run with a notebook and a shared channel.
Step 1: Write the symptom as an observation
Open a blank page and describe what happened using only things someone could replay. “Between two timestamps, the classifier returned the same label for every request from the mobile client” is an observation. “The model got worse after the update” is already an interpretation, because it contains a cause and a before-and-after claim.
If you cannot rewrite the symptom without verbs like “broke” or “drifted”, you have not yet got the observation. Pull a handful of raw requests and responses and describe those instead.
Step 2: Build the confirmed list
Under the symptom, list statements that you or a teammate directly verified in a system of record today: the deployed version string, the configuration actually loaded, the input schema in the last hour of traffic, the output distribution for a named window. Add where each came from, in a few words.
Keep this list boring. The aim is not to explain anything but to establish ground that will hold weight. Statements that someone remembers from last week, or that a dashboard implied, do not belong here until somebody re-opens the source.
Step 3: List assumptions with a reason and a check
Now write the things you are treating as true without having verified them: “feature preprocessing is unchanged”, “the labelling team used the same guidelines”, “traffic mix is the same as last month”. Beside each, add why you believe it and the cheapest way to find out. Teams that skip the reason discover later that a single stale conversation supported half the list. An article on how AI supports everyday business decision making makes a related point from the decision side: the conclusions only hold as well as the inputs behind them.
Cheap checks go first. Diffing two config files takes a minute and can eliminate an entire branch of suspicion.
Step 4: Name the guesses and rank them
Guesses are the leftover ideas: “maybe a tokenizer update”, “perhaps a vendor changed an upstream feed”, “could be seasonality”. Write each as a sentence that could be proven wrong. A good hypothesis predicts something you have not looked at yet: “if it is the tokenizer, older requests replayed through the new build should also fail.”
Rank by two things, how plausible the guess feels and how cheap the test is. Plausibility alone biases you toward the story you already like. Cost alone sends you after trivia. Together they give a sensible order.
Step 5: Test one guess at a time
Run the test that distinguishes the top guess from the rest, and record the result in the confirmed list, not in the guess list. A passed test moves a guess to confirmed. A failed one is also a confirmation, of what the cause is not, and it should be written down with the same care so nobody retests it tomorrow.
Resist changing two things together. A retrain with a new data filter and a new learning rate might fix the problem, and you will still not know why, which means you will not know whether it stays fixed. When images are involved, a plain-language walkthrough of how a classifier reads a picture is a useful reminder of how many invisible steps sit between pixels and labels, and each is a place where an assumption can hide.
Step 6: Change the system, then update the page
Only when one explanation has survived its test do you alter the model, the data or the pipeline. Make the smallest change that follows from the confirmed cause, and state in advance what output you expect to see afterwards. If the result differs, the cause was incomplete, and you are back at step four with a longer confirmed list.
Finally, clean up the page. Move everything that turned out to be true into a short summary, delete the dead guesses or label them as ruled out, and keep the timeline. The next person on call will read it far more closely than any dashboard.
A compact example
Suppose a recommendation service suddenly favours one category for almost all users. The observation: in the last six hours, one category accounts for nearly every top slot, across all regions. Confirmed: the model version is unchanged, the config hash matches the last release. Assumed: the feature store is serving fresh data, because its job showed green. Guessed: a catalogue import duplicated a category.
The cheap check on the assumption reveals that the feature store job went green while writing an empty table, so the service fell back to default scores. That is a confirmed cause reached by testing an assumption, not by retraining. The guess about duplicated catalogue entries is then dropped, and its removal is logged.
Where to practise the habit
Teams new to this routine often rehearse it on old incidents before using it live. Structured courses help here too. DevOpsSchool is an online learning platform that lists courses, certification tracks and study resources across DevOps and neighbouring engineering topics, and it also describes hands-on labs, which suit this kind of rehearsal. The point of any such material is repetition: sort, label, test, record.
Common slips to watch for
- Promoting a guess by repetition. If three people say “it must be the data” in a row, the third is repeating, not confirming.
- Treating a green check as proof. A passing status means the job finished, not that its output is right.
- Testing the favourite only. Run at least one test that could embarrass your leading theory.
- Losing the negative results. Ruled-out ideas are as valuable as confirmed ones.
None of this slows a good team down. It replaces the loop of retraining, hoping and retraining again with a short list of facts, a short list of assumptions and a ranked list of guesses, each with a test attached.