A friend of mine runs the machine-learning platform at a mid-sized fintech, and last year she told me a story I haven’t stopped thinking about. Her team spent four months polishing a fraud model, terrified of shipping it until it was “perfect”. Meanwhile the rule-based system it was meant to replace quietly misclassified thousands of transactions. The irony stung: in chasing zero errors on paper, they tolerated far more errors in reality. What finally unblocked them wasn’t a better model. It was a number — an agreed, written-down answer to the question “how wrong are we allowed to be?” In other words, an error budget for an AI system.

That idea comes from site reliability engineering, where teams define exactly how much failure a service may accumulate before new work stops and stabilisation begins. Applied to AI and AIOps, it is quietly transformative — and wildly misunderstood. So let’s do some myth-busting. Here are the six misconceptions I hear most often, and the stories that unravel them.

Myth 1: An AI system must be flawless before it goes live

Nobody holds a weather forecaster to 100 percent, yet somehow we expect it of models. The truth is that every production AI system already operates with an implicit error tolerance — you just haven’t written it down. My friend’s fintech team eventually agreed that one bad fraud flag per two hundred legitimate transactions was acceptable, because the human review queue could absorb exactly that volume. The moment the tolerance became explicit, shipping stopped feeling reckless. The model went live, the review queue hummed along, and the four-month paralysis looked, in hindsight, like the riskiest choice of all. Perfection is not a requirement; an honest, measurable ceiling on imperfection is.

Myth 2: Reliability targets slow innovation down

This one gets the mechanism exactly backwards. A budget is permission, not prohibition. While your error budget is healthy, you have mathematical proof that you can afford experiments — a new embedding model, a riskier retraining cadence, an aggressive caching layer. Only when the budget runs dry does the brake engage. Teams without explicit targets don’t move faster; they oscillate between recklessness and panic, freezing everything after each incident because nobody can say whether the system is “okay”. The budget replaces that mood swing with arithmetic. It’s the same logic that lets organisations use AI to sharpen everyday business decisions: numbers beat vibes, especially under pressure.

Myth 3: You can’t set targets for something probabilistic

I once watched an architect wave away the whole discussion with “models are stochastic, you can’t SLO a dice roll.” But reliability practitioners measure probabilistic systems all the time — that’s rather the point. You pick indicators you can observe: the share of predictions served within 300 milliseconds, the fraction of responses passing a quality filter, the drift score staying under a threshold, the pipeline completing before the morning batch window. Each becomes a service level indicator; each gets an objective; the shortfall allowance becomes budget. The discipline of choosing those indicators well is teachable — it sits at the heart of the curriculum at SRESchool, where engineers learn to define SLIs and SLOs and run systems against error budgets — and it works on stochastic systems precisely because it measures outcomes, not internals.

Myth 4: Error budgets only make sense for giant platforms

The opposite is closer to true. A small AIOps team of five feels every midnight page in its bones; it has less slack than anyone, which makes an explicit tolerance more valuable, not less. A startup serving one enterprise customer can define its budget in that customer’s terms: “no more than two missed anomaly-detection windows per month, because that’s what the contract tolerates.” The arithmetic scales down beautifully. What the big platforms have isn’t a monopoly on the method — it’s simply more public writing about it, which tricks smaller teams into thinking the method belongs to someone else.

Myth 5: Once the model is accurate, the operations look after themselves

Here’s a story from the other direction. A computer-vision startup I advised had a genuinely excellent model — and an inference service that fell over whenever traffic doubled. Their users never saw the model’s elegance; they saw timeouts. Accuracy is one indicator among several, and often not the one users feel first. Latency, availability, data freshness and pipeline punctuality all deserve their own targets, with observability — metrics, logs and traces — wired in so you can tell which part of the budget is actually burning. The teams that get this right think about serving infrastructure as carefully as model weights, the way the builders of high-throughput visual pipelines that scale without choke points obsess over every stage, not just the clever one in the middle.

Myth 6: Blowing the budget means finding someone to blame

An exhausted budget is information, not an indictment. It says: the system, as currently built and operated, cannot sustain this pace — so the pace changes. Feature work pauses, retraining experiments wait, and the team spends the recovered time on the unglamorous fixes that restore headroom. In healthy teams this handoff is almost boring, which is exactly how it should be. The budget depersonalises a conversation that otherwise turns into finger-pointing between the data scientists who want to ship and the operators who want to sleep. Both groups answer to the same number now. (If your organisation wants to build that shared vocabulary formally, foundation-level reliability certification such as the CSRE track gives ops and ML engineers a common language faster than any internal memo ever will.)

A bonus half-myth: “we’ll set the targets after launch”

I call this a half-myth because it sounds so reasonable. Why guess at tolerances before you have production data? Gather a month of real traffic, the argument goes, then set the numbers. The trouble is what happens during that month: every stakeholder forms a private, unspoken expectation of how the system should behave, and those expectations harden into politics. The product lead remembers the demo-day latency and treats it as a promise. The compliance officer assumes the false-positive rate from the test set will hold forever. By the time you convene to agree on targets, you aren’t negotiating numbers — you’re renegotiating beliefs, which is far harder.

The fix is to launch with provisional targets, loudly labelled as such. “For the first quarter, we’re treating 98 percent of predictions under 400 milliseconds as our working objective, and we’ll revise it with real data in January.” Provisional numbers anchor the conversation without pretending to false precision, and revising them becomes a scheduled, unemotional event instead of a crisis meeting. The half of the myth worth keeping is the humility: your first targets will be wrong. The half worth discarding is the delay. A wrong number you can correct beats a missing number every time, because only the written one generates a budget you can actually track.

What my friend’s team does differently now

Eighteen months on, that fintech platform team runs every model behind three explicit numbers: a prediction-quality floor, a latency ceiling, and a monthly allowance for missed retraining windows. When a quarter ends with budget to spare, they spend the surplus deliberately — one quarter it bought a migration to a faster feature store, another it bought a week of chaos-style failure drills on the inference tier. When the budget runs low, they stop arguing and stabilise. Nobody debates whether the system is “good enough” anymore, because good enough has a definition.

And that, really, is the whole secret the myths obscure. Error budgets aren’t a Big Tech ritual, a brake on ambition, or a blame instrument. They are a way of deciding — before the pager goes off, before the stakeholder meeting, before the model embarrasses you in production — exactly how much failure you can live with, and then living accordingly. AI is reshaping how whole sectors operate, from finance to the classrooms being transformed by intelligent learning tools; the systems doing that reshaping deserve reliability targets as thoughtful as the models themselves.

Write your numbers down this week. Pick one model in production, give it one honest indicator, one target, and one month of budget. You’ll be surprised how quickly the fog lifts once “how wrong are we allowed to be?” has an answer.