When a machine-learning project stalls, the team meeting usually ends with a list of additions. More training data, more input columns, a larger model, another preprocessing stage, an extra review gate. It feels like progress because additions are visible. Yet in practice, many stalled AI projects improve when someone removes something instead. This piece takes six common beliefs about adding to AI projects and tests each against what practitioners tend to observe. The through-line is simple: in AI work, subtraction is an underrated engineering move.
None of this argues for doing less work. It argues for doing the right work, and for noticing that the reflex to add can hide the real cause of a problem. Each myth below is followed by the evidence and a practical alternative.
Myth 1: More data always improves the model
Data volume is the most cherished belief in the field, and it is partly true. For many tasks, a model trained on more representative examples does better. The word that matters is representative. Appending a pile of duplicated, mislabeled or off-distribution records can quietly drag performance down while making training slower and evaluation harder to interpret.
What the evidence suggests is that curation often beats collection. Removing near-duplicate rows, discarding examples with conflicting labels and filtering out records from a pipeline that has since changed all reduce noise. A smaller, cleaner set is also easier to audit, which matters when someone asks why the model behaves oddly on a particular input.
Try instead: before collecting anything new, sample a few hundred existing records and read them. If you find obvious junk, delete it and retrain. Compare the result with the original run, and only then decide whether more data is warranted.
Myth 2: Every extra input feature carries signal
Feature lists in production systems tend to grow like attics. Someone adds a column for a hunch, it is never removed, and two years later nobody can say whether it helps. Features that carry no signal still cost something: they widen the schema, add upstream dependencies, create more places for data to break and invite accidental leakage of the very thing you are trying to predict.
Ablation studies, where you remove one feature or group at a time and measure the change, are a long-standing method for exactly this reason. When the metric holds steady after a removal, the feature was not earning its place. Simpler feature sets also tend to generalise more predictably when the world shifts.
Try instead: schedule a periodic pruning pass. For each feature, ask what breaks if it disappears. If the honest answer is nothing measurable, retire it and note the decision.
Myth 3: A bigger model is the safe fix
When accuracy plateaus, scaling up feels responsible. A larger architecture, more parameters, longer training. Sometimes that is right. But a larger model also raises serving cost, increases latency, expands the surface for odd failures and makes retraining slower. If the bottleneck is label quality or a leaky evaluation split, a bigger model simply learns the flaw more thoroughly.
A useful discipline is to establish how a deliberately simple baseline performs, such as a linear model or a small tree ensemble. If the simple version gets close, the extra complexity may not repay its running costs. Readers who want a gentle grounding in how image models reach their answers can look at a plain explanation of how classifiers interpret pictures, which also shows why input quality matters at least as much as network size.
Try instead: remove capacity until performance starts to drop, then step back one notch. The smallest model that meets the requirement is usually the cheapest to run and the easiest to explain.
Myth 4: Extra pipeline stages make results more reliable
Teams that have been burned once tend to bolt on safeguards: another validation step, another transformation, another intermediate store. Each stage seems harmless. Together they produce a pipeline where nobody knows which step is responsible for a surprising output, and where a failure in stage seven surfaces as a mystery in stage nine.
Every stage you remove is a stage that cannot fail, cannot drift out of sync and cannot need a hand-over document. Before adding a new check, ask whether an existing one already covers the same risk, or whether the upstream step that produced the problem could be fixed instead.
Try instead: draw the pipeline on one page and mark each stage with the specific failure it prevents. Any stage without a clear answer is a candidate for deletion.
Myth 5: Adding more people to the review loop raises quality
Human review is valuable for model outputs, but it is easy to inflate. A second approver, a third sign-off, a weekly committee. Past a small number, additional reviewers dilute responsibility. Each person assumes another has looked closely, and the average attention per decision falls.
Better results come from fewer reviewers who know exactly what they are checking. A narrow checklist owned by one named person usually outperforms a broad process owned by everyone. Applied to business use of AI, the same logic holds. A piece such as an article on AI-assisted choices inside a company is a reminder that the point of a model is to make a decision clearer, not to add ceremony around it.
Try instead: trim review down to the smallest group that can actually catch the failure you care about, and give them a short, specific brief.
Myth 6: Deleting things is riskier than keeping them
This is the belief that holds the other five in place. Keeping feels safe and deleting feels irreversible. But in a system with version control, experiment tracking and reproducible training runs, most removals are reversible. The truly risky state is the accumulating one, where abandoned experiments, orphaned model versions and stale scheduled jobs sit in production consuming budget and confusing newcomers.
Practical safeguards make deletion calm. Tag the last known good state. Archive rather than purge for a defined period. Write one sentence on why each removal happened. With those three habits, subtracting becomes a normal part of the workflow instead of a heroic act.
Try instead: keep a short “retired” list in the project repository. Seeing items move onto it builds the team’s confidence that removal is routine.
What the six myths share
Each myth assumes that a problem in an AI system is caused by something missing. Often the cause is something present: a noisy column, an oversized model, a redundant stage, a crowded review. Diagnosing by subtraction gives a cleaner experiment too. Remove one thing, hold everything else constant, and observe. That is a far more interpretable test than adding three things at once and hoping the metric rises.
A caution is in order. Subtraction is not a universal cure. Sometimes the data really is thin, the model really is underpowered, and the review really is absent. The claim here is narrower: try the removal experiment first, because it is cheap, reversible and informative.
A short routine for your next stalled project
- Write down the symptom in one sentence, with the metric that reveals it.
- List everything in the system that you could remove without rebuilding anything.
- Remove one item at a time and re-measure against the same test set.
- Keep the removals that cost nothing; restore those that hurt.
- Only after that pass, consider what to add.
Practitioners often discover that writing up these small experiments is as useful as running them. A concise log of what you removed and what happened becomes a reference for the next person. If you prefer to publish such notes outside a company wiki, a network like BlogOrbix, which presents itself as a distraction-free space for people who write, offers a clean place to do it.
Closing note
The next time a model underperforms and the room fills with proposals to add, try proposing a deletion first. It may feel less impressive than a new architecture. It also tends to be faster to test, easier to undo and kinder to the people who must maintain the system next year. Fewer moving parts leave more room to understand what the ones that remain are really doing.