Upgrade & Secure Your Future with DevOps, SRE, DevSecOps, MLOps!

We spend hours on Instagram and YouTube and waste money on coffee and fast food, but won’t spend 30 minutes a day learning skills to boost our careers.
Master in DevOps, SRE, DevSecOps & MLOps!

Learn from Guru Rajesh Kumar and double your salary in just one year.

Get Started Now!

The Review Comments That Slowed an ML Team: A Case Story

The Review Comments That Slowed an ML Team: A Case Story
Artificial Intelligence

A forecasting squad inside a regional freight firm had a backlog problem that nobody could see on a dashboard. Their sprint board looked healthy. Their model for predicting late deliveries had decent offline scores. And yet every release took a week longer than planned. This is the story of how they found out why, and how a small habit built around the two-minute rule changed their pace. Both the firm and the squad are made up, yet anyone who ships AI features will recognise the pattern.

The quiet pile behind the sprint board

The team lead did an unusual thing one Friday. The lead asked everyone to export every unfinished item, not just the tickets, but also the open review comments, the half-answered chat threads and the notes-to-self in their notebooks. The result was a sheet of about two hundred lines.

Roughly a hundred and forty of them were tiny. A reviewer had asked for a column in a feature table to be renamed from “d1” to something readable. A data engineer wanted a comment added above a threshold constant. Someone had spotted a stale path in a training script. A teammate had asked, three days earlier, whether the evaluation notebook used the March or April snapshot.

Not one of those took more than a couple of minutes. Yet each was waiting, and waiting items have a way of multiplying. The pile was the real bottleneck.

What the pile was costing

The team ran a small experiment to find out. Over one week, each person noted down how long they spent on each tiny item in two phases: the time to record and revisit it, and the time to do it. The totals were uncomfortable. Recording, re-reading and re-sorting tiny tasks took about twice as long as completing them would have.

There was a second effect, harder to measure. Pull requests with five unresolved nitpicks stayed open because nobody wanted to reopen the context. A reviewer would approve with “minor comments” and the author would postpone those comments, then the merge would stall until someone had a free hour. The free hour never came.

The change: do it now if it takes about two minutes

The lead proposed one rule, written on the team wiki in two sentences. If an item takes about two minutes and you have everything you need, do it immediately. If not, put it on the board with a concrete next step.

They agreed on a short list of what counted:

  • Renaming a variable or a column after a review comment points out that it is unclear.
  • Fixing a typo in a docstring, a log message or a model card line.
  • Replying to a review comment with a decision, including a plain “will handle in the follow-up, ticket attached”.
  • Adding a missing unit or a date to a chart axis label.
  • Pinning a dependency version someone has just noticed floating.

Everything else, retraining runs, schema changes, a new evaluation split, stayed on the board as before.

The first awkward week

It did not go smoothly at first. One engineer treated a “quick rename” as a two-minute task, then discovered the column was used in three downstream notebooks. An hour went by. In the retro, the team agreed on a clarification: the two minutes include checking what the change touches. If the check itself is long, the task is not small and goes on the board.

Another lesson came from the other direction. A reviewer began answering every comment instantly and shallowly, just to clear the queue. The lead pointed out that the rule is about small tasks, not about shallow thinking. A short reply must still be a correct reply.

What moved in the figures they tracked

Nobody claimed a dramatic result. They tracked two simple things: how many pull requests were open for more than three days, and how many lines were on the board at the end of the week. Within a month the first count roughly halved and the second shrank to a size that could be read in one glance. Release slippage dropped from a week to a day or two, though the team was careful to say other changes, a smaller review group and a calmer quarter, also played a part.

The softer effects were more visible. Standups got shorter because fewer items needed explaining. Reviewers stopped dreading the “approve with comments” button, since comments were resolved the same afternoon.

Borrowing from other fields

The team liked to look outside their own corner for ideas. A teammate pointed to a general piece about how data-driven tools shape everyday business choices and noted that the same logic applies to internal decisions: a quick, informed call usually beats a long deferral. Another read the site’s reflection on the ways machine learning is changing classrooms and borrowed the idea of immediate feedback loops for the team’s own onboarding.

Interest in other communities also crept in. A couple of people followed a community forum for talking about Indian listed companies, a site that says plainly that its content is not investment advice. The team’s takeaway was modest: public discussion spaces are worth reading with that kind of caution in mind, and they keep model-evaluation decisions inside their own review process.

How to try it on your own team

  1. Take a census. Export every open item, including review comments and chat promises. Count how many are under two minutes.
  2. Agree what counts. Write down five examples specific to your stack, as the team above did, and add one clear exclusion for anything that touches shared code.
  3. Add the check. Before you call something small, look at what depends on it. Ten seconds of checking saves the hour-long surprise.
  4. Do the small things in the moment. Reply, rename, fix, then move on. Resist adding a ticket for something faster to finish than to describe.
  5. Measure two numbers. Pick open-review age and end-of-week list length. Look at them monthly rather than daily.

An ending that is not a revolution

The logistics team never became famous for this. Nobody gave a conference talk about renaming a column on the spot. But months later, a new colleague joined and asked why the board was so short. The answer, delivered over coffee, was simple: most of what used to live there was cheaper to do than to write down. The remaining lines were the ones that deserved attention, and the team finally had room to give it.

Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x