Upgrade & Secure Your Future with DevOps, SRE, DevSecOps, MLOps!

We spend hours on Instagram and YouTube and waste money on coffee and fast food, but won’t spend 30 minutes a day learning skills to boost our careers.
Master in DevOps, SRE, DevSecOps & MLOps!

Learn from Guru Rajesh Kumar and double your salary in just one year.

Get Started Now!

Three Ways to Ask an ML Team What Would Make Reviews Easier

Three Ways to Ask an ML Team What Would Make Reviews Easier
Artificial Intelligence

Machine learning teams collect friction the way old servers collect dust. A pull request waits two days because nobody knows who owns the feature pipeline. A reviewer approves a model change without seeing the evaluation split. A new hire spends a week finding which notebook produces the real training set. Everyone feels it, and almost nobody mentions it, because each person assumes the awkwardness is theirs alone.

The cheapest fix we know is a question: what would make this easier? This piece compares three practical ways to ask it inside an AI or AIOps team, shows what each tends to surface, and ends with one concrete change you can try in a single sprint. No framework, no new platform, just the question and a bit of follow-through.

Option one: the written survey

A short form, three or four questions, sent to everyone who touches the review process. For example: “What slows your reviews down? What do you wish the pull request description told you? What would make your first week on this codebase easier?”

What it does well. It reaches quiet people. It leaves a record you can sort afterwards, and it lets someone answer at eleven at night when the idea actually occurs to them. Anonymous responses also make it easier to say things about a senior colleague’s habits.

Where it falls short. Written answers drift toward the generic: “more documentation”, “better communication”. Those are true and nearly useless. People also stop answering forms if the last one produced nothing visible. Treat a survey as a way to gather themes, not specifics.

Option two: the one-line stand-up question

Add a single sentence to an existing meeting: “Anything about this week’s reviews that was harder than it needed to be?” Give it two minutes, no discussion until everyone has answered.

What it does well. Spoken answers are specific because someone just lived the problem. “I could not tell whether the drift check had run.” “The diff was four hundred lines because the formatter touched everything.” A comment like that points straight at a fix.

Where it falls short. Loud voices dominate and junior engineers may hold back. You can soften that by going round in a fixed order, starting with whoever joined most recently, and by having one person note each answer in a shared file rather than trusting memory.

Option three: quiet observation

Instead of asking, sit beside someone for half an hour while they review a change, or while a newcomer sets up their environment, and then ask only one thing at the end: “What would have made that smoother?”

What it does well. You catch the friction people have stopped noticing. A reviewer who always opens three tabs to find the dataset version no longer thinks of it as a problem. Watching them do it, you do.

Where it falls short. It is slow and covers only one person at a time. It is best used selectively, as a follow-up to a theme that the survey or stand-up already hinted at. For a longer read on a related review topic, an article on AI and everyday business decision making shows how much of the work depends on what people actually see in front of them.

Side by side

  • Effort: survey is low once, stand-up question is lowest and repeatable, observation is the highest.
  • Specificity of answers: survey is low to medium, stand-up is medium to high, observation is highest.
  • Reach: survey reaches everyone, stand-up reaches whoever attends, observation reaches one person per session.
  • Risk of nothing happening: highest for the survey, because answers pile up with no owner.

With room for just one method, go with the stand-up question and run it for three weeks. It costs almost nothing and produces concrete items quickly. Use observation to dig into whatever keeps coming up.

One more practical note on mixing the methods. Start with the stand-up question because it is quick, then use a short written form only for the people who were absent or too reserved to speak. Finally, schedule a single observation session for the theme that appears twice or more. That order keeps the effort proportional to the evidence and stops anyone from feeling interrogated.

Turning answers into one concrete change

Teams tend to skip this step, yet it carries the real value. Do not try to fix every complaint. Sort the answers, find the one mentioned most often, and pick a change small enough to finish within a sprint. Here are some examples that come up in model-focused teams:

  • A pull request template with three required lines: what data version was used, which evaluation split, and what was deliberately left out.
  • A short CI step that posts the evaluation summary as a comment, so reviewers do not have to rerun a notebook to see it.
  • A formatter run as a separate commit, so the real diff stays readable.
  • A one-page “first week” note listing which repository to clone first, which environment file to copy and who answers questions.
  • A rule that large review requests wait until the author adds a two-sentence summary at the top.

Pick one. Write down what you expect to improve, such as fewer back-and-forth comments asking for context, and look again after two weeks. Should it fail to help, retire it quietly and try the next item. Changes you can undo easily are the ones people are willing to try.

A worked example

Consider a small team maintaining a recommendation model. The stand-up question produced the same complaint three weeks running: reviewers could not tell whether a change affected the training data or only the serving code. Nobody had raised it formally, because each reviewer assumed they were just slow at reading diffs.

The fix was tiny. The team added a checkbox to the pull request template, “touches training data: yes or no”, and a path-based label that applied automatically when files in the data folder changed. Review comments asking “does this change the dataset?” nearly vanished. The cost was perhaps an hour of setup, and the benefit showed up the very next week. A related read on this topic is a plain walkthrough of how image classifiers work, which is useful background when your reviewers are mixed between data and platform people.

Keeping the habit going

The question only works if people believe it leads somewhere. A few habits help.

  1. Report back. When a change ships, tell the team which comment inspired it.
  2. Keep a short visible list of “heard, not yet done” so nothing seems to vanish.
  3. Ask again after a month, in the same words, so you can see whether the answers shift.
  4. Rotate who runs the question so it does not belong to one manager.

You do not need to treat every answer as a mandate. Some friction is the price of a careful process, and a reviewer who slows down to check an evaluation split is doing exactly the right thing. The goal is to remove the accidental friction, not the useful kind.

Outside perspective

It helps to see the same idea in other settings. Informational websites in specialised subjects face the same question when they decide how to lay out a page for newcomers. Urology Hospitals, for example, presents itself as a plain-language website about urology with an unranked directory of hospitals and doctors, and it notes that it is not a medical service. A site like that depends on feedback about what readers find hard to locate, and the same logic applies to your team’s internal tools: the people using the process every day know where it snags.

Bottom line

Surveys gather themes, stand-up questions produce specifics, and observation catches what people have stopped noticing. Combine them lightly, choose one small change, and measure it honestly. Asking what would make reviews easier will not fix an architecture or a staffing problem, but it reliably finds the dozen small annoyances that make good engineers dread opening the review queue. Start with a single sentence at next week’s meeting and see what comes back.

Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x