Machine learning teams have a habit that rarely gets named: they keep half-built work to themselves for far too long. A notebook sits on a laptop for three weeks, a feature pipeline is tuned in private, a model write-up is polished before anyone outside the team has seen it. The state of early feedback in AI work is uneven, and what follows surveys the current position, what is changing and what a practical team can do today to show rough drafts to a colleague while changing direction is still cheap.
Where most teams actually are
Ask around and you will hear two stories. In the first, an engineer or analyst works alone until the result looks convincing, then presents it at a review. The review turns up a misunderstanding about the target variable, or a data source that was never meant for the purpose, and a month of work is quietly bent or thrown away. In the second, the same person shares a scrappy first sketch within days, a colleague asks what the model is supposed to decide, and the answer exposes a gap before a single training run has been paid for.
The second story is less common than it should be. Part of the reason is cultural: sharing something unfinished feels like admitting that you do not yet know the answer. Part is practical, because AI work often lives in tools that are awkward to hand around. A notebook with hidden state is hard to review, and a half-trained model has no tidy diff.
What “rough” means in AI work
It helps to be concrete about what is worth showing early. A rough draft does not have to be code that runs. Several forms of unfinished work reveal the big flaws well before you have invested in accuracy numbers:
- A one-page design note stating the decision the system will support, who will use the output and what happens when it is wrong.
- A pull request opened as a draft, with the data loading and a deliberately crude baseline, and nothing tuned.
- A small sample of raw inputs and the labels you intend to use, pasted into a document for someone else to eyeball.
- A sketch of how the output will be shown to the person who has to act on it.
Each of these can be put in front of a reader in an afternoon. Each also targets a different kind of early mistake: the wrong question, the wrong data, the wrong labels or the wrong interface.
The trend: draft pull requests and design notes become normal
Over the last few years, engineering practice has drifted toward opening work early. Draft pull requests, short design documents and shared experiment notes have become ordinary parts of many teams’ routines, and AI teams are slowly catching up. The tooling helps: most code hosts let you mark a change as a draft so reviewers know not to pick at style, and experiment trackers let people read a run without re-running it.
What has not caught up is the habit of saying what you want back. A draft with no instructions invites comments on variable names. A draft that says “I only want to know whether this baseline is a fair comparison” gets a useful answer in minutes. The most effective teams treat the request itself as part of the draft: one sentence stating the kind of feedback sought and, equally important, the kind that is not wanted yet.
What goes wrong when feedback comes late
Late feedback is expensive in AI work for a specific reason: the cost of change rises sharply once training and tuning are done. Consider a team building a classifier to route support requests. If the review comes after weeks of tuning, a reviewer may point out that the categories overlap in practice, or that the labels came from a process nobody trusts. At that point the choices are to retrain with new labels, which is slow, or to ship something known to be weak. If the same observation had arrived when the team had only a table of a few hundred examples, the fix would have been an hour of re-labelling.
There is also a social cost. People who have invested heavily in a result defend it, and the discussion shifts from “is this right?” to “do we really have to redo this?” Early feedback keeps ego out of the room because nobody has put much of themselves into the sketch.
A practical routine for a small team
No formal process is required. A routine that fits in a few lines is enough:
- Time-limit the solo phase. Decide at the start how many days you will work alone before showing anything, and keep that window short.
- Write the ask. Before sharing, write one sentence on what you want a reader to check, such as the framing of the problem or the choice of baseline.
- Pick one or two readers. One close colleague and one person from the audience who will use the result is plenty. More voices at this point create noise.
- Record what you learned. Keep the notes next to the work, so the reasoning survives after people move on.
- Change something visible. If you receive feedback and change nothing, people stop giving it.
One caution about timing: do not wait for a calm week. Early review pays for itself exactly because the work is still messy, so schedule it by the calendar instead of by how presentable the notebook feels. A reader who sees the crude baseline on day two can still redirect you; a reader who sees the tuned model on day twenty can mostly only admire it or object. The cheaper conversation is almost always the earlier one, and it is also the friendlier one for everybody involved.
What to ask for, and what to postpone
The framing of the request shapes the answer more than the quality of the draft. Early on, ask about direction: is this the right problem, is the baseline fair, do the inputs reflect what will happen in real use? Postpone questions about polish, such as plot formatting, naming or documentation. Those matter later, but raising them now pulls attention away from the big flaws, which are the whole point of showing something rough.
A useful trick is to name your own doubts. “I am least sure about how I handled missing values” gives a reader a place to start and shows them you are not asking for reassurance. Reviewers often find the problem you did not name, but naming one makes them braver about raising it.
Seeing the habit in other kinds of work
The idea is not unique to machine learning. Any place that mixes plain reference material with open conversation has the raw ingredients for it, and KeralaOrbit, which brings Kerala events, their venues, a discussion forum and a blog together on one site, is one such mix. Drafts of any sort do better when there is somewhere for a few readers to react.
The point for an AI team is simpler still: make the place to react easy to find. A shared folder, a standing thread or a fifteen-minute slot on a weekly call is enough, so long as people know that unfinished work is welcome there and that nobody expects polish.
Where early review goes next
Expect early review to extend into the data itself. Teams are starting to share label guidelines and sample batches before collecting at scale, and to circulate model summaries before deployment rather than after. Write-ups that describe intended use and known limits are easier to improve when they are drafted early and passed around. The same reasoning applies to sketches of how outputs will be shown to users.
If you are reading around the topic, two earlier pieces on this site give useful background. One covers how models feed into everyday business choices, which is a good reminder of why the decision a system supports should be written down first. Another is a short primer on how image classification works from the inside, a good example of a topic where a tiny labelled sample shared early can reveal whether the categories make sense.
A short list to take away
- Share a sketch, not a result. A design note or draft pull request is enough.
- State the question you want answered and the ones you do not.
- Choose readers who resemble the eventual users of the work.
- Name your own doubts out loud.
- Change something, then say what you changed.
Working this way will not make the hard parts of machine learning easier. It simply moves the moment of discovery earlier, to a point where a misunderstanding costs an afternoon rather than a quarter.