Teams that ship models and AI-backed features tend to collect obligations faster than they collect proof. A customer questionnaire asks how access to training data is controlled, a prospect wants to know who approved the last release, and suddenly nobody can point to a dated record. ComplianceOps is the habit that fixes this, and this guide turns it into seven concrete steps you can run on an AI platform team without buying anything first.

Step 1: Write down what you actually owe

Open a spreadsheet and list every obligation that touches your stack: contract clauses, customer security promises, regulations that apply to your sector, and internal policies. Give each line an owner and a plain description. Skip the temptation to organise by framework at this point. A flat list of duties is easier to reason about, and later steps will show how several duties collapse into one control.

The Ops Atlas page on ComplianceOps calls this the obligation inventory and puts it first in scope, ahead of tooling. That ordering is right, because software can only automate what a person has first named.

Step 2: Turn duties into a small library of controls

A control is one statement you can test, such as “everyone with write access to the model registry has been reviewed in the last quarter”. Aim for a short library where each control answers several obligations at once. The Atlas gives a tidy example: a quarterly access review satisfies a six-month rule in one payments standard, an annual rule in a New York financial regulation, and an interval-free clause in the ISO 27001 annex. Record which requirement each control answers, not merely that it exists, otherwise the mapping loses detail you will need during an audit.

Step 3: Decide what evidence looks like

For each control, define the evidence in four parts: the system it comes from, the time it was captured, the method used, and the test it supports. Then attach a freshness rule, for example thirty days for configuration exports and ninety days for a manager’s sign-off. Anything older than its rule is treated as missing. This single convention prevents the most common failure, which is a folder full of tidy-looking documents that describe last year.

One rule deserves its own line for AI teams. Never store a model’s output as proof that a control operated. A fluent summary of an access review might read beautifully and still be invented. The Atlas has a memorable phrase for it, fiction with a timestamp. Evidence should be raw exports, logs and approvals produced by the systems themselves.

Step 4: Schedule collection and tests

Start with your ten most important controls. For each, set up a scheduled job that pulls the evidence and runs the test. Open-source scanners can do some of this for cloud accounts, and policy engines such as Open Policy Agent can check configuration files before they ship. Use one or two tools you already understand rather than assembling a zoo. The point is a repeating schedule, not a clever stack.

Be honest about the word continuous. The Atlas reminds readers that NIST defines it as often enough to support risk decisions, not as real time. A daily or weekly check is perfectly reasonable if it matches how quickly the underlying thing can change.

Step 5: Run an exception register

Some checks will fail, and that is fine. What matters is how you handle it. Open an exception that names the exact resources failing, gives a reason, and carries an end date. When the date passes without a fix, the exception expires and the control counts as failed again. Resist renewing exceptions without adding a compensating safeguard, because the Atlas describes that habit, renewing to green, as one of the classic ways a dashboard lies.

Step 6: Report with ratios that cannot flatter you

Report a handful of numbers side by side: how much evidence is fresh, how many controls are automated, how many are mapped to each framework, and how many were tested at all. Report untested controls openly next to the pass rate. The Atlas works through a small example where a lenient checker would show one hundred percent from the same data that a strict one scores far lower, simply because it ignored untested items and stale sign-offs. A report that makes you look good is less useful than one that tells you where to look.

It is worth pausing on why we bother. If you read our note on how AI feeds better business decisions, the same logic applies in reverse: decisions about risk are only as good as the data behind them. NIST describes continuous monitoring as a way to move from compliance-driven to data-driven risk management, and that is the benefit hiding under all the paperwork.

Step 7: Rehearse the audit and the gap

Before an auditor arrives, hold your own dry run. Pick a control at random and ask a colleague to find the evidence in under ten minutes. Then rehearse the awkward case: a record is missing. Your runbook should say never to back-date, to reconstruct the item with a label such as “retrospective, collected on” plus the date, and to tell the auditor. That honesty is cheap now and enormously valuable later.

Remember the limits as well. Automated evidence does not replace auditor sampling; it changes what gets sampled. A SOC 2 report is an attestation from an accountant, not a certification. And the Atlas cites a well-known retail breach as a reminder that passing an assessment does not guarantee safety.

A thirty, sixty, ninety plan for the AI team

If you want a schedule, the Atlas suggests this rhythm. In the first thirty days, name an owner, build the obligation register and choose the first framework. By day sixty, have collectors and tests running for the top ten controls and an exception register open. By day ninety, open the formal observation window only when nothing is stale, stage a deliberate failure to confirm your tests notice it, and bring a second framework onto the shared library.

Image models and classifiers raise a few extra questions of their own, such as where labelled data lives and who can change it. A guide like our explainer on how image classification works in practice helps you see which stages of such a pipeline deserve a control of their own: data intake, training runs, release approval and monitoring after launch.

Common traps to avoid

• Cleaning up in the weeks before an audit and letting things lapse afterwards.
• Running a separate effort per framework so engineers answer the same question three times.
• Editing old evidence instead of superseding it with a new, dated record.
• Writing tests that are narrower than the requirement, so a policy merely existing passes as enforced.

None of this guarantees you will pass an assessment, and none of it is legal advice. What it gives you is a team that can answer questions from its own records, quickly and truthfully, which is the real point of the exercise.