You trained a model, wrapped it in a small web service and it runs beautifully on your laptop. Then a teammate runs it and gets a different result, or the server kills it halfway through a busy afternoon. Nine times out of ten the culprit is not the model. It is the box around the model. Looking after that box is what the Ops Atlas calls ContainerOps, and this post explains it in everyday words for people who build AI services.
Start with the picture: a lunchbox, not a kitchen
A container is a lunchbox for software. Inside go the program, the libraries it needs and the settings it expects. Outside is whatever machine happens to carry it. Because the lunchbox is sealed, your service sees the same contents whether it sits on a laptop, a build server or a rented cloud machine.
The Atlas defines ContainerOps as the care of single containers below the level of any large scheduler: building images you can trace, naming and storing them properly, and running them in a predictable way. It also owns up that nobody outside the Atlas has settled on this word. If you want the complete scope, the Atlas entry on ContainerOps is the place to look, and it is refreshingly clear about what it does not cover.
Images and layers, without jargon
An image is the frozen recipe result: a stack of layers, each recording the changes one build step made. Think of transparent sheets stacked on an overhead projector. Every step adds a sheet, and what you see at the top is all the sheets together.
This next detail catches many beginners off guard, and it matters even more in AI work, where API keys and dataset credentials are everywhere. Layers record changes, they do not erase history. If an early step copies in a secret and a later step deletes it, the secret is still sitting on the earlier sheet and whoever downloads the image can dig it out. The Atlas advises building with temporary secret mounts which keeps the value out of every layer.
A friendly rule of thumb: never write a secret into a build step, even briefly. If you already did, treat the credential as exposed and replace it.
Names drift, fingerprints do not
Images carry tags, such as a version label or the word latest. A tag is a sticky note and someone can move it to a different image tomorrow. A digest, by contrast, is a fingerprint computed from the content, so it names one exact build forever.
For a model service this is gold. When a result looks odd, you want to know exactly which build produced it. If production runs by digest, the answer is a single copy-and-paste. If production runs by the latest tag, you may never be able to reconstruct what was running last Tuesday. Our own comparison guide, on choosing between AI tools for your team, makes the same point about evidence: decisions get easier when you can name precisely what you are comparing.
Limits: the part that saves your afternoon
Models like memory. A service that loads a large file at start-up, or builds big batches in flight, can eat far more memory than anyone planned. Without a ceiling, one greedy container can squeeze its neighbours or force the whole machine to start killing things.
The Atlas is firm that you should always set memory and process-count limits. When a container hits its memory ceiling, the system stops it with a forced signal, and you will see an exit code of 137 in the logs. That number is simply 128 plus the signal number 9, and recognising it saves hours of confusion. On newer resource-control setups, the same label can also appear when the whole host runs out of memory, so check the machine as well as the container.
CPU limits deserve more care. A cap that looks generous on average can still slow a service in short bursts, because the system pauses the container when it uses its allowance early in a period. The Atlas suggests watching the share of periods that were throttled and keeping it below five percent for anything where response time matters.
The CPU-count trap
Here is a surprise that bites AI services in particular. Inside a container, the usual call that asks how many processors exist can still report the machine’s total, not the slice you were granted. A program that sizes its thread pool from that number may start dozens of workers on a container allowed only two processors, then spend its life being throttled.
The Atlas notes that the Go language began sizing its parallelism from the container’s CPU limit only from version 1.25. For other runtimes, set worker counts explicitly from configuration instead of trusting auto-detection. Computer vision pipelines, such as those in our simple guide to how machines classify images, are a classic example where oversized thread pools quietly hurt speed.
Stopping politely
When a container is told to stop, it receives a gentle termination signal first, then a forced kill after a grace period. The Atlas gives ten seconds as the default in Compose. If your service ignores the gentle signal, requests in flight are cut off and any half-written output is lost.
The usual cause is how the start command was written. The shell form launches your program underneath a shell that does not pass the signal along. The exec form starts your program directly so it can hear the request. Handle the signal in code, finish the current job, and exit, and add the init flag if your service spawns child processes.
Safe defaults worth adopting early
- Run as an ordinary user. Your inference service almost never needs administrator rights inside the box.
- Keep the final image small. Build in one stage, copy only what you need into a minimal last stage. The Atlas mentions a stripped-down static base of around two mebibytes as an example of how little can be left.
- Keep base images fresh. Pin them by digest and let an automatic pull request move them forward, so pinning never freezes security fixes.
- Rotate logs. Out of the box, the standard log files grow without any rotation, so a chatty service can quietly fill a disk.
A small worked scenario
Picture a team with a text-classification service. Version one runs on a developer laptop with whatever libraries happened to be installed. Version two is packaged: a first stage installs dependencies and compiles anything heavy, and a slim second stage carries only the result. The team records the digest of every build in the release notes, and an automated pull request moves the base image forward each time a new one appears.
Three weeks later a customer reports odd outputs from a Tuesday batch. Because the release notes name a digest, an engineer pulls that exact build, replays the input and reproduces the oddity in minutes. Without the fingerprint, the same investigation would start with guesswork about which of several similar-looking builds was live.
Where to go from here
Newer to the wider craft? The roadmap of skills for a first DevOps role on this site pairs well with this piece, because ContainerOps is one of the pieces that roadmap eventually asks you to master. Start small. Pick one service, check how it is built and named, add a memory limit, and read its exit codes the next time something stops.
What you gain is not glamour. It is the quiet confidence that the thing you tested is the thing that runs, and that when it fails, the failure leaves clues. That is worth more than another clever model tweak on a Friday afternoon.