What is MLOps?
MLOps is the set of practices that keeps machine learning systems working after they ship. It covers how models get trained reproducibly, how they reach production, how you know they still work next quarter, and how you roll back when they don’t.
The reason it exists as a discipline: a model is not a deployment artefact in the way a service is. Its behaviour depends on data that keeps changing after release, so “it passed tests and deployed cleanly” tells you much less than it does elsewhere in software.
What it actually covers
Reproducibility. Given a model in production, can you regenerate it from the same data snapshot, the same code and the same hyperparameters? Teams that can’t do this cannot debug a regression, only replace it.
Deployment. Getting a model behind an interface with versioning, rollback and a canary path. Batch scoring and real-time inference have different failure modes; both need one.
Monitoring. Two distinct things that get conflated. Operational monitoring is latency and errors, the same as any service. Model monitoring is input drift, prediction distribution shift, and accuracy against ground truth once labels arrive, often weeks later.
Retraining. Deciding when a model needs refreshing, and how much of that decision is automated.
Feature management. Ensuring the features computed at training time match those computed at inference. The mismatch between them, training-serving skew, is one of the most common causes of a model that scored well offline and disappoints in production.
The maturity ladder
Most teams sit further down this than they’d like to admit, and that’s usually fine.
Level 0, manual. Notebooks, a model file copied to a server, retraining when someone notices a problem. Adequate for one model with low stakes and honest expectations.
Level 1, reproducible. Training is a versioned pipeline. Data snapshots and artefacts are tracked. You can rebuild any deployed model. This is the level with the highest return on effort, and where we’d push most teams to reach first.
Level 2, automated delivery. Model deployment runs through CI/CD with tests, staged rollout and one-command rollback.
Level 3, continuous. Drift detection triggers retraining, candidates are evaluated against the incumbent automatically, and promotion is gated on measured performance.
Chasing level 3 with two models in production is a common and expensive mistake. Build the platform the number of models justifies.
What to build first
If you’re starting from notebooks, the order that pays back fastest is:
- Version the training data, or at minimum snapshot it per run. Everything else depends on this.
- Turn training into a script that runs end to end from a clean environment. No manual cell execution.
- Track experiments (parameters, metrics, artefacts) so comparisons are records rather than recollections.
- Add a rollback path before adding automation. Being able to revert quickly is worth more than deploying quickly.
- Then monitor drift. Useful once the first four exist, largely noise before that.
Go deeper
- Machine learning consulting — model development with MLOps built in
- Data engineering services — the pipelines feeding your features
- Databricks consulting — MLflow, Unity Catalog and lakehouse ML
- Hire ML engineers — engineers who have run this in production
Models in production but no platform underneath?
Bring your current setup to a free 30-minute call. You get an assessment of where you sit on the ladder, what to build next, and what it costs, including an honest answer if the sensible move is to build nothing yet.
staffai.eu · Senior AI and data engineers from Eastern Europe, on T&M