MLOps Engineer

MLOps is where the machine-learning rubber meets the production road. An MLOps engineer keeps models healthy, reproducible, and continuously improving in real systems - building the pipelines, tooling, and monitoring that let ML teams move fast without breaking things quietly.

What an MLOps engineer does day-to-day

MLOps engineers sit at the intersection of software engineering, data engineering, and machine-learning infrastructure. On any given day that might mean debugging a training pipeline that silently started producing stale features, wiring up a new model to the serving fleet, reviewing a drift alert that fired overnight, or writing Terraform to provision the GPU cluster the ML team needs for next week's training run.

The role is explicitly operational. Where an ML engineer optimises model accuracy, an MLOps engineer optimises the system around the model - reproducibility, deployment velocity, production reliability, and the feedback loops that keep models fresh. In smaller teams the same person does both; at larger organisations the roles are distinct and the MLOps function increasingly resembles a platform team.

Skills companies expect

MLOps job descriptions converge on a core set of competencies. The depth expected scales with company size - a startup wants a generalist who can own the entire stack; a large ML team wants deep specialisation in pipeline orchestration or model serving.

The interview process

MLOps interviews are heavier on system design and lighter on pure algorithms than a standard software engineering loop. Expect 4-5 rounds at most companies:

1. 1

   Recruiter and technical screen

   Round 130-45 min

   Background, motivation for MLOps, and a light technical warm-up - Python data structures, basic ML workflow concepts, a cloud service or two. Screening for baseline competence and communication clarity.
2. 2

   ML system design

   Round 260 min

   Design a training pipeline, a feature store, or a model-serving system at scale. Expect deliberate ambiguity - you'll need to ask clarifying questions, propose trade-offs, and describe your monitoring story. This is the most differentiating round for MLOps candidates.
3. 3

   Coding round

   Round 360 min

   Python-heavy: data manipulation with pandas/numpy, pipeline scripting, or light algorithmic work. Some companies add a take-home that involves writing a small Airflow DAG or an MLflow integration script.
4. 4

   Infrastructure and tooling deep-dive

   Round 445-60 min

   Kubernetes, Docker, Terraform, or cloud-specific questions. How have you handled GPU scheduling? How do you manage secrets in a training job? What does your CI/CD pipeline look like for a model update?
5. 5

   Behavioral / cross-functional

   Round 530-45 min

   Collaboration with ML engineers and data scientists, handling ambiguous priorities between ML velocity and production stability, incident response stories. STAR format throughout.

Common interview questions

Conceptual

• What is training-serving skew and how do you prevent it? The gap between how features are computed during training vs. at serving time - the leading silent killer in ML production. Prevention means using the same feature computation logic in both contexts, ideally through a shared feature store.
• Explain data drift vs. concept drift. Data drift: the input distribution shifts (the world changed). Concept drift: the relationship between inputs and the correct output shifts (what the model predicts is no longer what the business wants). Both require different responses - data drift often means retraining on fresh data; concept drift often means relabelling and rethinking the target.
• How do you decide when to trigger automatic retraining? Drift thresholds, scheduled cadence, production metric degradation below a threshold, or a combination. The key design decision is whether the trigger is stat-based (drift detection) or outcome-based (business metric regression).
• What does a healthy model registry look like? Every model version has: a linked experiment run, a snapshot of the training dataset version, evaluation metrics, a deployment status, and an audit trail of who promoted it to production and when.

Scenario-based

• You deploy a new model and predictions immediately degrade. Walk me through your debug process. Start with data: did the feature pipeline produce the right values? Compare online feature values to training-time values for the same inputs. Then serving: did the correct model artifact load? Check version. Then model: compare live prediction distribution to the baseline distribution from staging.
• How do you do A/B testing for models without corrupting product metrics? Shadow mode runs both models but serves only the old one - useful for validating quality before any user impact. Canary routes a small traffic slice to the new model. Full A/B splits traffic and measures business outcomes. The choice depends on how confident you are and how reversible the impact is.
• How would you build a retraining pipeline that handles both scheduled and drift-triggered runs? Separate the trigger logic from the pipeline logic. A lightweight monitor publishes events to a message bus (e.g. a scheduler fires hourly, a drift detector fires on threshold breach). The pipeline subscribes and runs identically regardless of which event triggered it - this keeps the pipeline pure and testable.

ML system design topics

System design rounds for MLOps roles consistently pull from the following problem space. Unlike standard SWE system design, the interviewer is probing ML-specific concerns: feature consistency, experiment reproducibility, model versioning, and production monitoring.

• Design a model training pipeline for a given use case (fraud detection, recommender, NLP classifier). Cover data ingestion, feature engineering, training job, evaluation gate, and model registry write. Be explicit about what triggers a retraining run.
• Design a feature store - online vs. offline stores, transformation consistency, low-latency retrieval for serving, and backfill for historical training. What storage layer do you pick and why?
• Design a model serving system at scale - REST or gRPC, request batching, response caching, autoscaling, GPU vs. CPU routing, and latency SLOs. How does the system handle a model rollback?
• Design a production ML monitoring system - what you monitor (inputs, outputs, latency, statistical drift), how you alert, how you triage false positives, and what automatically triggers downstream actions like retraining.
• Design a shadow-deployment system - run a candidate model alongside the production model, log both outputs, evaluate offline before switching any live traffic.

Salary ranges by region

MLOps commands a premium over general software engineering in most markets because the supply of engineers who can credibly operate ML in production is still thin relative to demand. Figures below are approximate median total compensation (base + bonus + equity) from public data sources, blended across company sizes. AI-lab and big-tech offers sit materially above these figures in every market.

30/60/90-day plan for a new MLOps role

1. 1

   First 30 days - map the system

   LearnMonth 1

   Understand every model in production: what it does, how it was trained, how it's deployed, and whether anyone is monitoring it. Document what you find. Identify the single biggest production risk - the model with no drift monitoring, the pipeline with a manual promotion step, or the feature computation that is inconsistent between training and serving.
2. 2

   First 60 days - close one critical gap

   BuildMonth 2

   Pick the most impactful gap from your audit and close it: add drift monitoring, automate a manual deployment step, migrate experiment tracking to MLflow, or wire an evaluation gate into the CI pipeline. Ship a concrete improvement the team uses every day.
3. 3

   First 90 days - own a pipeline end-to-end

   OwnMonth 3

   Design and ship a training-to-serving pipeline for one model - either from scratch or as a redesign of an existing one so it is reproducible, testable, and monitored. This is the definitive signal that you can operate independently at the MLOps engineer level.

Common mistakes MLOps engineers make

• Treating ML models like software. Code doesn't degrade silently; models do. Without continuous prediction monitoring you won't know the model is failing until users complain - or until a KPI moves for reasons nobody can explain.
• Skipping the feature store conversation. Training-serving skew is the industry's most common and most preventable production ML bug. Build or adopt a feature store early - retrofitting it later is a multi-month project with significant downtime risk.
• Manually promoting models. Any step in the model-release process that requires a human to do something manually will eventually be done incorrectly under deadline pressure. Automate the promotion gates: pass the evaluation threshold, pass the shadow test, then promote.
• No rollback story. You can't test your way to zero bugs in a model release. The safety net is a fast, practised rollback to the previous version. If you've never actually rolled back a model in production, you don't have a real rollback story.
• Optimising pipelines before they work correctly. A training pipeline that runs twice as fast but produces unreproducible results is worse than a slow one you trust. Correctness and reproducibility first; performance second.

MLOps vs related roles

As AI teams mature, the boundary between MLOps, ML engineering, and data engineering gets drawn in different places. Here is how the typical split looks at a well-staffed product organisation:

Related guides: ML Engineer for the model-training side of the house; System Design Fundamentals for the infra concepts that underpin every MLOps stack; and the Backend Roadmap for the software engineering foundations every MLOps engineer needs.