MLOps Base — Cheat Sheet

Tags: mlops, machine-learning, model-registry, ci-cd, monitoring, data-drift

Level: basic

TL;DR: Core practices to automate, version, deploy, and monitor machine learning models reliably in production environments.

Key concepts

MLOps unites Machine Learning system development with DevOps operational practices.

Data and code must both be versioned to guarantee reproducible training pipelines.

Feature stores provide unified, consistent feature definitions for training and online serving.

Model registries store versioned model artifacts, metadata, dependencies, and validation status.

CI/CD in MLOps tests code, data schemas, and model performance metrics automatically.

Continuous Training (CT) triggers model retraining upon data drift or performance degradation.

Data drift occurs when input feature distributions change over time.

Concept drift occurs when statistical relationships between features and targets change.

Serving strategies include batch inference, asynchronous queues, and real-time REST or gRPC endpoints.

Observability requires tracking system health, latency, throughput, prediction drift, and business metrics.

Typical questions and ready answers

Q: How do you distinguish between data drift and concept drift?R: Data drift is a shift in input feature distribution P(X) without changing target mapping. Concept drift changes the relationship between inputs and targets P(Y|X).

Q: Why is Git alone insufficient for ML version control?R: Git handles source code well but cannot efficiently version large binary model weights, large datasets, or pipeline runtime lineage.

Q: What is the primary role of a model registry?R: It acts as a centralized catalog tracking model artifacts, versions, lineage, deployment stages, and benchmark evaluations before promotion.

Pitfalls and common mistakes

Training-serving skew caused by calculating features differently in training versus production.

Monitoring only system metrics like CPU or memory while missing silent model accuracy degradation.

Deploying model weights without locking dependencies and environment runtime versions.

Lack of automated baseline data validation before ingestion into retraining pipelines.

Overwriting existing model artifacts instead of enforcing immutable, semantic versioning.

Snippets and commandsPython

```import mlflow

import mlflow.sklearn

from sklearn.ensemble import RandomForestClassifier

mlflow.set_experiment("fraud_detection_baseline")

with mlflow.start_run():

params = {"n_estimators": 100, "max_depth": 5}

mlflow.log_params(params)

clf = RandomForestClassifier(**params)

clf.fit(X_train, y_train)

score = clf.score(X_test, y_test)

mlflow.log_metric("accuracy", score)

mlflow.sklearn.log_model(

sk_model=clf,

artifact_path="model",

registered_model_name="FraudDetectionModel"

)

```

To review

[ ] Implement population stability index calculation for automated data drift detection.

[ ] Configure shadow deployment strategies alongside canary rollouts.

[ ] Connect DVC remote storage pipelines with object store backends.

[x] Configure experiment tracking and run logging with MLflow.

References

MLflow Documentation: Model Registry and Tracking

Google Cloud Architecture: MLOps Continuous Delivery and Automation Pipelines

Chip Huyen: Designing Machine Learning Systems