MLOps: Experiment Tracking and Model Management
Module 7 · MLOps

MLOps: Experiment Tracking and Model Management

Why machine learning needs its own operations discipline, the MLOps lifecycle, and MLflow's building blocks — experiments, runs, parameters, metrics, artifacts, and a model registry with versions — plus why a tracking server is a security-sensitive asset.

ML Is More Than Code

In Module 6 the recommender used a fixed formula. Real machine-learning systems train models — and a trained model’s behavior depends on far more than its source code:

  • the data it was trained on, and how that data was cleaned;
  • the hyperparameters chosen (settings like the number of trees or the learning rate);
  • randomness in how data was split and how training started;
  • the exact library versions used.

Change any one and you get a different model. Six weeks later, “which model is in production, and how was it made?” becomes surprisingly hard to answer. MLOps — machine-learning operations — applies DevOps thinking to this problem: automate, track, and manage models so they can be reproduced, compared, deployed, and improved reliably.

REST query recommend (direct) fetch movies load model log runs GitHub repo Codespaces dev env GitHub Actions test · ZAP scan · deploy MongoDB Atlas document database User Mobile + web app React Native Web API server Express · search + paging runs in Codespaces ML service FastAPI · model server CHANGED Training scripts LR · RF · GB · NN NEW MLflow tracking runs · metrics · models NEW
Module 7: training scripts log runs and models to an MLflow tracking server, and the ML service loads its model from the registry. Highlighted in red: what's new or changed in this module.

The MLOps Lifecycle

Data collect · clean Train try models Track log runs Register version it Deploy serve it Monitor watch drift

Four practices hold it together:

PracticeWhat it meansFor the movie recommender
Experiment trackingSystematically recording every training run’s configuration, code, and resultsCompare four candidate models side by side instead of by memory
Model managementOrganizing, storing, and versioning trained modelsKnow exactly which version is serving recommendations
Model deploymentMaking a model available to applications for real-time predictionsThe ML service loads a registered model at startup
MonitoringWatching live performance and data for drift (the world changing away from the training data)New releases shift tastes; the model needs retraining

What teams do without a tool

It’s possible to do all of this by hand: experiments logged in spreadsheets, models saved in carefully named folders or committed to Git, deployment through custom scripts or containers. It works for one person and a few models. It breaks down with teams, many runs, and audits — which is why dedicated platforms exist.


MLflow’s Building Blocks

MLflow is a widely used open-source platform for the ML lifecycle. It works with most ML libraries and scales from a laptop to large production setups. Its vocabulary:

  • Tracking server — a service with a web UI and API where runs are recorded.
  • Experiment — a named group of related runs, such as all attempts to predict movie ratings.
  • Run — one execution of training code. Each run records:
    • Parameters — inputs you chose, like n_estimators=100 or learning_rate=0.1;
    • Metrics — numbers measuring results, like mean squared error;
    • Tags — labels for searching and context, like the model type or the feature list;
    • Artifacts — output files: the serialized model, plots, data samples.
  • Model signature — a description of the model’s expected input and output, so callers know what to send.
  • Model registry — a catalog of named models with numbered versions. Applications load a model by name and version — for example models:/MovieRatingGB/1 — instead of by file path.

When training scripts run, MLflow writes two kinds of local storage: runs (the metadata — parameters, metrics, tags) and artifacts (the files). The tracking UI reads both to let you browse, filter, and compare runs.

Parameters, hyperparameters, and model parameters

These words get mixed up, so pin them down:

  • Model parameters are learned during training — the weights and biases inside a model, or the split points in a tree.
  • Hyperparameters are chosen before training — the number of trees, the learning rate, the depth.
  • In MLflow, “params” means whatever configuration you log, which is usually the hyperparameters.

Reproducibility

A run is reproducible only if you can recreate its inputs exactly. Good tracking records the code version (a Git commit), the data version or query, the library versions, and fixed random seeds (random_state=42 in scikit-learn) so data splits and initializations repeat. Leave any of these out and two “identical” runs can disagree.

⚠️ Common Pitfall: Leaving the Tracking Server Open

Common mistake: making the tracking server’s port public so teammates can see the UI, without adding authentication.

A basic tracking server has no login by default. Anyone who can reach it can read experiment details, download model files — which may encode information about training data — and in many setups register new model versions. If the serving code loads “the latest version,” an attacker who uploads a poisoned model has just changed what your application does. Put the tracking server behind authentication and network restrictions, and pin serving to a reviewed version.

🧠 Knowledge Check

1. Which of these is a hyperparameter rather than a learned model parameter?

2. Why should an application load a model as models:/Name/Version from a registry instead of a file path?

3. In MLflow, what is a run?

4. What does 'drift' refer to in model monitoring?


Up Next

With tracking in place, the next lesson trains four model types, compares them using the right metrics — and shows why reading those metrics honestly matters more than picking a winner.