arosplatforms™AI consultancy
ar
← All articles

MLOps Pipeline Design: Stages, Architecture and Best Practices

MLOps Pipeline Design: Stages, Architecture and Best Practices

Sketch motifs frame the MLOps article title

A good MLOps pipeline is a set of automated, versioned and tested stages that carries a model from raw data to production and back again when monitoring says it should change. Design it in this order: data and features, training, validation gates, a model registry, staged deployment, and monitoring that can trigger the next run. Everything else, including tool choice, is a detail of those six decisions.

This guide walks through each stage, shows how the components connect, covers batch versus real-time serving, and ends with the practices and metrics that keep a pipeline healthy.

Arosplatforms
Build AI Around Your Operations
Arosplatforms designs industry-specific AI systems that integrate with client operations, from automating manual processes to improving predictive analytics.
Explore AI consultancy

TL;DR

  • Design around six stages: data and features, training, validation, registry, deployment, monitoring. Monitoring feeds back into training.
  • Treat the pipeline itself as code: version control, tests and CI/CD apply to the training pipeline, not only to the model.
  • Separate dev, staging and production, and promote only registered, versioned models through explicit gates.
  • Match serving to latency: batch or streaming for minutes or longer, online REST serving for low latency.
  • Measure delivery speed, reliability, cost and model quality together, or you will optimize one at the expense of the others.

Table of Contents

What an MLOps pipeline covers

An MLOps pipeline automates the machine learning lifecycle end to end. Google Cloud’s reference architecture breaks it into development and experimentation, pipeline continuous integration, pipeline continuous delivery, automated triggering, model continuous delivery, and monitoring. Microsoft’s MLOps v2 architecture slices the same ground differently: a data estate, administration and setup, a model development inner loop, and a model deployment outer loop.

The vocabulary differs, but the shape is the same. There is an inner loop where data scientists ingest data, explore, experiment, evaluate and register a model. There is an outer loop where that registered model is tested, deployed and watched. Good design makes the handoff between the two boring and repeatable. If you want a short definition first, see our MLOps glossary entry.

The six stages, in the order you should design them

Six ordered stages of an MLOps pipeline

1. Data and features

Start here, because every later stage inherits its problems. Microsoft’s model development workflow begins with data ingestion, then moves through exploratory analysis, experimentation, development, evaluation and model registration.

Design decisions at this stage:

  • Where raw data lands and who can read it. Databricks recommends Delta tables for raw data and feature tables, with access controls.
  • How features are computed once and reused. A feature store keeps training and serving consistent.
  • What gets validated on arrival. Schema, null rates and distribution checks belong here, not after a model has already trained on bad data.

2. Training as a repeatable pipeline

A notebook is an experiment. A pipeline is a product. Google’s practitioner guide describes training operationalization as automating the packaging, testing and deployment of repeatable training pipelines.

The same guide notes that continuous training can run in response to new data, code changes, a schedule, or changed training settings. Pick the trigger on purpose. A nightly schedule suits slowly changing data. A new-data trigger suits feeds that arrive in bursts. A drift trigger, covered below, suits models whose accuracy decays fast.

Track experiments as you go. Databricks points to MLflow for experiment metadata, and Git for pipeline and code version control. If you are weighing tools, our comparison of Kubeflow and MLflow explains how the two differ in setup effort.

3. Validation gates

A pipeline that trains but never refuses to ship is a liability. Google Cloud recommends data and model validation, infrastructure compatibility checks, API and load tests, and predictive-performance gates, with deployment through test, preproduction and production stages.

Microsoft’s list of staging checks is a useful template: retraining on production data, endpoint-performance tests, data-quality checks, unit tests, and responsible-AI checks for bias. Azure’s Databricks architecture adds model evaluation against production criteria, compliance checks, A/B tests, infrastructure tests, and optional manual sign-offs.

Write each gate as a pass/fail rule with a threshold, such as “candidate must beat the current production model on the holdout set”. Vague gates get skipped under deadline pressure.

4. Model registry

The registry is the spine of the architecture. AWS’s ML lifecycle guidance identifies the model registry as the mechanism for model and data version control and lineage tracking. If you cannot answer “which data, code and parameters produced the model serving traffic right now?”, the registry is not doing its job.

Databricks recommends Unity Catalog for model versioning and governance. Whatever product you use, the requirement is the same: every production candidate is registered, versioned and linked to its lineage before it can be promoted.

5. Staged deployment

Microsoft recommends registering production candidates and promoting them through CI pipelines, automatic registration triggers, or gated human approval. Its baseline inference architecture uses isolated Dev, Stage and Prod environments, with versioned model and environment assets moving through a shared registry. Databricks likewise recommends separate development, staging and production environments with explicit transitions.

For the rollout itself, the same Microsoft baseline uses blue/green: the new deployment starts at 0% traffic, traffic shifts in stages, and rollback fires if thresholds are exceeded. OWASP’s guidance adds shadow deployments, canary releases, rapid rollback and incident-response procedures. Pick one of these patterns and automate the rollback before the first release, not after the first incident.

6. Monitoring that closes the loop

Monitoring is what makes a pipeline a cycle instead of a line. In Azure’s Databricks guidance, monitoring workflows can track input data, predictions, drift, performance, application metrics, logs and infrastructure, with drift thresholds triggering retraining.

Include security signals too. OWASP recommends monitoring input distribution, output entropy, latency, drift, unusual usage and traceable access logs, while keeping sensitive data out of logs. Unusual query patterns can be the first sign of model abuse or data exposure, and an accuracy dashboard will not show them.

MLOps architecture: how the components connect

The components are few. Google Cloud’s architecture connects source control and test/build services to deployment services, a model registry, a feature store, an ML metadata store and a pipeline orchestrator. Monitoring can then trigger another pipeline execution or a new experiment cycle.

Here is a plain-language map of who talks to whom:

Component Receives Hands off to
Source control Pipeline code, configs CI/CD build and test
Feature store Cleaned data Training and online serving
Orchestrator Trigger (schedule, data, drift) Training, validation, registration
Metadata store Run parameters, metrics, lineage Registry, audits
Model registry Validated candidate Deployment pipeline
Serving layer Approved model version Monitoring
Monitoring Predictions, inputs, logs Orchestrator (retrain) or rollback

Two design rules fall out of this table. First, no component should be reachable by a side door: a model that skips the registry has no lineage. Second, every arrow should be automated, with human approval placed deliberately at one or two of them rather than scattered.

Batch or real-time: choose serving by latency

The biggest architectural fork is how predictions are delivered. Azure’s Databricks orchestration guidance says batch or streaming scoring is generally appropriate for latencies of minutes or longer, while low-latency use cases generally need online REST serving.

In practice that means:

  • Batch scoring fits nightly risk scores, weekly demand forecasts and document backlogs. The pipeline is simpler, cost is predictable and rollback is a re-run.
  • Streaming fits event-driven work where minutes are acceptable, such as alert triage.
  • Online serving fits anything a person or system waits on. It needs load tests, autoscaling, versioned endpoints and the staged traffic shifting described above.

Start with the slowest serving mode the use case tolerates. Online serving adds infrastructure and failure modes that many projects do not need. The same logic applies to cloud versus on-premises: the stages stay identical, and only the hosting of the registry, orchestrator and serving layer changes. Regulated sectors often decide this on data residency, which is why we tailor our AI infrastructure and MLOps work by industry.

Best practices that hold up in production

The vendor architectures agree on more than they differ. These are the practices that recur.

  1. Version everything. Code, data, features, model and environment. AWS’s MLOps checklist covers data-centric management, experimentation, observability and model management, robust pipelines and promotion, CI, continuous monitoring, CD and continuous training, and versioning runs through all of it.
  2. Test the pipeline, not just the model. Unit tests for transformations, data-quality checks, load tests for endpoints.
  3. Isolate environments. Dev, staging and production, with explicit promotion.
  4. Gate on metrics. Predictive-performance thresholds decide promotion; humans approve where risk is high.
  5. Release gradually. Shadow, canary or blue/green, with automatic rollback.
  6. Build governance in. AWS defines ML governance as processes and frameworks addressing explainability, auditability, traceability and other lifecycle requirements. Lineage from the registry is what makes audits answerable.
  7. Keep sensitive data out of logs and restrict who can read training data and model artifacts.

How to measure pipeline health

A pipeline you cannot measure will drift toward whatever is easiest to measure. Track four groups together.

Delivery speed. Borrow from software delivery. The DORA research identifies four metrics: deployment frequency, lead time for changes, change failure rate and mean time to restore. Applied to ML, count how often a new model version reaches production and how long it takes from merged code or fresh data to a live model.

Reliability. Pipeline run success rate, failed-run recovery time, and how often a rollback is needed after release.

Model quality. The metric the business cares about on live traffic, plus drift scores on inputs and predictions, compared with the thresholds set in your validation gates.

Cost. Compute per training run, cost per thousand predictions, and idle serving capacity.

These are our suggested groupings, not a standard. Pick a few numbers per group, set a baseline in the first month and review them on a fixed cadence.

Perspective

Most failed ML projects we see did not fail on the model. They failed in the handoff: a notebook that nobody could rerun, a model with no lineage, or a deployment that nobody dared to touch.

Our approach is to design the pipeline before the model is finished. We start from the operation the system must serve, then choose the serving mode, the gates and the monitoring that fit it. Weekly demos on real data keep the pipeline honest while it is being built.

We also build for ownership. Our design goal is that your team can run, retrain and extend the system without vendor lock-in, which is why we favor standard components such as a registry, an orchestrator and version-controlled pipelines over opaque custom glue.

arosplatforms team

Getting a pipeline designed for your operation

If you want this architecture designed for your own data and constraints, our AI infrastructure and MLOps service covers the plumbing: pipelines, registry, deployment and monitoring. To start with scope, our custom AI development page describes the readiness assessment, proof of concept, production system and enterprise platform stages. We also publish industry-specific versions for healthcare, financial services and manufacturing.

FAQ

What is the difference between an ML pipeline and an MLOps pipeline?

An ML pipeline is the sequence that prepares data and trains a model. An MLOps pipeline wraps that in automation for testing, registration, deployment and monitoring, so the model can be released and updated repeatedly. The monitoring step feeds back into training.

Do I need a feature store?

It helps when several models share features or when training and online serving must compute features identically. Google Cloud’s architecture lists a feature store among its core components. A single batch model with simple inputs can start without one and add it later.

When should a model be retrained or rolled back?

Retrain when monitored drift or performance crosses a threshold you defined in advance, or on a schedule or new-data trigger. Roll back when a new release breaks latency, error or quality thresholds during staged rollout. Both rules should be automated before launch.

Should I use batch or real-time inference?

Use batch or streaming when minutes of latency are acceptable, and online REST serving when a user or system waits for the answer. Batch is simpler and cheaper to operate, so choose it whenever the use case allows.

Sources