MLOps (Deep Workflow)

MLOps connects research velocity to production reliability: version data, code, and artifacts together; monitor behavior after deploy.

When to Offer This Workflow

Trigger conditions:

• First production model; batch or online serving
• Drift, bias, or latency SLO misses
• Compliance needs for lineage and explainability

Initial offer:

Use six stages: (1) problem & risk class, (2) data & reproducibility, (3) training & evaluation, (4) packaging & deployment, (5) monitoring & feedback, (6) governance & rollback). Confirm batch vs real-time and regulatory tier.

Stage 1: Problem & Risk Class

Goal: Align ML to decision risk (credit, health vs recommendation).

Exit condition: Offline and online success metrics defined.

Stage 2: Data & Reproducibility

Goal: Snapshot training data; deterministic pipelines; PII handling.

Practices

• Feature stores optional but valuable for consistency
• Secrets not in notebooks; orchestrated jobs

Exit condition: Run id reproduces artifact hash within agreed bounds.

Stage 3: Training & Evaluation

Goal: Train/val/test without leakage; time-series splits careful.

Practices

• Model card with limits and metrics
• Fairness slices where policy requires

Stage 4: Packaging & Deployment

Goal: Immutable artifacts; canary or shadow before full cutover.

Practices

• Model + preprocessing code version pinned together

Exit condition: Rollback to previous artifact id documented.

Stage 5: Monitoring & Feedback

Goal: Data drift, concept drift, latency; business KPIs tied to model decisions.

Practices

• Human review queue for low-confidence predictions when needed

Stage 6: Governance & Rollback

Goal: Approvals for retrain/deploy; audit trail; A/B for big changes.

Final Review Checklist

• Offline metrics aligned with business risk
• Data and code reproducibility
• Packaged artifacts with versioning and rollback
• Online monitoring and drift strategy
• Governance and approval path

Tips for Effective Guidance

• Training-serving skew is a top bug—feature parity tests help.
• Offline accuracy ≠ online business outcome.
• Fairness needs explicit slices—not one headline number.

Handling Deviations

• LLM-heavy products: lean on eval harnesses and prompt versioning (see llm-evaluation).
• Tiny teams: start with artifact registry + dashboards before a full feature store.