Machine Learning (ML) Engineering DevOps Practice

Context:
- It can be measured by ML DevOps Metrics, such as: ML Model Change Lead Time, ML Model Deployment Frequency, Change Failure Rate, and Mean Time to Recovery (MTTR).
- It can (often) involve ML DevOps Engineers (performing ML DevOps tasks).
- It can be supported by a ML DevOps System (that may be based on a ML DevOps platform).
- …
See: Deployable ML Model, ML Model Monitoring.

References

https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
- QUOTE: This document is for data scientists and ML engineers who want to apply DevOps principles to ML systems (MLOps). MLOps is an ML engineering culture and practice that aims at unifying ML system development (Dev) and ML system operation (Ops). Practicing MLOps means that you advocate for automation and monitoring at all steps of ML system construction, including integration, testing, releasing, deployment and infrastructure management.
  Data scientists can implement and train an ML model with predictive performance on an offline holdout dataset, given relevant training data for their use case. However, the real challenge isn't building an ML model, the challenge is building an integrated ML system and to continuously operate it in production. With the long history of production ML services at Google, we've learned that there can be many pitfalls in operating ML-based systems in production. Some of these pitfalls are summarized in Machine Learning: The high-interest credit card of technical debt.
  As shown in the following diagram, only a small fraction of a real-world ML system is composed of the ML code. The required surrounding elements are vast and complex.