Machine Learning (ML) Model Deployment System
A Machine Learning (ML) Model Deployment System is a software deployment system that can solve an ML model deployment task (to convert a trained model into a deployed ML model).
- Context:
- It can range from being an Online ML Model Deployment System to being an Offline ML Model Deployment System.
- It can be based on a ML Model Deployment Platform, such as MLflow and Seldon.
- …
- Example(s):
- a Seldon.
- a Kubeflow.
- a MLCapsule System?
- …
- Counter-Example(s):
- See: Machine Learning System, Software Deployment, Software Development System, Application Programming Interface.
References
2018a
- (Lai & Suda, 2018) ⇒ Liangzhen Lai, and Naveen Suda. (2018). “Rethinking Machine Learning Development and Deployment for Edge Devices.” arXiv:1806.07846
- QUOTE: Current ML development and deployment approach is shown in Figure 1. Application developers use the ML frameworks to construct their NN models, perform training and evaluate the accuracy. Based on the evaluation results, they can go back and refine or optimize their models. After this development stage, a trained model is generated by the framework. The deployment tool, typically offered by the deployment platform vendor, takes this trained model as the input and constructs a deployable solution that runs on the target platform.
Figure 1: Current ML development and deployment approach. The shaded blocks are vendor-specific parts.
(...) we proposed the ML development and deployment approach shown in Fig. 2. Unlike current approach, the users, which are the solution developers, will specify their network models using the deployment tool with an operator library from the deployment platform vendor. The deployment tool will generate a deployable but untrained solution, i.e., a model that can run on their platform, but without the trained parameters. As part of the deployment tool, the vendor should also be responsible to create an operator model library that can represent their operators and be used to create a trainable model inside the ML framework. This trainable model should have a forward inference path that behaves exactly the same as the implemented operators, and a backward path for the ML framework to perform training in order to generate the appropriate weights and parameters. In this approach, the validation results from the ML framework will be the same as the deployment results, so that the user can use these results to refine or further optimize their network models.
Figure 2: Proposed ML development and deployment approach. The shaded blocks are vendor-specific parts.
- QUOTE: Current ML development and deployment approach is shown in Figure 1. Application developers use the ML frameworks to construct their NN models, perform training and evaluate the accuracy. Based on the evaluation results, they can go back and refine or optimize their models. After this development stage, a trained model is generated by the framework. The deployment tool, typically offered by the deployment platform vendor, takes this trained model as the input and constructs a deployable solution that runs on the target platform.
2018b
- (Hanzlik et al., 2018) ⇒ Lucjan Hanzlik, Yang Zhang, Kathrin Grosse, Ahmed Salem, Max Augustin, Michael Backes, and Mario Fritz. (2018). “MLCapsule: Guarded Offline Deployment of Machine Learning As a Service.” arXiv:1808.00590
- QUOTE: Machine learning as a service (MLaaS) has become increasingly popular during the past five years. Leading Internet companies, such as Google[1] Amazon[2] and Microsoft[3] have deployed their own MLaaS. It offers a convenient way for a service provider to deploy a machine learning (ML) model and equally an instant way for a user/client to make use of the model in various applications. Such setups range from image analysis over translation to applications in the business domain.
2018c
- (Wei & Cai, 2019) ⇒ Kewei Wei, and Guanjun Cai (2018) "Bring Intelligence to Where Critical Transactions Run–An Update from Machine Learning for z/OS". Released: June 7, 2018 08:12 AM, Updated: June 7, 2018 08:12 AM
- QUOTE: To support such a broadscale collaboration, your machine learning platform must have a well-defined machine learning workflow, from data ingestion and model training to model deployment, as shown below.
- QUOTE: To support such a broadscale collaboration, your machine learning platform must have a well-defined machine learning workflow, from data ingestion and model training to model deployment, as shown below.
2016
- (Tramer et al., 2016) ⇒ Florian Tramer, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. (2016). “Stealing Machine Learning Models via Prediction APIs.” In: Proceedings of the 25th USENIX Security Symposium. ISBN:978-1-931971-32-4. arXiv:1609.02943
- QUOTE: Machine learning (ML) aims to provide automated extraction of insights from data by means of a predictive model. A predictive model is a function that maps feature vectors to a categorical or real-valued output. In a supervised setting, a previously gathered data set consisting of possibly confidential feature-vector inputs (e.g., digitized health records) with corresponding output class labels (e.g., a diagnosis) serves to train a predictive model that can generate labels on future inputs. Popular models include support vector machines (SVMs), logistic regressions, neural networks, and decision trees.
ML algorithms’ success in the lab and in practice has led to an explosion in demand. Open-source frameworks such as PredictionIO and cloud-based services offered by Amazon, Google, Microsoft, BigML, and others have arisen to broaden and simplify ML model deployment.
- QUOTE: Machine learning (ML) aims to provide automated extraction of insights from data by means of a predictive model. A predictive model is a function that maps feature vectors to a categorical or real-valued output. In a supervised setting, a previously gathered data set consisting of possibly confidential feature-vector inputs (e.g., digitized health records) with corresponding output class labels (e.g., a diagnosis) serves to train a predictive model that can generate labels on future inputs. Popular models include support vector machines (SVMs), logistic regressions, neural networks, and decision trees.
1998
- (Kohavi & Provost, 1998) ⇒ Ron Kohavi, and Foster Provost. (1998). “Glossary of Terms.” In: Machine Leanring 30(2-3).
- Model deployment: The use of a learned model. Model deployment usually denotes applying the model to real data.