What Is Machine Learning Ops?
Machine Learning Ops encompasses the standards, tools, and techniques used to automate and streamline many aspects of machine learning, including the creation, training, running, and deployment of AI models. Machine learning ops automates parts of these tasks, making the production of machine learning systems simpler and more reliable.
Machine learning Ops started as a set of standards intended to help DevOps teams and data scientists better communicate and collaborate. However, it has since expanded to cover the tools and techniques used to manage the life-cycle of machine learning and data science tasks. Beyond this, MLOps also encompasses aspects of regulation and business strategies.
Machine learning Ops can essentially be divided into three components:
- Machine Learning
- DevOps (IT)
- Data Engineering
The machine learning aspects of MLOps include tools to manage the creation and registration of AI models, the training of models, and validation of AI models.
The DevOps aspects of Machine Learning Ops include the planning of AI systems, the integration of AI models into other systems, the deployment of AI models, and the continuous maintenance and management of models.
Finally, the data engineering aspects of Machine Learning Ops includes the collection of data, the ingestion of data into the ML system, the standardization/preparation of data, and the validation of data.
Benefits of MLOps
The use of machine learning ops brings many benefits to the companies that employ it. MLOps facilitates:
- Open communication
- Reusable workflows
- Regulatory compliance
- Bias reduction
- Precise feedback
When properly implemented, MLOps reduces friction and miscommunication between operations teams and data science teams. When complicated machine learning models are siloed in development, bottlenecks can arise between machine learning engineers and other teams. The open communications promoted by MLOps help reduce these bottlenecks.
MLOps lets development teams standardize workflow and reuse development strategies. This greatly simplifies and streamlines changes to models, as these standards can be reused to iterate upon previously deployed models. Thanks to these reusable workflows, models and updates can make their way through a development pipeline in a fast, automated fashion.
MLOps also aids in ensuring regulatory compliance. As guidelines for the deployment of machine learning models become more stringent, it becomes more important to make sure that every model deployed is in compliance with those guidelines. MLOps allows you to use reproducible models based on consistent standards that follow compliance guidelines.
Bias in machine learning models can lead to certain groups being under-represented when it comes to the model’s decisions, or even make a company open to legal scrutiny. MLOps can keep certain biases in check by preventing certain features from eclipsing the influence of other features. MLOps can hold features in check as the models evolve and respond to shifts in data.
MLOps helps you decipher the cause of alerts when anomalies are detected, delivering precision feedback when you have “alert storms”. Errors in machine learning deployment or operation can often cause a cascade of alerts that can be hard to troubleshoot, but MLOps tools can help you analyze the problem and find the source of the errors.
How Machine Learning Ops Works
Much like DevOps, Machine Learning Ops functions under a CI/CD framework. Code is developed and updated continuously, just like other DevOps schemes. The difference is that the code in question is focused on machine learning algorithms and data science techniques. The continuous integration stage typical of CI/CD pipelines has additional considerations in MLOps. In MLOPs, continuous integration includes the validation of data and models. Meanwhile, continuous delivery must respond to the complexities of deploying a machine learning model. Ultimately, CI/CD for Machine Learning Ops combines elements of code, data, and models to ensure the creation and continual updating of predictive systems.
While Machine Learning Ops possess a CI/CD framework like DevOps, it differs from DevOps by requiring a Continuous Training framework as well. Continuous Training endeavors to automate the model training process, programmatically handling the integration and delivery of data to a model, the training of that model, and the monitoring of the model’s performance in a production environment. A continuous training schema is able to update models when their performance begins to decay.
Overall, the Machine Learning Ops workflow can be divided into six main, repeating steps:
- Model training pipeline
- CI/CD Orchestration
The model training pipeline is where the data to be fed into the model is ingested. This is where ETL (Extract, Transform, Load) operations are carried out to prepare the data for use. Data from external repositories and databases is brought into the training platform and saved as features. Data engineering tools are used to standardize the data, clean the data, and make any necessary transformations, such as engineering new features. The data is also separated into training and validation datasets here. After the data has been readied, the model itself is trained.
Model validation is when the trained model is tested against a dataset the model hasn’t seen before. This helps ensure the model is ready for deployment. Statistical tests are carried out on the model and it is provided with relevant segments of the data that mimic the properties of data in the operational environment. The data training data also has to be validated to ensure that it is a good match for deployment data.
Serving is the process of launching models into a production environment. Models can be launched to an IoT device, to a dedicated web service available over a remote procedure call (RPC), or made available via REST APIs. Making the model available for access as a web service is a popular method of deploying machine learning models, as this decouples the machine learning model from other software and allows the model to be accessed by a variety of consumer applications.
Monitoring a deployed machine learning model involves using a variety of various metrics and tools to detect aberrant behavior in the model. A model’s performance can be affected by a variety of different variables, from environmental changes to a mismatch in training and live data. If a model’s accuracy suddenly degrades or falls below a certain threshold, the model should be retrained. Software solutions intended to identify anomalies are used to constantly monitor a deployed model and alert developers or clients to a potential issue with a model.
Versioning in MLOps is much like versioning in traditional DevOps. Using versions to control how software is deployed and updated helps you roll out new features, ensures that you always have a working product, and provides a place to fall back to if a production error occurs. Unlike in DevOps however, MLOps requires that you also track different versions of the ML model, the training data for the models, and meta-data like hyper-parameter values.
CI/CD Orchestration is used to automate the execution of the many tasks involved in building and deploying a machine learning model. CI/CD tools can be used to orchestrate aspects of the training pipeline, validate models, and deploy/update models. Most MLOps solutions are capable of integrating with the mainstream CI/CD platforms like Git.
MLOps Best Practices
When using MLOps, there are a handful of best practices you should follow to make your ML Ops strategy useful and reliable.
One best practice for MLOps should take effect even before any development has begun. You should always be thinking ahead.
Before any work begins on the machine learning model, you should plan out the logistical/architectural aspects of your machine learning project. Give consideration to how the scope of the AI project might benefit from different data storage, warehousing, integration and analysis platforms. Consider variables like how large the dataset could be, how much data preparation will be needed, how models will be validated and updated, and what kinds of analytical tools you will need.
For data storage, solutions like Google Cloud Storage, Azure Blob or Amazon S3 are commonly used. Cloud Dataflow is a common tool for data processing and streaming, while Big Query is able to handle analytics for big datasets.
Your needs will affect the kinds of cloud services and automation tools you should invest in. Also give considerations to how you will handle errors in your production model and when models should be retrained. The more planning you do before development commences, the less time you will have to spend handling edge cases and unexpected problems.
Design ML Models Carefully
When designing your machine learning model, give consideration to the ML framework of your choice and which deployment options the framework supports. Common ML frameworks include Tensorflow, Scikit-learn, and PyTorch, and they are easily integrated with most cloud deployment solutions. Many machine learning APIs exist that offer low-code or no-code plug-and-play machine learning models. These machine learning APIs make creating your own ML model much easier, but be sure that they integrate with the rest of your MLOps framework. Thinking ahead will let you determine the best model framework for you.
Monitoring and Retrain Your Models
After a model has been deployed, it needs to be monitored and maintained. The developers responsible for designing the model should provide maintenance instructions that an MLOps team can follow as they monitor the performance of the model. Baseline performance levels and key metrics should be defined by the engineering team and the MLOps team can monitor these model aspects either manually or automatically. Automatic monitoring is accomplished through the use of alert systems that are triggered when a metric rises above, or falls below, the defined threshold. These automatic checks can be scheduled to run at certain intervals (every hour, day, etc.). Manual checking involves having someone run the model and check the outputs themselves. A combination of automatic and manual checking is typically best.
If the model’s performance degrades, or some other issue is spotted, it will need to be retrained. You may also want to retrain a model even if there are no obvious problems, but you want to include new training data. A CI/CD pipeline can ensure that new data is served to a model, and the model retrained, according to a schedule. The pipeline will ensure that whenever a model is changed or a new model is created, it goes through a validation and testing phase.
A Multi-Tier Response to Incidents
If your monitoring system catches an error or degradation in performance, you can use a triage system to control how incidents associated with the model are handled.
For example, a three-tiered error-handling system can be used to solve an issue. Level one of the response consists of an assessment by the MLOps team that monitors the model. They will try a series of standard troubleshooting solutions, such as running certain scripts and checking for consistency between data and model versions.
If the MLOPs team cannot solve the issue, data scientists can take over. This is typically a team concerned only with responding to problems in deployed models and they will examine the model at a deeper level. The data science team may make changes to the distributions of features, create new features, or remove outliers in data.
Finally, if the second-level team can’t solve the issue, the problem is brought to the development team responsible for creating the model. The development team will solve the issue by tweaking the model or changing how the data is preprocessed. Ultimately, the development team should only be brought in when an issue can’t be solved by other teams, as this keeps them focused on the creation of new features and models.
Machine Learning Ops is a series of tools, standards, and techniques used to automate and standardize the creation, training, deployment, and maintenance of machine learning models. It includes aspects of data engineering, DevOps, and data science.
MLOps benefits companies by promoting open communication, lets development strategies be reused, ensuring regulatory compliance, reducing bias, and enabling precision feedback. MLOps is concerned with the continuous deployment and integration of the model training pipeline, the model validation process, model serving, the monitoring of metrics, version control, and general CI/CD orchestration.
When you implement a MLOps strategy, you should always be thinking ahead and planning out the design of your machine learning models. Giving careful consideration to your model’s needs, how you’ll monitor and retrain the model, and how you’ll handle errors and anomalies will help you get the most out of MLOps.