When the new data deviates from the original trained data that the model was trained, the model performance deteriorates. This concept, known as model drift, can be mitigated by retraining the model when new data becomes available, to reflect the current reality.
In Azure DevOps, you can retrain the model on a schedule or when new data becomes available. The machine learning pipeline orchestrates the process of retraining the model in an asynchronous manner. A simple evaluation test compares the new model with the existing model. Only when the new model is better does it get promoted. Otherwise, the model is not registered with the Azure ML Model Registry.
- To retrain your model, update the training code with new data.
- Pull up the training code in
transactions_arima.pyinscripts/training/folder. The initial model was trained on 2013 transactions data. - Along with 2013 data, also read 2014 transactions data that was created in
service/code/AcquireData.py. - Concatenate these two datasets/dataframes and build an ARIMA trained model on this bigger dataset.
- Pull up the training code in
- Re-run the
Buildpipeline to reflect the changes in training. - Re-run the
Releasepipeline. If the new model has better evaluation metrics than the previous model, then a new web service is created for your retrained model. - Review artifacts and outputs from
BuildandReleasepipelines.
- A retrained model (if necessary with better performance) is created and registered within the Azure ML Model Registry.
- A container image for your retrained model is created under Azure ML Images.
- A “healthy” ACI deployment for your retrained model is created under Azure ML Endpoints.
- MLOps documentation: Model management, deployment, and monitoring with Azure Machine Learning
- MLOps Reference Architecture
You have finished the challenges for this Hack!