Rate this post

Deployment of Machine learning Models Demystified (Part 1)

How to design an algorithm to predict loan default rates

This is the beginning of a two-part series. The second part can be read after this.

Imagine building a supervised machine learning model to decide if a loan application should be approved. The model confidence level can be used to calculate the risk-free loanable amount. The goal of this project is to deploy such a model.

Join me as I walk you through my internship project at Data Science Nigeria where I have been able to work alongside some of the best Data Scientists, Software Engineers and Researchers in the world. The goal is to raise 1 million Ai talent in 10 years.

Data scientists build models in jupyter lab, but machine learning engineers build models in production.

The integration of the model into an existing production environment can take in an input and return an output that can be used in making practical business decisions.

This post is divided into four parts to help you understand the deployment process. In chapter 1, we cover the process of building a baseline model and in chapter 2, we use the model in production ready code. Part 2 of this series covers Chapters 3 and 4.

Chapter 1 builds a base-line model.

A base-line model is one that has the minimum number of features but has good evaluation measures. For deployment purposes, you must always have your end- users in mind. A model with thousands of features might not be good enough for deployment.

  • Software can be transferred from one machine to another. A portable model decreases the response time of your code and the amount of code you have to rewrite when your goals change.
  • Is a program able to scale ? A model does not need to be changed to maintain effective performance.
  • The deployment of models to predict the target value of a classification/regression problem is referred to as operationalization.
  • Refers to the validation of output and processes. It will be difficult to test a robust model.

Click here to download the dataset that will be used for this project, the notebook and the exclusive summary that contain Exploratory Data Analysis, Data Preprocessing, Feature Engineering, Feature Selection, Training the ML model and Model Evaluation.

pandas are used to import our test and training data.

The train is pd. “ train ” is read in thecsv. The test has a pd. “ test ” is read_csv

Data preprocessing is the transformation of raw data into an understandable format to train a model. It involves taking care of missing values.

For the purpose of deployment, you want to always keep a record of the values you are using to fill the missing values and the approach has to be replicable. The code snippets below show features with missing values.

The horizontal white stripes indicate the locations of missing values.The horizontal white stripes indicate the locations of missing values.

There are many ways to take care of the missing values in numerical features, but to keep things simple, I will experiment with the mean and median.

For each of the numerical features with missing values, you calculate the mean and median and fit a model on the data to see the effect on the validation sets. We can experiment with a simple python function that takes in the dataframe, variable of interest, mean and median and fills the missing parts of the features with the mean and median.

split into training and testing sets and defined a function to fill in missing values

A simple feature engineering that can be done when filling missing values is to create a feature that tells you if the missingness has a positive predict effect or not.

Adding missingness features.

There is a variable indicating missingness in the application. Where ( X_train ) ? The person has an income. X_test [ ‘applicant_income_NA ‘ ] is null. Where ( X_test ) ? The person has an income. Isnull

Logistic regression model can be used to determine the best approach to filling missing values

kindly download the notebook to see the full code snippetsthe output of the code snippet above

The best decision is made based on the model outcome for features with missing values.

The process of using domain knowledge of the data to create features that make machines work is called feature engineering. The income_loan_ratio is the only feature I will create.

The process of selecting features which contribute most to your prediction variable is called feature selection. The accuracy of the models can be decreased by having irrelevant features in your data.

The final model has several methods of selecting features.

Constant features elimination, quasi-constant features elimination, duplicate feature elimination, fisher score, univariate method, mutual information, correlation are some of the filter methods.

Wrapper methods include step forward selection, step backward selection and exhaustive search.

Embedded methods include LASSO, Decision tree derived importance and Regression coefficients.

Training the machine learning model is supervised and it is a classification model. We need to separate the features from the target variable before training the model.

The experiment explored the use of xgboost, Microsoft, and GradientBoostingClassifier, but 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884 888-349-8884

kindly check the notebook for the output

Evaluation of model

The accuracy on test data is 70 from the output of the trained model. 3 % with ROC-AUC of 77 70 % accuracy is good enough for the problem case scenario since the goal of the project is to predict acceptance status of mortgage applications and safe amount on accepted loan applications. The model can be improved later, but we want to make deployment as simple as possible for now.

Chapter 2 is about writing production ready codes.

According to ISO/IEC 42010, architecture can be defined as the way software components are arranged and the interactions between them.

The way in which the software components that makeup system are arranged and the interaction between them achieve a preset goal is referred to as the ML system architecture. The development and deployment of models is cheap and fast but it is difficult to maintain over time. Depending on your company ‘s capacity and goal, choosing the best architecture might be a bit challenging.

This is the beginning of a two-part series. The second part can be read after this.

The following should be considered when designing a system architecture.

  • The code used in preprocessing/feature engineering should be arranged.
  • The output of each component must be replicable.
  • The model needs to be serveable to a large number of customers.
  • It should be easy to modify future tasks.
  • The ability to test variation between model versions is called testing.
  • Eliminate manual steps wherever possible to reduce error chances.

There is an overview of the architectures.

  • Training and persistence are done offline while prediction is done in real-time.
  • Train by batches, predict by batches, serve through a shared database, while predictions are done in a distributed queue which is almost similar to a real-time prediction.
  • Train, predict by streaming, both training and predicting are done on different streams.
  • Train by batches, predict on mobile, similar to type 1 but the prediction is made on customer gadgets.

There is a comparison of architecture.

The predictions can be single or in batches. We will expose the predict function via flask and uploaded our test batches for prediction.

Source: Building a Reproducible Machine Learning Pipeline, Surgimura and Hart

The whole system is divided into two parts, development and production. Offline work is done on the Training Data, Feature Extractor and Model Builder section.

Training Phase

The unit deals with the data to be used in training the model into the training environment and how to format it for training. Depending on the architecture you are working on, this unit can be simple or complex. For the purpose of this project, a simple python script will be written to fetch the dataset already downloaded into a folder/directory to the training environment and then passed through a data pipeline.

Features Extractor is the unit that selects/generates essential features needed in training the model. This is part of the process. The only library that can be used to achieve the same task is the sklearn library.

The unit where serializing and persisting of the trained models is called Model Builder. This can be the unit that involves in package building using setuptools or simply saving the model in a pickle format depending on the architecture you adopt. The differences between these approaches will not be discussed in this article. I will be sharing the latter approach so as to keep things simple.

The output of the three sub-topics discussed above can be easily deployed via a trained model.

Prediction Phase

This is the beginning of a two-part series. The second part can be read after this.

In production, a request is sent in batches, cleaned and preprocessed, then a trained model is used to predict if the loan application is accepted or not. Before returning the prediction through our end-point, the accepted loans are analysed for risk-free loanable amount.

In Part 2, I will discuss how to convert the notebook into production ready-code and how to use Postman to test the end-points.

This is the beginning of a two-part series. The second part can be read after this.

You can clone the repository using the link below.

They are wrapping up.

There are many ways to deploy models into production, many ways to store them, and different ways to manage them after deployment. Choosing the best approach to a use case can be challenging but with the team ‘s technical and analytics maturity, the overall organization structure and its interactions can help in selecting the right approach for deployment.

You can connect with me on social media.

Check out my other articles and read further in the reference section. To get notified of my publications, please follow me.

You can checkout NeptuneAi post for other approaches to model deployment strategies.

There are deployment strategies for NeptuneAi. There are model deployment-strategies.

There are references.

  • There are different approaches to putting Machine Learning models in production.
  • Mark Hornick is the author of the Data Science Maturity Model.
  • The deployment of machine learning models was discussed.

Special thanks to my mentors. The entire staff of Data Science Nigeria is led by James Allingham.

10 years is the number one million Ai talent.

Cheers!

Source: https://nhadep247.net
Category: Machine