4 ways to deploy your machine learning model
Deploying models to production means integrating them with existing products or workflows.
Once the team has built a model that has promising results, it is time that its outputs (e.g. predictions, classifications, or recommendations) are being used by consumers (e.g. customers or internal stakeholders).
This is what we refer to as model productionization, and it is a crucial step for businesses to get value from their data.
In this post, we will show how to determine which is the best strategy of deployment depending on the business use case.
In fact, these strategies were presented by the AI Guild practitioners during the online events of #datalift season 1. While the specific challenges may be unique to each organization, it is possible to abstract the methods from the concrete use cases in production across different business industries and domains of AI.
Watch the recorded live sessions in our #datalift playlist
Let’s get started!
Different use cases present different requirements for deploying machine learning models, such as:
Latency: How fast do the predictions need to be served to the consumers?
Context Sensitivity: Will we know the features ahead of inference time?
In the simplest case (bottom-left), the use cases are those in which model outputs are mostly used for offline decisions. This means the model can be deployed as a batch scoring job. One example is the pharmaceutical use case of bioactivity prediction for drug discovery presented by KNIME and Dr Janina Mothes at #datalift No 5. The input features for this type of use case vary from contextual or non-contextual. Again looking at the bioactivity prediction use case, the context is made of the compounds or small molecules that the chemists choose to upload to the model, that is, a static context for the model inference.
To illustrate a use case in which the context is changing, let's take a look into the MYTOYS product list sorting use case presented by Dr Sebastian Rose at #datalift No 2: for each product category that the visitor wishes to browse, the website loads a new product list. In this case, the context is the product category and the products that belong to it are pre-ranked and the list is stored for quick access by the frontend when the visitor clicks on it.
On the opposite corner (top-right) there are the models that are an integrated part of a product experience, and that the context is not available until a user interacts with the product. In such cases, results often need to be returned really fast, therefore they require online or real-time inference. This is clear in the financial use case presented by Vaibhav Singh at #datalift No 1: the payment solutions Klarna company uses ML to do credit risk and fraud risk assessments at scale. The features used by the model are related to the merchant and the customer, and also the transaction information like shopping cart, billing and shipping details.
In this child malnutrition detection use case that Markus Hinsche presented on #datalift No 2 the scenario is again of a context that is only available on request, that is, when the child is scanned via the mobile app. And in this case, the model's prediction is used to support the diagnosis of malnutrition that can trigger a helpful response from emergency workers as well as policy makers.
What #datalift achieves
We understand the current challenges and can assess data projects for technical quality, review the architecture, and determine the use case value.
Across more than ten industries, from leading corporates to leading startups, and on to new firms, we have heard from those having use cases in production. With a loyal audience of practitioners, of which more than half have roles as Seniors, Leads, Directors, and CxO, the #datalift event series from No 1 to 5 has become the key forum for exchange on what works and what is best practice.
Scaling solutions? Ask the AI Guild to help secure the quality and productivity of your deployed solutions. Book your conversation with us.