Skip to content

Latest commit

 

History

History

ml_inference

README.md

License

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

     https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

License

How to make an ML model inference on KFServing from container apps (web, Spark) running on Google Cloud Kubernetes Engine?

You will learn how to take a pre-trained model, deploy it in KFServing, and infer the model from a Web Application and a Spark job. The article walks through a step by step approach from development to deployment of the opinionated solution.

The article uses a pre-trained model created in the How to build an end-to-end propensity to purchase solution using BigQuery ML and Kubeflow Pipelines.

Objectives

To help you be conversant on the following:

  1. Inspect a pre-built Machine Learning model signature (in development envrionment)
  2. Develop a Web App (HTML/CSS/JS for front end and Java for backend) (in development envrionment)
  3. Test the Web App (in development envrionment)
  4. Develop a Spark app (Scala) (in development envrionment)
  5. Test the scala app (in development envrionment)
  6. Setup Google Cloud Kuberentes Engine Cluster
  7. Deploy Spark, Web App, and KFServing in Google Kuberenetes Engine
  8. Test the Web App app running in the Google Kuberenetes Engine
  9. Submit the Spark job to the Spark running in the Google Kuberenetes Engine
  10. Setup all required softwares in development envrionment (TensorFlow, saved_model_cli, Docker, TFServing, Apache Tomcat, sbt, kubectl, scala, java)

Costs

This tutorial uses billable components of Google Cloud:

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

Before you begin

For this reference guide, you need a Google Cloud project.

You can create a new one, or select a project you already created. The following steps are required.

The git repo contents

The git repo contains all the commands and code to successfully build and deploy the end to end solution.

  1. Commands
  2. A pre-trained model
  3. Input file with sample data for testing
  4. The web app war file
  5. The Spark app jar file
  6. Code
    1. Java Servlet
    2. Scala
    3. HTML/CSS
    4. sbt build file
  7. GKE deployment files

Delete the GCP Project

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial is to Delete the project.

The easiest way to eliminate billing is to delete the project you created for the tutorial.

Caution: Deleting a project has the following effects:

  • Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
  • Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.

  1. In the Cloud Console, go to the Manage resources page. Go to the Manage resources page
  2. In the project list, select the project that you want to delete and then click Delete Trash icon.
  3. In the dialog, type the project ID and then click Shut down to delete the project.