Skip to content

Latest commit

 

History

History
54 lines (40 loc) · 1.91 KB

tf_serving_gpu.md

File metadata and controls

54 lines (40 loc) · 1.91 KB
Raw

Serving an object detection model with GPU

Setup

If you followed previous steps to train the model, skip to deploy section.

Alternatively, you can do the following to deploy kubeflow and get a model.

Reference blog

Deploy Kubeflow

Follow getting started guide to deploy kubeflow.

Prepare model

Download a model from model zoo. The model should be in SavedModel format (including a saved_model.pb file and a optional variables/ folder.

wget http://download.tensorflow.org/models/object_detection/faster_rcnn_nas_coco_2018_01_28.tar.gz
tar -xzf faster_rcnn_nas_coco_2018_01_28.tar.gz
gsutil cp faster_rcnn_nas_coco_2018_01_28/saved_model/saved_model.pb gs://YOUR_BUCKET/YOUR_MODEL/1/

Or you can use the above model uploaded at gs://kubeflow-examples-data/object-detection-coco/.

Deploy serving component

After deploying Kubeflow, you should have a ksonnet app; cd to that directory.

cd YOUR_KS_APP  # you can use the ks-app in this dir.
ks pkg install kubeflow/tf-serving  # If you haven't done it

ks generate tf-serving model1 --name=coco
ks param set model1 modelPath gs://YOUR_BUCKET/YOUR_MODEL/
ks param set model1 numGpus 1
ks param set model1 deployHttpProxy true
ks apply $ENV -c model1

Send prediction

cd serving_script
python predict.py --url=YOUR_KF_HOST/models/coco

If you expose the TF Serving service as a LoadBalancer, change the url to EXTERNAL_IP:8000/model/coco:predict

The script takes an input image (by default image1.jpg) and should save the result as output.jpg. The output image has the bounding boxes for detected objects.