Model Inference

One Convergence™ has sponsored several chapters in the book Kubeflow for Machine Learning, and we provide instructions on how to execute some of the examples in the book on the DKube™ platform. The book can be obtained from www.dkube.io/ebooks/.

This example takes several trained models and explains how to serve them within DKube. The instructions on how to execute the DKube-specific actions are available in the DKube User Guide. The general setup instructions are the same regardless of which model is being served

1. Create a new model repo as explained in the DKube User Guide. The specific inputs for each type of model are shown in the table below. For all of the examples, the following fields are used:

Name From table
Model Source GCS
Bucket kfserving-samples
Prefix From table

2. Add Model

3. Choose the Model name that was given when creating the Model repo, and select “Test Inference” as described in the DKube User’s Guide

   The Serving Image is selected from the table:

Framework Name Prefix Serving Image
TensorFlow Serving-TF models/tensorflow/flowers ocdr/tensorflowserver:1.14
Pytorch Serving-PT models/pytorch/cifar10 ocdr/pytorchserver:1.6
Scikit Learn Serving-SK models/sklearn/iris ocdr/sklearnserver:0.23.2

Other fields can be left at their default value

4. The served model will show up in the “Test Inferences” menu screen.