Model Inference
One Convergence™ has sponsored several chapters in the book Kubeflow for Machine Learning, and we provide instructions on how to execute some of the examples in the book on the DKube™ platform. The book can be obtained from www.dkube.io/ebooks/.
This example takes several trained models and explains how to serve them within DKube. The instructions on how to execute the DKube-specific actions are available in the DKube User Guide. The general setup instructions are the same regardless of which model is being served
1. Create a new model repo as explained in the DKube User Guide. The specific inputs for each type of model are shown in the table below. For all of the examples, the following fields are used:
| Name | From table |
| Model Source | GCS |
| Bucket | kfserving-samples |
| Prefix | From table |
2. Add Model
3. Choose the Model name that was given when creating the Model repo, and select “Test Inference” as described in the DKube User’s Guide
The Serving Image is selected from the table:
| Framework | Name | Prefix | Serving Image |
|---|---|---|---|
| TensorFlow | Serving-TF | models/tensorflow/flowers | ocdr/tensorflowserver:1.14 |
| Pytorch | Serving-PT | models/pytorch/cifar10 | ocdr/pytorchserver:1.6 |
| Scikit Learn | Serving-SK | models/sklearn/iris | ocdr/sklearnserver:0.23.2 |
Other fields can be left at their default value
4. The served model will show up in the “Test Inferences” menu screen.