Kubeflow Pipelines

One Convergence™ has sponsored several chapters in the book Kubeflow for Machine Learning, and we provide instructions on how to execute some of the examples in the book on the DKube™ platform. The book can be obtained from www.dkube.io/ebooks/.

This example uses a simple pipeline to show how to compile and execute a Kubeflow PIpeline within DKube. The instructions on how to execute the DKube-specific actions are available in the DKube User Guide. The steps to use this example are as follows:

1. Create a Code repo as described in the DKube User Guide with the following fields:

Other fields can be left at their default value

2. Create and open a JupyterLab IDE as explained in the DKube User’s Guide with the following fields:

Name KF-pipeline
Code Source Use the Code repo created in step 1

Other fields can be left at their default value

3. Navigate to the folder workspace/KF-pipeline/ch04/code

   There are 2 ipynb examples that can be modified to work within DKube

a. ControlStructures.ipynb

   Replace the contents of cell 4 with:

import os
existing_token = os.getenv("DKUBE_USER_ACCESS_TOKEN")
client = kfp.Client(existing_token=existing_token)
client.create_run_from_pipeline_func(conditional_pipeline, arguments={})

b. Lightweight Pipeline.ipynb

   Replace the following line in cell 8:

client = kfp.Client()

with:

import os
existing_token = os.getenv("DKUBE_USER_ACCESS_TOKEN")
client = kfp.Client(existing_token=existing_token)


4. In each case, run all of the cells in the notebook

5. This will start a Kubeflow Pipeline run within DKube. The run can be viewed within the DKube UI on the PIpelines screen.