Kubeflow Pipelines
One Convergence™ has sponsored several chapters in the book Kubeflow for Machine Learning, and we provide instructions on how to execute some of the examples in the book on the DKube™ platform. The book can be obtained from www.dkube.io/ebooks/.
This example uses a simple pipeline to show how to compile and execute a Kubeflow PIpeline within DKube. The instructions on how to execute the DKube-specific actions are available in the DKube User Guide. The steps to use this example are as follows:
1. Create a Code repo as described in the DKube User Guide with the following fields:
| Name | KF-pipeline |
| Code Source | Git |
| URL | https://github.com/intro-to-ml-with-kubeflow/intro-to-ml-with-kubeflow-examples/tree/master/ch04/code |
Other fields can be left at their default value
2. Create and open a JupyterLab IDE as explained in the DKube User’s Guide with the following fields:
| Name | KF-pipeline |
| Code Source | Use the Code repo created in step 1 |
Other fields can be left at their default value
3. Navigate to the folder workspace/KF-pipeline/ch04/code
There are 2 ipynb examples that can be modified to work within DKube
a. ControlStructures.ipynb
Replace the contents of cell 4 with:
import os
existing_token = os.getenv("DKUBE_USER_ACCESS_TOKEN")
client = kfp.Client(existing_token=existing_token)
client.create_run_from_pipeline_func(conditional_pipeline, arguments={})
b. Lightweight Pipeline.ipynb
Replace the following line in cell 8:
client = kfp.Client()
with:
import os
existing_token = os.getenv("DKUBE_USER_ACCESS_TOKEN")
client = kfp.Client(existing_token=existing_token)
4. In each case, run all of the cells in the notebook
5. This will start a Kubeflow Pipeline run within DKube. The run can be viewed within the DKube UI on the PIpelines screen.