Installing DKube on Amazon EKS¶

DKube is a portable, end-to-end, Kubeflow-based MLOps platform that enables data scientists to develop, tune, and deploy complex models. It is based on Kubernetes, and will run on-premises and on the most popular cloud providers. It has the same look, feel, and workflow on all of them, and migrating back and forth between providers is fast and simple.
This page provides step-by-step instructions of how to install DKube on an existing Amazon EKS system.
Cluster Configuration¶
These instructions assume that an EKS cluster has already been installed, and DKube is being installed on that cluster. The cluster should consist of one or more master nodes, and optional worker nodes.
The Master nodes coordinate the cluster, and can optionally contain GPUs
Each Worker node provides more resources, and is a way to expand the capability of the cluster
Installation Configuration¶
The overall flow of installation is as follows:
Copy the Docker installation scripts and associated files to the installation node
Ensure that the installation node has passwordless access to all of the DKube cluster nodes
Edit the installation ini files with the appropriate options
Install DKube and its required software components
Access DKube through a browser
Prerequisites¶
Supported Platforms¶
The DKube installation scripts handle most of the work required to get DKube installed, including the installation of the software packages on the cluster. There are some prerequisites for each node, described below.
Node Requirements¶
Installation Node Requirements¶
The installation node has the following requirements:
Ubuntu 18.04
Docker CE
Docker Installation on Ubuntu¶
The following commands can be used to install Docker on Ubuntu:
DKube Cluster Node Requirements¶
The DKube Cluster nodes have the following requirements:
Ubuntu 18.04
Nodes should all have static IP addresses
Node names must be lower case
All nodes must be on the same subnet
All nodes must have the same user name and ssh key
Each node on the cluster should have the following minimum resources:
16 CPU cores
64GB RAM
200GB Storage
Access to the Cluster¶
In order to run DKube both during and after installation, a minimum level of security access must be provided from any system that needs to use the node. This includes access to the url in order to open DKube from a browser.
Protocol |
Port Range |
Source |
---|---|---|
TCP |
30002 |
Access IP |
TCP |
32222 |
Access IP |
TCP |
32223 |
Access IP |
TCP |
32323 |
Access IP |
TCP |
6443 |
Access IP |
TCP |
443 |
Access IP |
TCP |
22 |
Access IP |
All |
0-65535 |
Private Subnet |
ICMP |
0-65535 |
Access IP |
The source IP access range is in CIDR format. It consists of an IP address and mask combination. For example:
192.168.100.14/24 would allow IP addresses in the range 192.168.100.x
192.168.100.14/16 would allow IP addresses in the range 192.168.x.x
Note
The source IP address 0.0.0.0/0 can be used to allow access from any browser client. If this is used, then a firewall can be used to enable appropriate access.
Getting the DKube Files¶
The files necessary for installation, including the scripts, the .ini files, and any other associated files are pulled from Docker, using the following commands:
Note
The docker credentials and DKube version number (x.y.z) are provided separately
This will copy the necessary files to the folder $HOME/.dkube
dkubeadm |
Tool used to install DKube on the cluster |
k8s.ini |
Configuration file for Kubernetes cluster setup |
dkube.ini |
Configuration file for DKube installation |
Installation Setup¶
EKS Cluster Installation Files¶
When installing DKube on an existing EKS cluster, there are files required to set up the installation node and provide access to the EKS cluster.
The EKS preinstallation files are cloned from a GitHub repository.
Running the EKS Preinstallation Script¶
Navigate to the dkube-eks folder. Before running the preinstall script:
The [AWS] section of the terraform-eks.ini file needs to be completed to provide access to the EKS cluster
The following information needs to be completed in the terraform-eks.ini file:
aws_access_key_id
aws_secret_access_key
The rest of the fields are not used by the preinstall script, and can be left in their default state.
After running the preinstall script, the config file can be copied by including:
The region where the EKS cluster is running (e.g. us-west-2)
The EKS cluster name
The following commands will accomplish these steps:
The output from the “kubectl get nodes” command will provide the list of IP addresses and node names to use in the k8s.ini and dkube.ini files later in the installation procedure.
EKS Cluster Access¶
The section Getting the DKube Files provides instructions on getting the DKube installation files. The installation is performed from the $HOME/.dkube folder.
In order to set up the cluster access, all of the nodes in the EKS cluster must be identified in the k8s.ini file, as shown below. The following fields must be completed. The rest of the fields can stay at their default value.
Field |
Value |
---|---|
provider |
eks |
distro |
Ubuntu |
nodes |
IP addresses for EKS cluster |
user |
Username on EKS cluster |
Note
The IP addresses were obtained in the previous step with the “kubectl get nodes” command

In order to access the EKS cluster, the pem file from the AWS cluster is used. In order to use that file, the following steps should be followed:
Copy the .pem key to the $HOME/.dkube folder
Use the following commands to set up cluster access and install the necessary packages
DKube Installation¶
Editing the DKube ini File¶
Before installing DKube, the dkube.ini configuration file must be completed.

Field |
Value |
---|---|
KUBE_PROVIDER |
eks |
USERNAME |
User-chosen initial login username |
PASSWORD |
User-chosen initial login password |
Username and Password¶
This provides the credentials for initial DKube local login. The initial login user has both Operator and Data Scientist access.
The Username has the following restrictions:
Do not use the following names:
dkube
monitoring
kubeflow
Storage Options¶
The storage options are configured in the [STORAGE] section of the dkube.ini file.
Field |
Value |
---|---|
STORAGE_TYPE |
disk |
STORAGE_DISK_NODE |
EKS host name |
The “EKS Host Name” for each node in the cluster can be identified using the “kubectl get nodes” command. For this field, choose the node that will be used for DKube storage.

Cluster Access Options¶
Cluster access is configured in the [EXTERNAL] section of the dkube.ini file. The fields should be configured as follows:
Field |
Value |
---|---|
ACCESS |
nodeport |
INSTALL_LOADBALANCER |
false |

Node Setup¶
Before installing DKube, the appropriate software packages are required to be installed on each node on the DKube cluster. This is accomplished through the following command.
Completion of the Installation¶
This section describes the final installation procedure.
Installing DKube¶
Accessing the Installer UI¶
The installation progress can be viewed at the following url:
Accessing DKube¶
The DKube UI can be accessed from the installation dashboard, or accessed at the following url:
Initial Login¶
The initial login after installation is accomplished with the username and password entered in the dkube.ini file.
DKube Installation Failure¶
If the DKube install procedure detects that some of the prerequisites are not correct, the first troubleshooting step is to uninstall and cleanup the system. The command to uninstall and cleanup is:
After this successfully completes, run the DKube install command again at Installing DKube . If it still fails, contact your IT manager.
Uninstalling DKube¶
DKube can be uninstalled by following the steps in this section from the $HOME/.dkube directory: