Deploy the Example to a Kubernetes Cluster

Optionally, you can now deploy the example to a Kubernetes cluster. If you have not already installed Cloudflow on a Kubernetes Cluster, please install Cloudflow first.

Once the application is built you need to publish the docker images to a container registry that stores, manages and secures your application images. In this example we will use a Docker Hub registry.

Follow these steps to complete the process of publishing:

  1. Set up the target environment of publishing in your local build process. Create a file named target-env.sbt in the root of your project folder (the same level where you have build.sbt). The file needs to have the following 2 lines:

    Configure your Docker repository
    ThisBuild / cloudflowDockerRegistry := Some("")
    ThisBuild / cloudflowDockerRepository := Some("<your docker hub username>")

    Replace <docker hub username> with your Docker Hub username. Publish the docker image of your project to the above registry.

  2. Make sure you are logged in to Docker Hub:

    docker login --username <your docker hub username>

  3. Build and publish the application with:

    'sbt buildApp'

Once the images are successfully published, you will see something like the following as output of buildApp in your console:

[info] Successfully built and published the following image:
[success] Cloudflow application CR generated in /.../sensor-data-scala/target/sensor-data-scala.json
[success] Use the following command to deploy the Cloudflow application:
[success] kubectl cloudflow deploy /path/to/sensor-data-scala/target/sensor-data-scala.json
[success] Total time: 41 s, completed Jun 10, 2020 10:10:10 PM

At this point, the Docker image containing the streamlets has been published to the Docker repository. The sensor-data-scala.json file that the build produces contains the application descriptor. We use the Cloudflow CLI to deploy that application description to a cluster.

Deploy Application to the Cluster

This is quite straightforward and the output of buildApp actually tells you what command to run for deployment. If your Docker Hub repository is private, you need to supply the credentials along with the CLI deploy command, so that Kubernetes can pull the images when the containers are created.

The following example reads a password from a file, and passes it to the kubectl cloudflow deploy command using STDIN:

$ cat my-dockerhub-password.txt | kubectl cloudflow deploy /path/to/sensor-data-scala/target/sensor-data-scala.json -u <your docker hub username> --password-stdin

If your docker repository is public, or if image pull secrets are already setup in your Kubernetes cluster to pull from your docker repository, you can omit credentials with the --no-registry-credentials flag, which is shown in the following example:

$ kubectl cloudflow deploy /path/to/sensor-data-scala/target/sensor-data-scala.json --no-registry-credentials

If the command goes through you will see the following output:

[Done] Deployment of application `sensor-data-scala` has started.

Once all streamlets are up in their pods, you can see them running using the following command:

$ kubectl cloudflow status sensor-data-scala
Name:             sensor-data-scala
Namespace:        sensor-data-scala
Version:          484-199a3c1-dirty
Created:          2020-06-10 22:28:32 +0200 CEST
Status:           Running

STREAMLET         POD                                              READY             STATUS            RESTARTS
http-ingress      sensor-data-scala-http-ingress-6f4b677c84-8gtk7  1/1               Running           0
invalid-logger    sensor-data-scala-invalid-logger-f76667866-lffmk 1/1               Running           0
metrics           sensor-data-scala-metrics-859d9cf974-vrpzs       1/1               Running           0
valid-logger      sensor-data-scala-valid-logger-6946746b4b-bpqdc  1/1               Running           0
validation        sensor-data-scala-validation-b7f85669d-kkpx9     1/1               Running           0

Note that all streamlets run in a namespace that matches the name of the application.

Congratulations! You have deployed and started your first Cloudflow application.

What’s next

Now, we are ready to Exercise the example.