Run a sample application

The following steps describe how to deploy a sample Cloudflow call-record-aggregator new tab application to the cluster. This sample aggregates streaming data for phone call records and contains both Akka and Spark-based streamlets.

Prerequisites

With Cloudflow installed on a GKE cluster through the Marketplace, you can interact with Cloudflow via its CLI: kubectl cloudflow. If you haven’t already, download and set up the CLI.

1. Connect to the cluster

From the Clusters view, find the command to connect your local kubectl to the GKE cluster on which Cloudflow is installed:

Connect kubectl to GKE cluster

The command includes the following, where cluster-name refers to your Kubernetes cluster, as identified here new tab, and gcp-project refers to your Google Cloud project new tab:

$ gcloud container clusters get-credentials <cluster-name> --zone <zone> --project <gcp-project>

2. Build and deploy call-record-aggregator

  1. Clone the Cloudflow Github repo new tab and in a command window cd into the examples/call-record-aggregator directory.

  2. You will need a file named target-env.sbt containing the Docker registry and repository that application images will be pushed to. Refer to target-env.sbt.example in the root sample directory for a template. For example, if you are using Google Container Registry (GCR), your target-env.sbt should look like this, where region is us, eu, asia, or empty. If the region is empty, omit the first dot:

    ThisBuild / cloudflowDockerRegistry := Some("<region>.gcr.io")
    ThisBuild / cloudflowDockerRepository := Some("<gcloud-project-id>")
  3. Log in to the Docker registry specified in target-env.sbt.

    For most registries, docker login will work. For GCR, you can also run gcloud auth configure-docker.

  4. Run the following command to build the application:

    $ sbt buildApp

    This step may take several minutes to complete, as it will build and push multiple Docker images to the Docker repository configured in target-env.sbt. Once it’s complete, you should see at the end of the output a kubectl cloudflow deploy command to deploy your application.

  5. Run the kubectl cloudflow deploy command:

    $ kubectl cloudflow deploy call-record-aggregator.json
    You might be prompted to enter credentials for the Docker registry that hosts your application images, if this is your first time deploying an application using that registry.

Once the deployment of the application starts, you can view the progress of the application using the kubectl cloudflow status command:

$ kubectl cloudflow status call-record-aggregator
Name:             call-record-aggregator
Namespace:        call-record-aggregator
Version:          473-d654162-dirty
Created:          2020-06-23 10:17:00 -0400 EDT
Status:           Running

STREAMLET         POD                                                        READY             STATUS            RESTARTS
cdr-aggregator    call-record-aggregator-cdr-aggregator-driver               1/1               Running           0
cdr-aggregator    call-record-aggregator-cdr-aggregator-1592921825294-exec-1 1/1               Running           0
cdr-generator1    call-record-aggregator-cdr-generator1-driver               1/1               Running           0
cdr-generator1    call-record-aggregator-cdr-generator1-1592921825612-exec-1 1/1               Running           0
cdr-generator2    call-record-aggregator-cdr-generator2-driver               1/1               Running           0
cdr-generator2    call-record-aggregator-cdr-generator2-1592921825136-exec-1 1/1               Running           0
cdr-ingress       call-record-aggregator-cdr-ingress-665b79f7b5-b8k6g        1/1               Running           0
console-egress    call-record-aggregator-console-egress-dd8667b84-lv7tm      1/1               Running           0
error-egress      call-record-aggregator-error-egress-87574b5c4-w7mqs        1/1               Running           0
split             call-record-aggregator-split-54f5844cf5-z8fp2              1/1               Running           0

3. Using Lightbend Console

Lightbend Console is browser-based tool for observing and monitoring Kubernetes applications. To monitor the call-record-aggregator application that you just deployed, follow these steps:

  1. Run kubectl port-forward on the console-frontend deployment:

    $ kubectl -n <cloudflow-namespace> port-forward deployment/console-frontend 8080:8080
    For this example, we simply opened a port. In a production environment, setting up Ingress is the recommended way to open access to Console, as described in the Lightbend Console documentation new tab.
  2. Open http://127.0.0.1:8080/ in a browser and you should see the following page:

    Lightbend Console
    Figure 1. Lightbend Console

    This overview shows the workloads running in the cluster.

  3. Click the Cloudflow logo in the Controls panel on the upper left corner of the screen to see a list of Cloudflow applications that you have deployed:

    List of Deployed Cloudflow Applications
    Figure 2. List of Deployed Cloudflow Applications

    Currently only call-record-aggregator is running.

  4. Click call-record-aggregator to see a detailed view of the streamlets:

    call-record-aggregator
    Figure 3. Call Record Aggregator View
  5. Click one of the streamlets and you should see that the data is being processed in the throughput window on the right.

This completes the steps to run a sample Cloudflow app and Lightbend Console on GCP Marketplace. If you have additional questions, please refer to the FAQ new tab.

4. What’s next

For more detailed information about Lightbend Console, refer to its documentation new tab. For information on how to develop Cloudflow applications, see the Cloudflow documentation new tab.