Exercising the deployed example application

Once you have the sensor-data Cloudflow application deployed, you can use some CLI helpers to monitor the status of your application.

  1. Try --help to see all options available.

    $ kubectl cloudflow --help
    This command line tool can be used to deploy and operate Cloudflow applications.
      cloudflow [command]
    Available Commands:
      configure                 Configures a deployed Cloudflow application.
      deploy                    Deploys a Cloudflow application to the cluster.
      help                      Help about any command
      list                      Lists deployed Cloudflow application in the current cluster.
      scale                     Scales a streamlet of a deployed Cloudflow application to the specified number of replicas.
      status                    Gets the status of a Cloudflow application.
      undeploy                  Undeploys a Cloudflow application.
      update-docker-credentials Updates docker registry credentials that are used to pull Cloudflow application images.
      version                   Prints the plugin version.
      -h, --help   help for cloudflow
    Use "cloudflow [command] --help" for more information about a command.
  2. Try list to see all applications deployed in the cluster.

    $ kubectl cloudflow list
    NAME              NAMESPACE         VERSION           CREATION-TIME
    sensor-data-scala sensor-data-scala 484-199a3c1       2020-06-10 22:28:32 +0200 CEST
  3. Try status to see details of a running application, for example:

    $ kubectl cloudflow status sensor-data-scala
    Name:             sensor-data-scala
    Namespace:        sensor-data-scala
    Version:          484-199a3c1
    Created:          2020-06-10 22:28:32 +0200 CEST
    Status:           Running
    STREAMLET         POD                                                 READY             STATUS            RESTARTS
    http-ingress      sensor-data-scala-http-ingress-86dd5b8747-vnmxq     1/1               Running           0
    invalid-logger    sensor-data-scala-invalid-logger-789d679855-9l7qz   1/1               Running           0
    metrics           sensor-data-scala-metrics-6b7475c4f9-sxxw6          1/1               Running           0
    valid-logger      sensor-data-scala-valid-logger-78594f47ff-v82zt     1/1               Running           0
    validation        sensor-data-scala-validation-d8858dff-6xswj         1/1               Running           0

Push data to the Application

Our application uses an http based ingress to ingest data. Follow these steps to push JSON data through the ingress into the application:

  1. Get the port details of our ingress streamlet:

    $ kubectl describe pod -n sensor-data-scala sensor-data-http-ingress-fd9cdb66f-jbsrm

    The results should look similar to the following:

    Name:               sensor-data-http-ingress-fd9cdb66f-jbsrm
    Namespace:          sensor-data-scala
    Priority:           0
    PriorityClassName:  <none>
    Node:               gke-dg-gke-1-default-pool-162a09d5-ddnq/
    Start Time:         Tue, 12 Nov 2019 12:47:20 +0530
    Labels:             app.kubernetes.io/component=streamlet
    Annotations:        prometheus.io/scrape: true
    Status:             Running
    Controlled By:      ReplicaSet/sensor-data-http-ingress-fd9cdb66f
        Container ID:  docker://9149cd757094e7ea1b943076048b7efc7aa343da8c2d598bba31295ef3cbfd6b
        Image:         eu.gcr.io/bubbly-observer-178213/sensor-data@sha256:ee496e8cf3a3d9ab71c3ef4a4929ed8eeb6129845f981c33005942314ad30f18
        Image ID:      docker-pullable://eu.gcr.io/bubbly-observer-178213/sensor-data@sha256:ee496e8cf3a3d9ab71c3ef4a4929ed8eeb6129845f981c33005942314ad30f18
        Ports:         3000/TCP, 2048/TCP, 2049/TCP, 2050/TCP
  2. The streamlet exposes its HTTP endpoint for uploading data on port 3000 - let’s set up a port forwarding on it.

    1. Check the created service:

      $ kubectl get svc -n sensor-data-scala

      You should see something like this:

      NAME                                     TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
      sensor-data-scala-http-ingress-service   ClusterIP   <none>        3000/TCP   1m
    2. Set up local port forwarding for the Pod port:

      $ kubectl port-forward svc/sensor-data-scala-http-ingress-service -n sensor-data-scala 3000:3000

      You should see something like this:

      Forwarding from -> 3000
      Forwarding from [::1]:3000 -> 3000
      Handling connection for 3000
The port-forward solution used here is a temporary route to the application from localhost. If you want to create a permanent route that can be accessed by anyone, please see the following chapter on Kubernetes ingresses: Providing External Access to Cloudflow Services
  1. Push data to the application following these steps:

    1. Create a json file named data.json with the following content:

        "deviceId": "c75cb448-df0e-4692-8e06-0321b7703992",
        "timestamp": 1495545346279,
        "measurements": {
          "power": 1.7,
          "rotorSpeed": 23.4,
          "windSpeed": 100.1
    2. Run the following command:

      curl -i -X POST http://localhost:3000 -H "Content-Type: application/json" --data @data.json

      Note that we are using the port number 3000 of localhost to which we forward the pod port. This JSON record will pass through the stages of transformation within the pipeline that we defined in the blueprint. When using this port-forward method, it’s also possible to use the same send-local-data.sh script that we used to send data to the local sandbox. For that, run the following command:


      And you should see output similar to the following:

      Sending {"deviceId":"c75cb448-df0e-4692-8e06-0321b7703992","timestamp":1495545646279,"measurements":{"power":1.7,"rotorSpeed":3.9,"windSpeed":105.9}}
      HTTP/1.1 202 Accepted
      Server: akka-http/10.1.11
      Date: Thu, 11 Jun 2020 10:20:48 GMT
      Content-Type: text/plain; charset=UTF-8
      Content-Length: 88
      The request has been accepted for processing, but the processing has not been completed.Sending {"deviceId":"c75cb448-df0e-4692-8e06-0321b7703992","timestamp":1495545646279,"measurements":{"power":1.7,"rotorSpeed":3.9,"windSpeed":105.9}}
      HTTP/1.1 202 Accepted
      Server: akka-http/10.1.11
      Date: Thu, 11 Jun 2020 10:20:48 GMT
      Content-Type: text/plain; charset=UTF-8
      Content-Length: 88

Verify the Application works

Check the log of the streamlet valid-logger to verify that you get the proper transformed metric.

kubectl logs -n sensor-data-scala sensor-data-valid-logger-76884bb775-86pwh

Towards the end of the log you will see something like the following getting printed out:

[INFO] [06/11/2020 12:20:49.288] [akka_streamlet-akka.actor.default-dispatcher-5] [akka.actor.ActorSystemImpl(akka_streamlet)] valid-logger {"deviceId": "c75cb448-df0e-4692-8e06-0321b7703992", "timestamp": 1495545646279, "name": "rotorSpeed", "value": 3.9}
[INFO] [06/11/2020 12:20:49.293] [akka_streamlet-akka.actor.default-dispatcher-6] [akka.actor.ActorSystemImpl(akka_streamlet)] valid-logger {"deviceId": "c75cb448-df0e-4692-8e06-0321b7703992", "timestamp": 1495545646279, "name": "windSpeed", "value": 105.9}

Undeploy the Application

When we are done experimenting with the application, we can undeploy it to terminate its operations.

kubectl cloudflow undeploy sensor-data-scala