Maven Example: Lagom Java

Here are instructions for how to take a sample application and add Telemetry to it for Lagom. In this example you will add Cinnamon and a Coda Hale Console reporter will be used to print Telemetry output to the terminal window.

Official sample

See a comprehensive sample application using Lagom, Maven, and Lightbend Telemetry here: https://github.com/lightbend/telemetry-samples/tree/master/lagom/shopping-cart-java.

Prerequisites

The following must be installed for these instructions to work:

Commercial credentials

To gain access to Lightbend Telemetry you must have a Lightbend subscription and Lightbend account.

Once you have logged in, the Lightbend platform credentials guide contains instructions for configuring your sbt, maven or gradle project to access Lightbend’s commercial dependencies, including Telemetry. If you have previously used a username and password to configure your commercial credentials, the guide will instruct you how to update to using the new scheme.

Note

The URLs generated as part of your Lightbend platform credentials should not be committed to public source control repositories.

If you do not have a Lightbend subscription, please contact us to request an evaluation.

Sample application

We will use a stripped down version of the Hello Service from the Java with Maven version of the Lagom getting started guide. To demonstrate circuit breaker metrics, we will create a second service call that does an outgoing service call to the Hello Service.

Start off by creating a folder lagom-java-example with the following files (content of files will be added later):

  • pom.xml
  • hello-service-api/pom.xml
  • hello-service-api/src/main/java/cinnamon/lagom/api/HelloService.java
  • hello-service-impl/pom.xml
  • hello-service-impl/src/main/resources/application.conf
  • hello-service-impl/src/main/java/cinnamon/lagom/impl/HelloServiceImpl.java
  • hello-service-impl/src/main/java/cinnamon/lagom/impl/HelloModule.java

Next step is to add content to the files created.

Add to pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                      http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.lightbend.telemetry</groupId>
  <artifactId>hello-service</artifactId>
  <version>1.0-SNAPSHOT</version>

  <packaging>pom</packaging>

  <modules>
    <module>hello-service-api</module>
    <module>hello-service-impl</module>
  </modules>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <scala.bin.version>2.12</scala.bin.version>
    <lagom.version>1.6.5</lagom.version>
    <cinnamon.version>2.16.1</cinnamon.version>
  </properties>

  <!--
  Generate your Lightbend commercial <repository> config for maven at:
  https://www.lightbend.com/account/lightbend-platform/credentials
  -->

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <version>3.1.1</version>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-install-plugin</artifactId>
        <version>2.5.2</version>
      </plugin>
      <plugin>
        <groupId>com.lightbend.lagom</groupId>
        <artifactId>lagom-maven-plugin</artifactId>
        <version>${lagom.version}</version>
        <configuration>
          <cassandraEnabled>false</cassandraEnabled>
          <kafkaEnabled>false</kafkaEnabled>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.1</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
          <compilerArgs>
            <arg>-parameters</arg>
          </compilerArgs>
        </configuration>
      </plugin>

      <!-- Add plugin to correctly copy Lightbend Telemetry (Cinnamon) agent -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>3.1.2</version>
        <executions>
          <execution>
            <id>copy</id>
            <phase>compile</phase>
            <goals>
              <goal>copy</goal>
            </goals>
            <configuration>
              <artifactItems>
                <artifactItem>
                  <groupId>com.lightbend.cinnamon</groupId>
                  <artifactId>cinnamon-agent</artifactId>
                  <version>${cinnamon.version}</version>
                  <overWrite>true</overWrite>
                  <destFileName>cinnamon-agent.jar</destFileName>
                </artifactItem>
              </artifactItems>
            </configuration>
          </execution>
        </executions>
      </plugin>

      <!-- Use agent when running the application -->
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>3.0.0</version>
        <configuration>
          <executable>java</executable>
          <!--
          We are gonna skip it for the main project, but it is
          configured to run for the implementation subprojects.
          -->
          <skip>true</skip>
          <arguments>
            <!-- Use Cinnamon agent when running the application -->
            <argument>-javaagent:${project.build.directory}/dependency/cinnamon-agent.jar</argument>
            <argument>-classpath</argument>
            <classpath/>
            <argument>play.core.server.ProdServerStart</argument>
          </arguments>
        </configuration>
      </plugin>

      <!-- OPTIONAL: Use agent when running the tests -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.22.2</version>
        <configuration>
          <!-- Use Cinnamon agent when running the application -->
          <argLine>-javaagent:${project.build.directory}/dependency/cinnamon-agent.jar</argLine>
        </configuration>
      </plugin>

    </plugins>
  </build>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.lightbend.lagom</groupId>
        <artifactId>lagom-maven-dependencies</artifactId>
        <version>${lagom.version}</version>
        <scope>import</scope>
        <type>pom</type>
      </dependency>
      <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.8</version>
      </dependency>
    </dependencies>
  </dependencyManagement>
</project>

Add to hello-service-api/pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                      http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.lightbend.telemetry</groupId>
    <artifactId>hello-service</artifactId>
    <version>1.0-SNAPSHOT</version>
  </parent>

  <artifactId>hello-service-api</artifactId>

  <packaging>jar</packaging>

  <dependencies>
    <dependency>
      <groupId>com.lightbend.lagom</groupId>
      <artifactId>lagom-javadsl-api_${scala.bin.version}</artifactId>
    </dependency>
    <dependency>
      <groupId>org.projectlombok</groupId>
      <artifactId>lombok</artifactId>
    </dependency>
  </dependencies>
</project>

Add to hello-service-impl/pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                      http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.lightbend.telemetry</groupId>
    <artifactId>hello-service</artifactId>
    <version>1.0-SNAPSHOT</version>
  </parent>

  <artifactId>hello-service-impl</artifactId>

  <packaging>jar</packaging>

  <dependencies>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>hello-service-api</artifactId>
      <version>${project.version}</version>
    </dependency>
    <!-- Add Lightbend Telemetry (Cinnamon) dependencies -->
    <dependency>
      <groupId>com.lightbend.cinnamon</groupId>
      <artifactId>cinnamon-lagom_${scala.bin.version}</artifactId>
      <version>${cinnamon.version}</version>
    </dependency>
    <!-- Use Coda Hale Metrics and Akka instrumentation -->
    <dependency>
      <groupId>com.lightbend.cinnamon</groupId>
      <artifactId>cinnamon-chmetrics</artifactId>
      <version>${cinnamon.version}</version>
    </dependency>

    <!-- Lagom dependencies -->
    <dependency>
      <groupId>com.lightbend.lagom</groupId>
      <artifactId>lagom-javadsl-server_${scala.bin.version}</artifactId>
    </dependency>
    <dependency>
      <groupId>com.lightbend.lagom</groupId>
      <artifactId>lagom-logback_${scala.bin.version}</artifactId>
    </dependency>
    <dependency>
      <groupId>com.typesafe.play</groupId>
      <artifactId>play-akka-http-server_${scala.bin.version}</artifactId>
    </dependency>
    <dependency>
      <groupId>com.lightbend.lagom</groupId>
      <artifactId>lagom-javadsl-testkit_${scala.bin.version}</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.projectlombok</groupId>
      <artifactId>lombok</artifactId>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>com.lightbend.lagom</groupId>
        <artifactId>lagom-maven-plugin</artifactId>
        <configuration>
          <lagomService>true</lagomService>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <configuration>
          <executable>java</executable>
          <skip>false</skip>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>
Note

Lightbend Telemetry (Cinnamon) dependencies need to be added to every implementation project.

Add to hello-service-api/src/main/java/cinnamon/lagom/api/HelloService.java:

package cinnamon.lagom.api;

import akka.NotUsed;
import com.lightbend.lagom.javadsl.api.*;
import static com.lightbend.lagom.javadsl.api.Service.*;

public interface HelloService extends Service {
    /**
     * Example: curl http://localhost:9000/api/hello/Alice
     */
    ServiceCall<NotUsed, String> hello(String id);

    /**
     * Example: curl http://localhost:9000/api/hello-proxy/Alice
     */
    ServiceCall<NotUsed, String> helloProxy(String id);

    @Override
    default Descriptor descriptor() {
        return named("hello").withCalls(
            pathCall("/api/hello/:id", this::hello),
            pathCall("/api/hello-proxy/:id", this::helloProxy)
        ).withAutoAcl(true);
    }
}

Add to hello-service-impl/src/main/java/cinnamon/lagom/impl/HelloServiceImpl.java:

package cinnamon.lagom.impl;

import akka.NotUsed;
import cinnamon.lagom.api.HelloService;
import com.lightbend.lagom.javadsl.api.ServiceCall;
import java.util.concurrent.CompletionStage;
import javax.inject.Inject;
import static java.util.concurrent.CompletableFuture.completedFuture;

public class HelloServiceImpl implements HelloService {

    private final HelloService helloService;

    @Inject
    public HelloServiceImpl(HelloService helloService) {
        this.helloService = helloService;
    }

    @Override
    public ServiceCall<NotUsed, String> hello(String id) {
        return request -> {
            return completedFuture("Hello " + id);
        };
    }

    @Override
    public ServiceCall<NotUsed, String> helloProxy(String id) {
        return msg -> {
            CompletionStage<String> response = helloService.hello(id).invoke(NotUsed.getInstance());
            return response.thenApply(answer -> "Hello service said: " + answer);
        };
    }

}
Note

Since circuit breakers are used for outgoing connections in Lagom, we have a service call, named hello-proxy, that does an outgoing call to the first service call, named hello.

Now you need to register the service in a module. Please also note that we are registering the ConfigurationServiceLocator, so that we can run this sample standalone using Maven.

Add to hello-service-impl/src/main/java/cinnamon/lagom/impl/HelloModule.java:

package cinnamon.lagom.impl;

import cinnamon.lagom.api.HelloService;
import com.google.inject.AbstractModule;
import com.lightbend.lagom.javadsl.api.ServiceLocator;
import com.lightbend.lagom.javadsl.client.ConfigurationServiceLocator;
import com.lightbend.lagom.javadsl.server.ServiceGuiceSupport;
import com.typesafe.config.Config;
import play.Environment;

public class HelloModule extends AbstractModule implements ServiceGuiceSupport {

    private final Environment environment;
    private final Config config;

    public HelloModule(Environment environment, Config config) {
        this.environment = environment;
        this.config = config;
    }

    @Override
    protected void configure() {
        bindService(HelloService.class, HelloServiceImpl.class);

        // Only needed to allow the sample to run from test and dist with the config Service Locator
        if (environment.isProd()) {
            bind(ServiceLocator.class).to(ConfigurationServiceLocator.class);
        }
    }
}

Finally, add to hello-service-impl/src/main/resources/application.conf:

play.modules.enabled += cinnamon.lagom.impl.HelloModule

lagom.circuit-breaker.default.max-failures = 10

// Only needed to allow the sample to run from test and dist with the config Service Locator
lagom.services {
  hello  = "http://localhost:9000"
}

lagom.spi.circuit-breaker-metrics-class = "cinnamon.lagom.CircuitBreakerInstrumentation"

cinnamon {

  application = "hello-lagom"

  chmetrics.reporters += "console-reporter"

  akka.actors {
    default-by-class {
      includes = "/user/*"
      report-by = class
      excludes = ["akka.http.*", "akka.stream.*"]
    }
  }

  lagom.http {
    servers {
      "*:*" {
        paths {
          "*" {
            metrics = on
          }
        }
      }
    }
    clients {
      "*:*" {
        paths {
          "*" {
            metrics = on
          }
        }
      }
    }
  }
}

The application.conf file is where the actual wiring of telemetry takes place so let us dissect this further:

Setting Explanation
lagom.spi.circuit-breaker-metrics-class The class that should handle all circuit breaker related metrics. Lightbend Telemetry provides an implementation for this. You can also use your own implementation if you would like. For more information see the Lagom SPI implementation.
cinnamon.chmetrics.reporters Specifies the Coda Hale reporter you wish to use. For more information see Coda Hale. Note that there are other ways to send data, e.g. Prometheus or StatsD.
cinnamon.akka.actors Specifies which actors to collect metrics for. For more information see Actor Configuration.
cinnamon.lagom.http Specifies which HTTP servers and clients to collect metrics for. For more information see Lagom Configuration.

The lagom.services describes where to find the hello service for outgoing service calls, and is only needed since we use the ConfigurationServiceLocator, so that we can run this sample standalone using Maven.

Note

A Lagom application normally consists of multiple projects, one for each microservice, and you need to make sure there is an application.conf file for each project that you would like to instrument.

Running

When you have added the files above you need to start the application. To start your application you can run the underlying Play server.

Note

Lagom has a special development mode for rapid development, and does not fork the JVM when using the runAll or run commands in sbt. A forked JVM is necessary to gain metrics for actors and HTTP calls, since those are provided by the Cinnamon Java Agent.

mvn compile exec:exec --projects hello-service-impl

The output should look something like this:

[INFO] Scanning for projects...
[INFO]
[INFO] ------------< com.lightbend.telemetry:hello-service-impl >--------------
[INFO] Building hello-service-impl 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- exec-maven-plugin:3.0.0:exec (default-cli) @ hello-service-impl ---
[INFO] [11/20/2020 17:44:34.549] [main-1] [Cinnamon] Agent version 2.16.1
[INFO] [11/20/2020 17:44:36.177] [main-1] [Cinnamon] Agent found Akka Actor version: 2.6.15
[INFO] [11/20/2020 17:44:36.189] [main-1] [Cinnamon] Agent found Akka version: 2.6.15
[INFO] [11/20/2020 17:44:36.194] [main-1] [Cinnamon] Agent found Akka Cluster version: 2.6.15
[INFO] [11/20/2020 17:44:36.197] [main-1] [Cinnamon] Agent found Akka Cluster Sharding version: 2.6.15
[INFO] [11/20/2020 17:44:36.199] [main-1] [Cinnamon] Agent found Akka Cluster Sharding Typed version: 2.6.15
[INFO] [11/20/2020 17:44:36.201] [main-1] [Cinnamon] Agent found Akka Cluster version: 2.6.15
[INFO] [11/20/2020 17:44:36.267] [main-1] [Cinnamon] Agent found Akka Persistence version: 2.6.15
[INFO] [11/20/2020 17:44:36.277] [main-1] [Cinnamon] Agent found Akka Streams version: 2.6.15
[INFO] [11/20/2020 17:44:36.313] [main-1] [Cinnamon] Agent found Akka Actor Typed version: 2.6.15
[INFO] [11/20/2020 17:44:37.426] [main-1] [Cinnamon] Agent found Lagom Java DSL Persistence version: 1.6.5

...

2020-11-20T22:44:41.975Z [info] play.core.server.AkkaHttpServer [] - Listening for HTTP on /0:0:0:0:0:0:0:0:9000

...

[info] -- Gauges ----------------------------------------------------------------------
[info] metrics.akka.systems.application.dispatchers.akka_actor_default-dispatcher.active-threads
[info]              value = 0
[info] metrics.akka.systems.application.dispatchers.akka_actor_default-dispatcher.parallelism
[info]              value = 8
[info] metrics.akka.systems.application.dispatchers.akka_actor_default-dispatcher.pool-size
[info]              value = 3
[info] metrics.akka.systems.application.dispatchers.akka_actor_default-dispatcher.queued-tasks
[info]              value = 0
[info] metrics.akka.systems.application.dispatchers.akka_actor_default-dispatcher.running-threads
[info]              value = 0
[info] metrics.akka.systems.application.dispatchers.akka_io_pinned-dispatcher.active-threads
[info]              value = 1
[info] metrics.akka.systems.application.dispatchers.akka_io_pinned-dispatcher.pool-size
[info]              value = 1
[info] metrics.akka.systems.application.dispatchers.akka_io_pinned-dispatcher.running-threads
[info]              value = 0
...

To try out the hello-proxy service call and see metrics for the HTTP endpoints, as well as the hello circuit breaker you can either point your browser to http://localhost:9000/api/hello-proxy/World or simply run curl from the command line:

curl -v http://localhost:9000/api/hello-proxy/World

The output from the server should now also contain metrics like this:

[info] -- Gauges ----------------------------------------------------------------------
...
[info] metrics.lagom.circuit-breakers.hello.state
[info]              value = 3
...
[info] -- Histograms ------------------------------------------------------------------
[info] metrics.akka-http.systems.application.http-servers.0_0_0_0_0_0_0_1_9000.request-paths._api_hello-proxy__id.endpoint-response-time
[info]              count = 1
[info]                min = 472903565
[info]                max = 472903565
[info]               mean = 472903565.00
[info]             stddev = 0.00
[info]             median = 472903565.00
[info]               75% <= 472903565.00
[info]               95% <= 472903565.00
[info]               98% <= 472903565.00
[info]               99% <= 472903565.00
[info]             99.9% <= 472903565.00
...
[info] metrics.lagom.circuit-breakers.hello.latency
[info]              count = 1
[info]                min = 331715012
[info]                max = 331715012
[info]               mean = 331715012.00
[info]             stddev = 0.00
[info]             median = 331715012.00
[info]               75% <= 331715012.00
[info]               95% <= 331715012.00
[info]               98% <= 331715012.00
[info]               99% <= 331715012.00
[info]             99.9% <= 331715012.00
...

That is how easy it is to get started. You can find more information about the configuration, plugins, etc. in the rest of this documentation.