Lagom configuration

Lagom endpoints need to be selected for Telemetry using configuration. Cinnamon accesses the same configuration as used to configure an ActorSystem. The default configuration is in application.conf.

Server metrics

Server metrics are automatically enabled when Cinnamon’s Lagom dependency is on the classpath.

It is possible to turn server-metrics off for a server with configuration, see Example configuration 3 here below.

Server metrics configuration

Endpoint metrics are not enabled by default. In other words, without any specific configuration, there will be no metrics for the endpoints.

Endpoint configuration settings are grouped per “host:port.” Cinnamon will inquire the configuration to determine how to set things up.

The configuration should define the following settings:

Setting Explanation
host:port The IP address and port the setting is valid for. Can be set to "*:*" to be valid for all hosts/ports.
server-metrics An optional setting (by default set to on) that can be used to turn metrics off for server(s).
paths A section in which endpoint paths are defined with specific settings per endpoint.

Each endpoint in the paths section should define the following:

Setting Explanation
endpoint path The endpoint path to configure. A * wildcard can be used at the end to select all subpaths.
metrics Should be set to on, true or yes for metrics to be created.

Server endpoint naming

By default, endpoint metrics created use the endpoint path as name for the metric. Let’s look at an example to clarify this feature:

Path directive Service definition Metric name
pathCall("/api/hello/:id", hello _) def hello(id: String) /api/hello/_id
pathCall("/order/:orderId/item/:itemId", getItem _) def getItem(orderId: Long, itemId: String) /order/_orderId/item/_itemId

Configuration conflict resolution

If there are several HTTP related configuration settings available on the classpath, e.g. Lagom and Akka HTTP, Lightbend Telemetry will merge all settings if possible. Should there be conflicting settings, the following order is applied: Lagom > Play > Akka HTTP.

Example configurations

Example configuration 1

The simplest configuration possible is the one that matches all paths, hosts and ports:

cinnamon.lagom.http.servers {
  "*:*" {
    paths {
      "*" {
        metrics = on
      }
    }
  }
}

Example configuration 2

Below is an example configuration that is more specific than Example configuration 1:

cinnamon.lagom.http.servers {
  "192.168.10.1:8080" {
    paths {
      "users/us/*" {
        metrics = on
      }
      "users/se/*" {
        metrics = on
      }
      "users/nz/*" {
        metrics = on
      }
    }
  }
}

The above configuration will create metrics for paths that map to users in us, se, or nz. Notice that all users in these domains are valid since we use * as part of the paths. Furthermore, this configuration is only valid for a server started on IP 192.168.10.1 and port 8080.

Example configuration 3

Turning all metrics off for a particular server:

cinnamon.lagom.http.servers {
  "192.168.0.1:*" {
    server-metrics = off
  }
}

The server-metrics setting is optional and if not present it will automatically be set to on.

Client metrics configuration

Client metrics configuration settings are grouped per “host:port”. In the context of a Lagom service client we refer to “host:port” as a service. Each service can define which “paths” should generate metrics.

Let us look at a couple of client metrics configuration examples.

Example configuration 1

The simplest configuration possible is the one that matches all paths, hosts and ports:

cinnamon.lagom.http.clients {
  "*:*" {
    paths {
      "*" {
        metrics = on
      }
    }
  }
}

With the above configuration all service calls will be monitored and metrics generated will be under the service name “:”.

Example configuration 2

In the configuration below we use the setting service-name which means that metrics will be generated in the context of “accountService” and “customerService”.

cinnamon.lagom.http.clients {
  "192.168.10.1:*" {
    service-name = "accountService"
    paths {
      "*" {
        metrics = on
      }
    }
  },
  "192.168.10.10:8080" {
    service-name = "customerService"
    paths {
      "*" {
        metrics = on
      }
    }
  }
}

Example configuration 3

It is also possible to define what paths of a service that should generate and contribute to the metrics.

cinnamon.lagom.http.clients {
  "192.168.10.1:8080" {
    paths {
      "customer/address/*" {
        metrics = on
      }
    }
  }
}

Note: On the contrary to server endpoint metrics, in which unique metrics are created per path endpoint, any defined paths in client metrics configuration contribute to one overall metrics per service.

Circuit breaker configuration

To wire up Lagom circuit breakers telemetry in your application you must add the following line to the configuration file:

lagom.spi.circuit-breaker-metrics-class = "cinnamon.lagom.CircuitBreakerInstrumentation"

This instructs Lagom to provide data to Lightbend Telemetry rather than the default behavior.

The information gathered include:

  • State of circuit breaker — closed, open, half-open

  • Latency

  • Throughput

  • Success and failure counts

Lagom circuit breakers

It is important to have an understanding of the circuit breakers in Lagom in order to interpret the data that is gathered. In Lagom each client talking to another service uses a circuit breaker. This basically means that the circuit breaker is on the client instead of the server (in a traditional sense). This means that if your code calls a service directly there will not be any information gathered about this call. Currently only service to service calls are instrumented. See the Lagom documentation for more information on service clients and circuit breakers.

The state of a circuit breaker is one of:

State Interpretation Data Representation
open something is wrong 1
half-open things are going from bad to good 2
closed everything is fine 3

The Data Representation in the above table is the number associated with a certain state, i.e. this is what comes out of Lightbend Telemetry to any reporter listening to the data.