Cinnamon 2.5 migration guide

This is a guide for migrating from Cinnamon 2.4 to Cinnamon 2.5.

Trace Spans renamed to Stopwatch

To avoid confusion with the OpenTracing integration, the Spans extension has been renamed to Stopwatch. The Stopwatch API is otherwise used in the same way as the previous Tracer API, for recording span times across asynchronous flows. The Tracer extension is still available, forwarding to the Stopwatch extension, but marked deprecated. The end method has also been deprecated and renamed to stop. The deprecated Tracer will be removed in Cinnamon 2.6.

Previously the Tracer API would be used like this:

Tracer(context.system).start("my-span-name") {
  actor ! "message"
}

Tracer(context.system).end("my-span-name")

With the new Stopwatch API this is now:

Stopwatch(context.system).start("my-stopwatch-name") {
  actor ! "message"
}

Stopwatch(context.system).stop("my-stopwatch-name")

Reported metrics are now reported as stopwatches rather than trace-spans. If you are using Grafana dashboards, please upgrade to the new Stopwatch dashboards for Cinnamon 2.5, which use the new metric identifiers.

Host and application identifiers

The host and application identifiers used by metrics and events can now be configured in one place. Previously this was handled differently by different reporters and backends. These can be configured using the cinnamon.host and cinnamon.application config settings (or as system properties) or with the CINNAMON_HOST and CINNAMON_APPLICATION environment variables.

StatsD reporter server and application identifiers

The StatsD reporter now uses the shared host and application identifiers. Previously these settings were available:

cinnamon.chmetrics {
  statsd-reporter {
    server-id = ${?SERVER_ID}
    application-id = ${?APPLICATION_ID}
  }
}

Use the new cinnamon.host and cinnamon.application settings instead.

The overall wire format for StatsD has stayed the same (with servers and apps set to the host and application identifiers).

DogStatsD reporter host and application tags

The host and application identifiers can be included as DogStatsD tags by enabling the unique-dimensions setting. This is off by default.

cinnamon.chmetrics {
  statsd-reporter {
    dogstatsd {
      # Whether to include "unique dimensions" as tags.
      # These are tags that are unique to this reporter,
      # such as host name and application identifier.
      # Off by default for compatibility with earlier versions.
      # Note: host tag is also already provided by DogStatsD
      unique-dimensions = off
    }
  }
}

JVM metrics

The JVM metrics now use Cinnamon metric identities, to make use of the new host and application identities, rather than registering directly with the Coda Hale Metrics registry. This means that some of the reported JVM metrics have different formatting (with . characters replaced with _).

JVM metrics and DogStatsD reporter

This change to metric identities means that the metric names for JVM metrics reported by DogStatsD no longer include the metrics.jvm. prefix.

JVM metrics and Grafana dashboards

Please upgrade to the new JVM metrics Grafana dashboards for Cinnamon 2.5, which use the new formatting.

Contrail events

The hostname and app-name header fields can be configured using the cinnamon.host and cinnamon.application settings.

Some of the key names for event data have changed. For example, previously an actor event would include these data keys:

"actors-class", "actors-name",
"dispatchers-class", "dispatchers-name",
"systems-class", "systems-name"

Actor events now include these data keys:

"actor", "actor-class",
"dispatcher", "dispatcher-class",
"actor-system", "actor-system-class"

SLF4J events

Some of the key names for SLF4J event data have changed. For example, previously an actor event would include these data keys:

"actors-class", "actors-name",
"dispatchers-class", "dispatchers-name",
"systems-class", "systems-name"

Actor events now include these data keys:

"actor", "actor-class",
"dispatcher", "dispatcher-class",
"actor-system", "actor-system-class"

Lagom circuit breakers

The metrics format for Lagom circuit breakers has been updated to be aligned with Akka metrics. For example, the previous StatsD format was:

servers.<server>.apps.<app>.circuit-breaker.<circuit-breaker>.<metric>.<field>

The new StatsD format for circuit breakers is:

servers.<server>.apps.<app>.metrics.lagom.circuit-breakers.<circuit-breaker>.<metric>.<field>

Some metric and event names have also been updated:

Previous Updated
state (gauge) state (gauge)
throughput (rate) throughput (rate)
success-counter (rate) success (rate)
failure-counter (rate) failure (rate)
throughput-failure (rate) (removed, same as failure)
latency (recorder) latency (recorder)
state-event (event) circuit-breaker-state-change (event)

Circuit breakers and Grafana dashboards

Please upgrade to the new Grafana dashboards for Cinnamon 2.5, which use the new formatting.