SLF4J MDC

Lightbend Telemetry supports propagation of the SLF4J Mapped Diagnostic Context (MDC) through Akka actors, Akka HTTP requests, and Scala Futures. When an MDC is set on the current thread, this will be copied with any subsequent message sends, HTTP requests, or asynchronous callbacks. The MDC will be reinstated when the receiving actor processes the message, the server receives the request, the client receives the response, or when the callback runs. For example, this allows a request or transaction ID to be set and to appear in all connected log messages.

Cinnamon dependency

First make sure that your build is configured to use the Cinnamon Agent.

To enable SLF4J MDC support, add the following dependency to your build:

sbt
libraryDependencies += Cinnamon.library.cinnamonSlf4jMdc
Maven
<dependency>
  <groupId>com.lightbend.cinnamon</groupId>
  <artifactId>cinnamon-slf4j-mdc_2.11</artifactId>
  <version>2.5.3</version>
</dependency>
Gradle
dependencies {
  compile group: 'com.lightbend.cinnamon', name: 'cinnamon-slf4j-mdc_2.11', version: '2.5.3'
}

No further configuration is required. All actors (including the Akka logging actors and remote actors), Akka HTTP servers and clients, and Scala Future callbacks will transport and receive the current MDC.

Note: You do not need to use any special APIs to modify the MDC; you can use the org.slf4j.MDC API directly to put, get, remove, or clear values. For more information on using the MDC see the Logback MDC documentation.

Note: You do not need to use the DiagnosticLoggingAdapter or other MDC utilities in Akka when using the Cinnamon MDC instrumentation. The current MDC values are automatically transferred to the logging actors. There is also no need for special MDC cleanup; Cinnamon MDC will swap the MDC in and out of context as messages or callbacks are processed.

MDC filter

The MDC keys that get transferred across asynchronous boundaries can be filtered using configuration. For example, use the following configuration to only propagate the "id" value:

cinnamon.slf4j.mdc {
  filter = ["id"]
}

Serialization limit

The MDC is transported along with remote messages and requests. To avoid inadvertently sending a lot of data, there is a limit on the size of the serialized MDC. If the limit is exceeded then the MDC will not be propagated remotely and a warning will be logged. If you reach this limit, you may want to only transport specific keys using the MDC filter configuration, or the limit can be increased. For example, you can use the following configuration to set a different limit:

cinnamon.slf4j.mdc {
  serialization.limit = 2048 bytes
}

Akka HTTP headers

For Akka HTTP requests and responses, the MDC is propagated as HTTP message headers. There are two ways to propagate the MDC in headers: either as a single encoded header, or as individual headers for each MDC value with a prefixed key used as the header name.

Akka HTTP encoded header

The default MDC propagation for Akka HTTP is as a single encoded header, where the MDC context map is serialized and then encoded in Base64. For example, an MDC that contains:

"Correlation-ID" -> "abc123"

would be propagated with this header:

Cinnamon-MDC: AAAAAQAAAA4AAAAOQ29ycmVsYXRpb24tSUQAAAAGYWJjMTIz

The header name is Cinnamon-MDC by default and can be configured. For example, the following configuration sets a different header name:

cinnamon.slf4j.mdc {
  http {
    encoded-header {
      name = "X-MDC"
    }
  }
}

The filter and serialization limit are also applied to the encoded MDC header.

Akka HTTP mapped headers

The MDC propagation for Akka HTTP also supports individual headers, where each MDC value has its own header with a prefixed key. For example, an MDC that contains:

"Correlation-ID" -> "abc123"

would be propagated with this header:

CNMDC-Correlation-ID: abc123

To enable mapped headers, set the propagation method using this configuration:

cinnamon.slf4j.mdc {
  http.propagation = mapped-headers
}

Note: MDC keys and values must be suitable for being HTTP message headers. MDC keys can be filtered, or otherwise use the default encoded header propagation if MDC values are inappropriate to be header values.

The prefix used for mapped headers is configurable. For example, the following configuration sets a different header prefix:

cinnamon.slf4j.mdc {
  http {
    mapped-headers {
      prefix = "X-"
    }
  }
}