Metrics and Events

This section contains a list of the different metrics and events for the different instrumentations. Please note that the units described here in the meta data is just a textual representation of the units that the metrics will be collected and reported in regardless of any changes to this configuration. That means that for example changing a nanoseconds to milliseconds will not change the unit that the metric is collected and reported in. Changing the unit that for example times are reported in is something that has to be done in the plugin or reporter configuration.

Note

Any metric or event can be disabled by setting the enabled configuration to false as described here. It is also possible to rate limit events to manage the amount of data being reported.

Akka Actor

Metric	Type	Description
ActorFailure	Event	when an actor fails, i.e. throws an exception. Event information: actor-ref — the actor failing cause — the exception being thrown OpenTelemetry `akka.actor.actor-failure` Prometheus `akka_actor_actor_failure` Elastic `actors.actor-failure` Config `cinnamon.akka.meta.descriptor { actor { actor-failure { key = "actor-failure" } } }`
DeadLetter	Event	when a message is sent to an actor that no longer exists. Event information: recipient — the intended recipient of the message message — the message being sent actor-ref — the actor sending the message OpenTelemetry `akka.actor.dead-letter` Prometheus `akka_actor_dead_letter` Elastic `actors.dead-letter` Config `cinnamon.akka.meta.descriptor { actor { dead-letter { key = "dead-letter" } } }`
Dropped messages	Rate	statistics for the number of messages dropped from bounded mailboxes per actor OpenTelemetry `akka.actor.dropped-messages` Prometheus `akka_actor_dropped_messages` Elastic `actors.dropped-messages` Config `cinnamon.akka.meta.descriptor { actor { dropped-messages { key = "dropped-messages" unit-type = "custom" unit-suffix = "msg" } } }`
LogError	Event	when an actor logs an error. Event information: actor-ref — the actor logging the error error — the error being logged OpenTelemetry `akka.actor.log-error` Prometheus `akka_actor_log_error` Elastic `actors.log-error` Config `cinnamon.akka.meta.descriptor { actor { log-error { key = "log-error" } } }`
LogWarning	Event	when an actor logs a warning. Event information: actor-ref — the actor logging the warning warning — the warning being logged OpenTelemetry `akka.actor.log-warning` Prometheus `akka_actor_log_warning` Elastic `actors.log-warning` Config `cinnamon.akka.meta.descriptor { actor { log-warning { key = "log-warning" } } }`
Mailbox size	Recorder	statistics for actor mailbox sizes OpenTelemetry `akka.actor.mailbox-size` Prometheus `akka_actor_mailbox_size` Elastic `actors.mailbox-size` Config `cinnamon.akka.meta.descriptor { actor { mailbox-size { key = "mailbox-size" unit-type = "custom" unit-suffix = "msg" } } }`
MailboxSizeLimit	Event	mailbox queue grows too large. Event information: actor-ref — the actor whose mailbox size has exceeded the limit message — the message being enqueued in the mailbox size — the mailbox size limit — the mailbox size limit OpenTelemetry `akka.actor.mailbox-size-limit` Prometheus `akka_actor_mailbox_size_limit` Elastic `actors.mailbox-size-limit` Config `cinnamon.akka.meta.descriptor { actor { mailbox-size-limit { key = "mailbox-size-limit" } } }`
Mailbox time	Recorder	statistics for the time that messages are in the mailbox OpenTelemetry `akka.actor.mailbox-time` Prometheus `akka_actor_mailbox_time_ns` Elastic `actors.mailbox-time` Config `cinnamon.akka.meta.descriptor { actor { mailbox-time { key = "mailbox-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
MailboxTimeLimit	Event	message has been in the mailbox for too long. Event information: actor-ref — the actor whose mailbox time has exceeded the limit message — the message being dequeued from the mailbox nanos — the mailbox time threshold-nanos — the mailbox time limit OpenTelemetry `akka.actor.mailbox-time-limit` Prometheus `akka_actor_mailbox_time_limit` Elastic `actors.mailbox-time-limit` Config `cinnamon.akka.meta.descriptor { actor { mailbox-time-limit { key = "mailbox-time-limit" } } }`
Processed messages	Rate	the number of messages that actors have processed in the selected time frame OpenTelemetry `akka.message.processed-messages` Prometheus `akka_message_processed_messages` Elastic `messages.processed-messages` Config `cinnamon.akka.meta.descriptor { actor { processed-messages { key = "processed-messages" unit-type = "custom" unit-suffix = "msg" } } }`
Processing time	Recorder	statistics for the processing time of actors OpenTelemetry `akka.message.processing-time` Prometheus `akka_message_processing_time_ns` Elastic `messages.processing-time` Config `cinnamon.akka.meta.descriptor { actor { processing-time { key = "processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
ProcessingTimeLimit	Event	message processing takes too long. Event information: actor-ref — the actor whose processing time has exceeded the limit message — the message that was just processed nanos — the processing time threshold-nanos — the processing time limit OpenTelemetry `akka.actor.processing-time-limit` Prometheus `akka_actor_processing_time_limit` Elastic `actors.processing-time-limit` Config `cinnamon.akka.meta.descriptor { actor { processing-time-limit { key = "processing-time-limit" } } }`
Remote deserialization time	Recorder	statistics for the time that deserialization takes OpenTelemetry `akka.message.remote-deserialization-time` Prometheus `akka_message_remote_deserialization_time_ns` Elastic `messages.remote-deserialization-time` Config `cinnamon.akka.meta.descriptor { actor { remote-deserialization-time { key = "remote-deserialization-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Remote large message received	Event	a message larger than the threshold has been received Event information: actor-ref — the actor who is receiving the large message message-class — the message class of the large message size — the size in bytes of the large message sender — the sender of the large message OpenTelemetry `akka.actor.remote-large-message-received` Prometheus `akka_actor_remote_large_message_received` Elastic `actors.remote-large-message-received` Config `cinnamon.akka.meta.descriptor { actor { remote-large-message-received { key = "remote-large-message-received" } } }`
Remote large message sent	Event	a message larger than the threshold has been sent Event information: actor-ref — the actor who is sending the large message message-class — the message class of the large message size — the size in bytes of the large message recipient — the recipient of the large message OpenTelemetry `akka.actor.remote-large-message-sent` Prometheus `akka_actor_remote_large_message_sent` Elastic `actors.remote-large-message-sent` Config `cinnamon.akka.meta.descriptor { actor { remote-large-message-sent { key = "remote-large-message-sent" } } }`
Remote received message size	Recorder	statistics for remote received message sizes OpenTelemetry `akka.message.remote-received-message-size` Prometheus `akka_message_remote_received_message_size` Elastic `messages.remote-received-message-size` Config `cinnamon.akka.meta.descriptor { actor { remote-received-message-size { key = "remote-received-message-size" unit-type = "custom" unit-suffix = "byte" } } }`
Remote received messages	Rate	statistics for the number received remote messages OpenTelemetry `akka.message.remote-received-messages` Prometheus `akka_message_remote_received_messages` Elastic `messages.remote-received-messages` Config `cinnamon.akka.meta.descriptor { actor { remote-received-messages { key = "remote-received-messages" unit-type = "custom" unit-suffix = "msg" } } }`
Remote sent message size	Recorder	statistics for remote sent message sizes OpenTelemetry `akka.message.remote-sent-message-size` Prometheus `akka_message_remote_sent_message_size` Elastic `messages.remote-sent-message-size` Config `cinnamon.akka.meta.descriptor { actor { remote-sent-message-size { key = "remote-sent-message-size" unit-type = "custom" unit-suffix = "byte" } } }`
Remote sent messages	Rate	statistics for the number sent remote messages OpenTelemetry `akka.message.remote-sent-messages` Prometheus `akka_message_remote_sent_messages` Elastic `messages.remote-sent-messages` Config `cinnamon.akka.meta.descriptor { actor { remote-sent-messages { key = "remote-sent-messages" unit-type = "custom" unit-suffix = "msg" } } }`
Remote serialization time	Recorder	statistics for the time that serialization takes OpenTelemetry `akka.message.remote-serialization-time` Prometheus `akka_message_remote_serialization_time_ns` Elastic `messages.remote-serialization-time` Config `cinnamon.akka.meta.descriptor { actor { remote-serialization-time { key = "remote-serialization-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Actors (running)	Counter	the number of running actors (of an actor class or group) OpenTelemetry `akka.actor.running-actors` Prometheus `akka_actor_running_actors` Elastic `actors.running-actors` Config `cinnamon.akka.meta.descriptor { actor { running-actors { key = "running-actors" unit-type = "custom" unit-suffix = "actor" } } }`
Sent messages	Rate	statistics for the number of sent messages per actor OpenTelemetry `akka.message.sent-messages` Prometheus `akka_message_sent_messages` Elastic `messages.sent-messages` Config `cinnamon.akka.meta.descriptor { actor { sent-messages { key = "sent-messages" unit-type = "custom" unit-suffix = "msg" } } }`
Stash size	Recorder	statistics for actor stash sizes OpenTelemetry `akka.actor.stash-size` Prometheus `akka_actor_stash_size` Elastic `actors.stash-size` Config `cinnamon.akka.meta.descriptor { actor { stash-size { key = "stash-size" unit-type = "custom" unit-suffix = "msg" } } }`
StashSizeLimit	Event	stash queue grows too large. Event information: actor-ref — the actor whose stash size has exceeded the limit message — the message being stashed size — the stash size limit — the stash size limit OpenTelemetry `akka.actor.stash-size-limit` Prometheus `akka_actor_stash_size_limit` Elastic `actors.stash-size-limit` Config `cinnamon.akka.meta.descriptor { actor { stash-size-limit { key = "stash-size-limit" } } }`
UnhandledMessage	Event	when an actor does not handle a message sent to it. Event information: actor-ref — the actor not handling the message message — the message not being handled sender — the sender of the message OpenTelemetry `akka.actor.unhandled-message` Prometheus `akka_actor_unhandled_message` Elastic `actors.unhandled-message` Config `cinnamon.akka.meta.descriptor { actor { unhandled-message { key = "unhandled-message" } } }`

Akka Circuit-Breaker

Metric	Type	Description
CircuitBreakerStateChange	Event	state change event: 0 - none, 1 - open, 2 - half-open, 3 - close OpenTelemetry `akka.circuit-breaker.circuit-breaker-state-change` Prometheus `akka_circuit_breaker_circuit_breaker_state_change` Elastic `circuit-breakers.circuit-breaker-state-change` Config `cinnamon.akka.meta.descriptor { circuit-breaker { state-change { key = "circuit-breaker-state-change" } } }`
Failure	Rate	the rate of failed calls OpenTelemetry `akka.circuit-breaker.failure` Prometheus `akka_circuit_breaker_failure` Elastic `circuit-breakers.failure` Config `cinnamon.akka.meta.descriptor { circuit-breaker { failure { key = "failure" unit-type = "custom" unit-suffix = "request" } } }`
Latency	Recorder	the elapsed duration of the call in nanoseconds OpenTelemetry `akka.circuit-breaker.latency` Prometheus `akka_circuit_breaker_latency_ns` Elastic `circuit-breakers.latency` Config `cinnamon.akka.meta.descriptor { circuit-breaker { latency { key = "latency" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
State	GaugeLong	current circuit-breaker state: 0 - none, 1 - open, 2 - half-open, 3 - close OpenTelemetry `akka.circuit-breaker.state` Prometheus `akka_circuit_breaker_state` Elastic `circuit-breakers.state` Config `cinnamon.akka.meta.descriptor { circuit-breaker { state { key = "state" } } }`
Success	Rate	the rate of successful calls OpenTelemetry `akka.circuit-breaker.success` Prometheus `akka_circuit_breaker_success` Elastic `circuit-breakers.success` Config `cinnamon.akka.meta.descriptor { circuit-breaker { success { key = "success" unit-type = "custom" unit-suffix = "request" } } }`
Throuphput	Rate	the total rate of calls OpenTelemetry `akka.circuit-breaker.throughput` Prometheus `akka_circuit_breaker_throughput` Elastic `circuit-breakers.throughput` Config `cinnamon.akka.meta.descriptor { circuit-breaker { throughput { key = "throughput" unit-type = "custom" unit-suffix = "request" } } }`

Akka Cluster

Metric	Type	Description
Cluster domain event	Event	cluster domain events like leader changed, role leader changed or cluster shutting down. Event information: event — type of domain event, e.g. LeaderChanged role — the role for the domain event if any OpenTelemetry `remote-node.cluster-domain-event` Prometheus `remote_node_cluster_domain_event` Elastic `remote-nodes.cluster-domain-event` Config `cinnamon.akka.meta.descriptor { node { cluster-domain-event { key = "cluster-domain-event" } } }`
Cluster member event	Event	cluster member events like member up, unreachable, reachable, exited or removed. Event information: event — type of domain event, e.g. MemberUp version — node version member-status — member status, e.g. Joining previous-member-status — previous member status, e.g. Up OpenTelemetry `remote-node.cluster-member-event` Prometheus `remote_node_cluster_member_event` Elastic `remote-nodes.cluster-member-event` Config `cinnamon.akka.meta.descriptor { node { cluster-member-event { key = "cluster-member-event" } } }`
Node quarantined event	Event	node quarantine event information OpenTelemetry `remote-node.node-quarantined-event` Prometheus `remote_node_node_quarantined_event` Elastic `remote-nodes.node-quarantined-event` Config `cinnamon.akka.meta.descriptor { node { node-quarantined-event { key = "node-quarantined-event" } } }`
Phi accrual value	GaugeDouble	statistics for the Phi accrual failure detector. A Phi value represents the connection between two nodes; self and remote. A self node can have a connection to any number of remote nodes and each connection will have its own Phi value. Note that internally in Akka the Phi accrual value can become Double.Infinity. If this happens Cinnamon will convert this value to 10241024. The reason for this is that most visualizers cannot handle infinity. If you therefore see the value 1048576 (10241024) this means that the Phi value has reached infinity. OpenTelemetry `remote-node.phi-accrual-value` Prometheus `remote_node_phi_accrual_value` Elastic `remote-nodes.phi-accrual-value` Config `cinnamon.akka.meta.descriptor { node { phi-accrual-value { key = "phi-accrual-value" } } }`
Cluster current event	Event	a one time event, per cluster node, containing information about the state of the cluster OpenTelemetry `self-node.cluster-current-event` Prometheus `self_node_cluster_current_event` Elastic `self-nodes.cluster-current-event` Config `cinnamon.akka.meta.descriptor { node { cluster-current-event { key = "cluster-current-event" } } }`
Cluster node status	Event	if cinnamon.akka.cluster.node-status is enabled one event per cluster member with the node status as seen from the emitting node. The events are emitted once on joining the cluster and then periodically. Interval for emitting is configured using cinnamon.akka.cluster.node-status-interval, default value is an interval of 30 seconds. OpenTelemetry `self-node.cluster-node-status` Prometheus `self_node_cluster_node_status` Elastic `self-nodes.cluster-node-status` Config `cinnamon.akka.meta.descriptor { node { cluster-node-status { key = "cluster-node-status" } } }`
Cluster singleton event	Event	cluster singleton events with information about node, actor singleton class and name. Event information: event — type of singleton event, e.g. started class — singleton actor class actor — singleton actor name OpenTelemetry `self-node.cluster-singleton-event` Prometheus `self_node_cluster_singleton_event` Elastic `self-nodes.cluster-singleton-event` Config `cinnamon.akka.meta.descriptor { node { cluster-singleton-event { key = "cluster-singleton-event" } } }`
Node unable to join event	Event	cluster node unable to join event with information about seed nodes and number of attempts to join OpenTelemetry `self-node.node-unable-to-join-event` Prometheus `self_node_node_unable_to_join_event` Elastic `self-nodes.node-unable-to-join-event` Config `cinnamon.akka.meta.descriptor { node { node-unable-to-join-event { key = "node-unable-to-join-event" } } }`
Phi accrual thresdhold value	GaugeDouble	the configured Phi accrual threshold value OpenTelemetry `self-node.phi-accrual-threshold-value` Prometheus `self_node_phi_accrual_threshold_value` Elastic `self-nodes.phi-accrual-threshold-value` Config `cinnamon.akka.meta.descriptor { node { phi-accrual-threshold-value { key = "phi-accrual-threshold-value" } } }`
Reachable nodes	Counter	number of reachable nodes in the cluster at any point in time OpenTelemetry `self-node.reachable-nodes` Prometheus `self_node_reachable_nodes` Elastic `self-nodes.reachable-nodes` Config `cinnamon.akka.meta.descriptor { node { reachable-nodes { key = "reachable-nodes" unit-type = "custom" unit-suffix = "node" } } }`
Shard region count value	GaugeLong	number of shards regions per node OpenTelemetry `self-node.shard-region-count-value` Prometheus `self_node_shard_region_count_value` Elastic `self-nodes.shard-region-count-value` Config `cinnamon.akka.meta.descriptor { shard { shard-region-count-value { key = "shard-region-count-value" unit-type = "custom" } } }`
Shard region event	Event	cluster shard region started/stopped event with information about actor, node, type name and type of shard region. Event information: event — type of shard region event, e.g. STARTED shard-region-actor — the actor controlling the shard region type-name — the entity type name of the shard region OpenTelemetry `self-node.shard-region-event` Prometheus `self_node_shard_region_event` Elastic `self-nodes.shard-region-event` Config `cinnamon.akka.meta.descriptor { shard { shard-region-event { key = "shard-region-event" } } }`
Shard region proxy event	Event	cluster shard region started/stopped event with information about actor, node, type name and type of shard region proxy. Event information: event — type of shard region event, e.g. STARTED shard-region-actor — the actor controlling the shard region type-name — the entity type name of the shard region OpenTelemetry `self-node.shard-region-proxy-event` Prometheus `self_node_shard_region_proxy_event` Elastic `self-nodes.shard-region-proxy-event` Config `cinnamon.akka.meta.descriptor { shard { shard-region-proxy-event { key = "shard-region-proxy-event" } } }`
Split brain resolver event	Event	Split brain resolver decision taken event with information about the decision, reachable and unreachable nodes. Event information: decision — type of SBR decision, e.g. DownUnreachable nodes — the nodes that this node finds reachable unreachable-nodes — the nodes that this node finds unreachable OpenTelemetry `self-node.split-brain-resolver-event` Prometheus `self_node_split_brain_resolver_event` Elastic `self-nodes.split-brain-resolver-event` Config `cinnamon.akka.meta.descriptor { node { split-brain-resolver-event { key = "split-brain-resolver-event" } } }`
Unreachable nodes	Counter	number of unreachable nodes in the cluster at any point in time OpenTelemetry `self-node.unreachable-nodes` Prometheus `self_node_unreachable_nodes` Elastic `self-nodes.unreachable-nodes` Config `cinnamon.akka.meta.descriptor { node { unreachable-nodes { key = "unreachable-nodes" unit-type = "custom" unit-suffix = "node" } } }`
Shard count value	GaugeLong	number of shards per shard region OpenTelemetry `shard-region.shard-count-value` Prometheus `shard_region_shard_count_value` Elastic `shard-regions.shard-count-value` Config `cinnamon.akka.meta.descriptor { shard { shard-count-value { key = "shard-count-value" unit-type = "custom" } } }`
Shard region messages delivered	Rate	statistics for the number of messages that have been delivered by the shard region actor (regardless of where the shard resides) OpenTelemetry `shard-region.shard-region-messages-delivered` Prometheus `shard_region_shard_region_messages_delivered` Elastic `shard-regions.shard-region-messages-delivered` Config `cinnamon.akka.meta.descriptor { shard { shard-region-messages-delivered { key = "shard-region-messages-delivered" unit-type = "custom" unit-suffix = "msg" } } }`
Shard entity count value	GaugeLong	number of shard entities per shard OpenTelemetry `shard.shard-entity-count-value` Prometheus `shard_shard_entity_count_value` Elastic `shards.shard-entity-count-value` Config `cinnamon.akka.meta.descriptor { shard { shard-entity-count-value { key = "shard-entity-count-value" unit-type = "custom" } } }`

Akka Dispatcher (All time metrics)

Metric	Type	Description
Processing	Counter	how many tasks are being processed right now OpenTelemetry `akka.dispatcher.processing` Prometheus `akka_dispatcher_processing` Elastic `dispatchers.processing` Config `cinnamon.akka.meta.descriptor { dispatcher { processing { key = "processing" unit-type = "custom" unit-suffix = "task" } } }`
Processing time	Recorder	statistics for how long the processing takes OpenTelemetry `akka.dispatcher.processing-time` Prometheus `akka_dispatcher_processing_time_ns` Elastic `dispatchers.processing-time` Config `cinnamon.akka.meta.descriptor { dispatcher { processing-time { key = "processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Queue size	Counter	the number of tasks waiting to be processed OpenTelemetry `akka.dispatcher.queue-size` Prometheus `akka_dispatcher_queue_size` Elastic `dispatchers.queue-size` Config `cinnamon.akka.meta.descriptor { dispatcher { queue-size { key = "queue-size" unit-type = "custom" unit-suffix = "task" } } }`
Queue time	Recorder	statistics for how long tasks are in the queue OpenTelemetry `akka.dispatcher.queue-time` Prometheus `akka_dispatcher_queue_time_ns` Elastic `dispatchers.queue-time` Config `cinnamon.akka.meta.descriptor { dispatcher { queue-time { key = "queue-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.

Akka Dispatcher (Fork-Join)

Metric	Type	Description
Active thread count	ProvidingGaugeLong	an estimate of the number of threads running or stealing tasks OpenTelemetry `akka.dispatcher.active-threads` Prometheus `akka_dispatcher_active_threads` Elastic `dispatchers.active-threads` Config `cinnamon.core.meta.descriptor { executor { active-thread-count { key = "active-threads" unit-type = "custom" unit-suffix = "thread" } } }`
Parallelism	ProvidingGaugeLong	the parallelism setting OpenTelemetry `akka.dispatcher.parallelism` Prometheus `akka_dispatcher_parallelism` Elastic `dispatchers.parallelism` Config `cinnamon.core.meta.descriptor { executor { parallelism { key = "parallelism" } } }`
Pool size	ProvidingGaugeLong	the current size of the thread pool OpenTelemetry `akka.dispatcher.pool-size` Prometheus `akka_dispatcher_pool_size` Elastic `dispatchers.pool-size` Config `cinnamon.core.meta.descriptor { executor { pool-size { key = "pool-size" unit-type = "custom" unit-suffix = "thread" } } }`
Queued task count	ProvidingGaugeLong	an estimate of the total number of tasks currently in queues OpenTelemetry `akka.dispatcher.queued-tasks` Prometheus `akka_dispatcher_queued_tasks` Elastic `dispatchers.queued-tasks` Config `cinnamon.core.meta.descriptor { executor { queued-task-count { key = "queued-tasks" unit-type = "custom" unit-suffix = "task" } } }`
Runnning thread count	ProvidingGaugeLong	an estimate of the number of threads not blocked in managed synchronization OpenTelemetry `akka.dispatcher.running-threads` Prometheus `akka_dispatcher_running_threads` Elastic `dispatchers.running-threads` Config `cinnamon.core.meta.descriptor { executor { running-thread-count { key = "running-threads" unit-type = "custom" unit-suffix = "thread" } } }`

Akka Dispatcher (Pinned)

Metric	Type	Description
Active thread count	GaugeLong	an estimate of the number of threads running tasks OpenTelemetry `akka.dispatcher.active-threads` Prometheus `akka_dispatcher_active_threads` Elastic `dispatchers.active-threads` Config `cinnamon.core.meta.descriptor { executor { active-thread-count { key = "active-threads" unit-type = "custom" unit-suffix = "thread" } } }`
Pool size	GaugeLong	the current size of the pinned thread pool OpenTelemetry `akka.dispatcher.pool-size` Prometheus `akka_dispatcher_pool_size` Elastic `dispatchers.pool-size` Config `cinnamon.core.meta.descriptor { executor { pool-size { key = "pool-size" unit-type = "custom" unit-suffix = "thread" } } }`
Runnning thread count	GaugeLong	an estimate of the total number of currently runnning threads OpenTelemetry `akka.dispatcher.running-threads` Prometheus `akka_dispatcher_running_threads` Elastic `dispatchers.running-threads` Config `cinnamon.core.meta.descriptor { executor { running-thread-count { key = "running-threads" unit-type = "custom" unit-suffix = "thread" } } }`

Akka Dispatcher (Thread-Pool)

Metric	Type	Description
Active thread count	ProvidingGaugeLong	an estimate of the number of threads running tasks OpenTelemetry `akka.dispatcher.active-threads` Prometheus `akka_dispatcher_active_threads` Elastic `dispatchers.active-threads` Config `cinnamon.core.meta.descriptor { executor { active-thread-count { key = "active-threads" unit-type = "custom" unit-suffix = "thread" } } }`
Core pool size	ProvidingGaugeLong	the minimum size of the thread pool OpenTelemetry `akka.dispatcher.core-pool-size` Prometheus `akka_dispatcher_core_pool_size` Elastic `dispatchers.core-pool-size` Config `cinnamon.core.meta.descriptor { executor { core-pool-size { key = "core-pool-size" unit-type = "custom" unit-suffix = "thread" } } }`
Max pool size	ProvidingGaugeLong	the maximum size of the thread pool OpenTelemetry `akka.dispatcher.max-pool-size` Prometheus `akka_dispatcher_max_pool_size` Elastic `dispatchers.max-pool-size` Config `cinnamon.core.meta.descriptor { executor { max-pool-size { key = "max-pool-size" unit-type = "custom" unit-suffix = "thread" } } }`
Pool size	ProvidingGaugeLong	the current size of the thread pool OpenTelemetry `akka.dispatcher.pool-size` Prometheus `akka_dispatcher_pool_size` Elastic `dispatchers.pool-size` Config `cinnamon.core.meta.descriptor { executor { pool-size { key = "pool-size" unit-type = "custom" unit-suffix = "thread" } } }`
Processed task count	ProvidingGaugeLong	an estimate of the number of processed tasks OpenTelemetry `akka.dispatcher.processed-tasks` Prometheus `akka_dispatcher_processed_tasks` Elastic `dispatchers.processed-tasks` Config `cinnamon.core.meta.descriptor { executor { processed-task-count { key = "processed-tasks" unit-type = "custom" unit-suffix = "task" } } }`

Akka Http Client

Metric	Type	Description
Akka HTTP client connections	Counter	the number of connections used in a pool OpenTelemetry `akka-http.client-pool.connections` Prometheus `akka_http_client_pool_connections` Elastic `client-pools.connections` Config `cinnamon.akka.http.meta.descriptor { client-pool { connections { key = "connections" unit-type = "custom" unit-suffix = "connection" } } }`
Akka HTTP client requests	Rate	the rate of requests from a client to a service OpenTelemetry `akka-http.http-client.http-client-requests` Prometheus `akka_http_http_client_http_client_requests` Elastic `http-clients.http-client-requests` Config `cinnamon.akka.http.meta.descriptor { client { requests { key = "http-client-requests" unit-type = "custom" } } }`
Akka HTTP client service response time	Recorder	the time for responses from a service OpenTelemetry `akka-http.http-client.http-client-service-response-time` Prometheus `akka_http_http_client_http_client_service_response_time_ns` Elastic `http-clients.http-client-service-response-time` Config `cinnamon.akka.http.meta.descriptor { client { service-response-time { key = "http-client-service-response-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.

Akka Http Server

Metric	Type	Description
Outgoing endpoint response time	Recorder	the time for responses per endpoint OpenTelemetry `akka-http.request-path.endpoint-response-time` Prometheus `akka_http_request_path_endpoint_response_time_ns` Elastic `request-paths.endpoint-response-time` Config `cinnamon.akka.http.meta.descriptor { endpoint { response-time { key = "endpoint-response-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint response time 2xx	Recorder	the time for endpoint responses with 200-299 status codes (optional) OpenTelemetry `akka-http.request-path.endpoint-response-time-2xx` Prometheus `akka_http_request_path_endpoint_response_time_2xx_ns` Elastic `request-paths.endpoint-response-time-2xx` Config `cinnamon.akka.http.meta.descriptor { endpoint { response-time-2xx { key = "endpoint-response-time-2xx" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint response time 3xx	Recorder	the time for endpoint responses with 300-399 status codes (optional) OpenTelemetry `akka-http.request-path.endpoint-response-time-3xx` Prometheus `akka_http_request_path_endpoint_response_time_3xx_ns` Elastic `request-paths.endpoint-response-time-3xx` Config `cinnamon.akka.http.meta.descriptor { endpoint { response-time-3xx { key = "endpoint-response-time-3xx" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint response time 4xx	Recorder	the time for endpoint responses with 400-499 status codes (optional) OpenTelemetry `akka-http.request-path.endpoint-response-time-4xx` Prometheus `akka_http_request_path_endpoint_response_time_4xx_ns` Elastic `request-paths.endpoint-response-time-4xx` Config `cinnamon.akka.http.meta.descriptor { endpoint { response-time-4xx { key = "endpoint-response-time-4xx" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint response time 5xx	Recorder	the time for endpoint responses with 500-599 status codes (optional) OpenTelemetry `akka-http.request-path.endpoint-response-time-5xx` Prometheus `akka_http_request_path_endpoint_response_time_5xx_ns` Elastic `request-paths.endpoint-response-time-5xx` Config `cinnamon.akka.http.meta.descriptor { endpoint { response-time-5xx { key = "endpoint-response-time-5xx" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint responses	Rate	the rate of responses per endpoint OpenTelemetry `akka-http.request-path.endpoint-responses` Prometheus `akka_http_request_path_endpoint_responses` Elastic `request-paths.endpoint-responses` Config `cinnamon.akka.http.meta.descriptor { endpoint { responses { key = "endpoint-responses" unit-type = "custom" unit-suffix = "response" } } }`
Outgoing endpoint responses 2xx	Rate	the rate of endpoint responses with 200-299 status codes (optional) OpenTelemetry `akka-http.request-path.endpoint-responses-2xx` Prometheus `akka_http_request_path_endpoint_responses_2xx` Elastic `request-paths.endpoint-responses-2xx` Config `cinnamon.akka.http.meta.descriptor { endpoint { responses-2xx { key = "endpoint-responses-2xx" unit-type = "custom" unit-suffix = "response" } } }`
Outgoing endpoint responses 3xx	Rate	the rate of endpoint responses with 300-399 status codes (optional) OpenTelemetry `akka-http.request-path.endpoint-responses-3xx` Prometheus `akka_http_request_path_endpoint_responses_3xx` Elastic `request-paths.endpoint-responses-3xx` Config `cinnamon.akka.http.meta.descriptor { endpoint { responses-3xx { key = "endpoint-responses-3xx" unit-type = "custom" unit-suffix = "response" } } }`
Outgoing endpoint responses 4xx	Rate	the rate of endpoint responses with 400-499 status codes (optional) OpenTelemetry `akka-http.request-path.endpoint-responses-4xx` Prometheus `akka_http_request_path_endpoint_responses_4xx` Elastic `request-paths.endpoint-responses-4xx` Config `cinnamon.akka.http.meta.descriptor { endpoint { responses-4xx { key = "endpoint-responses-4xx" unit-type = "custom" unit-suffix = "response" } } }`
Outgoing endpoint responses 5xx	Rate	the rate of endpoint responses with 500-599 status codes (optional) OpenTelemetry `akka-http.request-path.endpoint-responses-5xx` Prometheus `akka_http_request_path_endpoint_responses_5xx` Elastic `request-paths.endpoint-responses-5xx` Config `cinnamon.akka.http.meta.descriptor { endpoint { responses-5xx { key = "endpoint-responses-5xx" unit-type = "custom" unit-suffix = "response" } } }`
Incoming connections	Rate	the rate of connections per server OpenTelemetry `akka-http.http-server.connections` Prometheus `akka_http_http_server_connections` Elastic `http-servers.connections` Config `cinnamon.akka.http.meta.descriptor { server { connections { key = "connections" unit-type = "custom" unit-suffix = "connection" } } }`
Incoming requests	Rate	the rate of requests per server OpenTelemetry `akka-http.http-server.requests` Prometheus `akka_http_http_server_requests` Elastic `http-servers.requests` Config `cinnamon.akka.http.meta.descriptor { server { requests { key = "requests" unit-type = "custom" unit-suffix = "request" } } }`
Outgoing response time	Recorder	the time for responses per server OpenTelemetry `akka-http.http-server.response-time` Prometheus `akka_http_http_server_response_time_ns` Elastic `http-servers.response-time` Config `cinnamon.akka.http.meta.descriptor { server { response-time { key = "response-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing response time 2xx	Recorder	the time for responses with 200-299 status codes per server OpenTelemetry `akka-http.http-server.response-time-2xx` Prometheus `akka_http_http_server_response_time_2xx_ns` Elastic `http-servers.response-time-2xx` Config `cinnamon.akka.http.meta.descriptor { server { response-time-2xx { key = "response-time-2xx" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing response time 3xx	Recorder	the time for responses with 300-399 status codes per server OpenTelemetry `akka-http.http-server.response-time-3xx` Prometheus `akka_http_http_server_response_time_3xx_ns` Elastic `http-servers.response-time-3xx` Config `cinnamon.akka.http.meta.descriptor { server { response-time-3xx { key = "response-time-3xx" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing response time 4xx	Recorder	the time for responses with 400-499 status codes per server OpenTelemetry `akka-http.http-server.response-time-4xx` Prometheus `akka_http_http_server_response_time_4xx_ns` Elastic `http-servers.response-time-4xx` Config `cinnamon.akka.http.meta.descriptor { server { response-time-4xx { key = "response-time-4xx" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing response time 5xx	Recorder	the time for responses with 500-599 status codes per server OpenTelemetry `akka-http.http-server.response-time-5xx` Prometheus `akka_http_http_server_response_time_5xx_ns` Elastic `http-servers.response-time-5xx` Config `cinnamon.akka.http.meta.descriptor { server { response-time-5xx { key = "response-time-5xx" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Outgoing responses	Rate	the rate of responses per server OpenTelemetry `akka-http.http-server.responses` Prometheus `akka_http_http_server_responses` Elastic `http-servers.responses` Config `cinnamon.akka.http.meta.descriptor { server { responses { key = "responses" unit-type = "custom" unit-suffix = "response" } } }`
Outgoing responses 2xx	Rate	the rate of responses with 200-299 status codes per server OpenTelemetry `akka-http.http-server.responses-2xx` Prometheus `akka_http_http_server_responses_2xx` Elastic `http-servers.responses-2xx` Config `cinnamon.akka.http.meta.descriptor { server { responses-2xx { key = "responses-2xx" unit-type = "custom" unit-suffix = "response" } } }`
Outgoing responses 3xx	Rate	the rate of responses with 300-399 status codes per server OpenTelemetry `akka-http.http-server.responses-3xx` Prometheus `akka_http_http_server_responses_3xx` Elastic `http-servers.responses-3xx` Config `cinnamon.akka.http.meta.descriptor { server { responses-3xx { key = "responses-3xx" unit-type = "custom" unit-suffix = "response" } } }`
Outgoing responses 4xx	Rate	the rate of responses with 400-499 status codes per server OpenTelemetry `akka-http.http-server.responses-4xx` Prometheus `akka_http_http_server_responses_4xx` Elastic `http-servers.responses-4xx` Config `cinnamon.akka.http.meta.descriptor { server { responses-4xx { key = "responses-4xx" unit-type = "custom" unit-suffix = "response" } } }`
Outgoing responses 5xx	Rate	the rate of responses with 500-599 status codes per server OpenTelemetry `akka-http.http-server.responses-5xx` Prometheus `akka_http_http_server_responses_5xx` Elastic `http-servers.responses-5xx` Config `cinnamon.akka.http.meta.descriptor { server { responses-5xx { key = "responses-5xx" unit-type = "custom" unit-suffix = "response" } } }`

Akka Persistence (Classic / Typed)

Metric	Type	Description
Active entities	Counter	number of active entities (in-memory, excludes passivated entities) OpenTelemetry `akka.entity.active-entities` Prometheus `akka_entity_active_entities` Elastic `entities.active-entities` Config `cinnamon.akka.persistence.meta { descriptor { active-entities { key = "active-entities" } } }`
Command in-stash time	Recorder	time that commands spend in the stash. See Akka Persistence internal stash for the details OpenTelemetry `akka.command.in-stash-time` Prometheus `akka_command_in_stash_time_ns` Elastic `commands.in-stash-time` Config `cinnamon.akka.persistence.meta { descriptor { command-in-stash-time { key = "in-stash-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
RecoveryPermitter max permits	GaugeLong	max number of permits (set via Akka configuration) OpenTelemetry `akka.recovery-permitter.max-permits` Prometheus `akka_recovery_permitter_max_permits` Elastic `recovery-permitter.max-permits` Config `cinnamon.akka.persistence.meta { descriptor { max-permits { key = "max-permits" unit-type = "custom" unit-suffix = "permit" } } }`
RecoveryPermitter pending entities	Recorder	number of entities waiting for a permit to start recovery OpenTelemetry `akka.recovery-permitter.pending-entities` Prometheus `akka_recovery_permitter_pending_entities` Elastic `recovery-permitter.pending-entities` Config `cinnamon.akka.persistence.meta { descriptor { pending-entities { key = "pending-entities" unit-type = "custom" unit-suffix = "entity" } } }`
Persist failed	Event	Event created whenever a message persist fails. Event information: actor-ref — the actor failing to persist an event failure — the exception that occurred event — the event that failed if available sequence-number — the event sequence number OpenTelemetry `akka.entity.persist-failed` Prometheus `akka_entity_persist_failed` Elastic `entities.persist-failed` Config `cinnamon.akka.persistence.meta { descriptor { persist-failed { key = "persist-failed" } } }`
Persist rejected	Event	Event created whenever a message persist is rejected. Event information: actor-ref — the actor failing to persist an event failure — the exception that occurred event — the event that failed if available sequence-number — the event sequence number OpenTelemetry `akka.entity.persist-rejected` Prometheus `akka_entity_persist_rejected` Elastic `entities.persist-rejected` Config `cinnamon.akka.persistence.meta { descriptor { persist-rejected { key = "persist-rejected" } } }`
Persisted events	Rate	number of persisted events acknowledged by the journal OpenTelemetry `akka.event.persisted-events` Prometheus `akka_event_persisted_events` Elastic `events.persisted-events` Config `cinnamon.akka.persistence.meta { descriptor { persisted-events { key = "persisted-events" unit-type = "custom" unit-suffix = "event" } } }`
Event persistence time	Recorder	time from the moment the persist/persistAll method was called to the moment when it was acknowledged by the journal OpenTelemetry `akka.event.persistence-time` Prometheus `akka_event_persistence_time_ns` Elastic `events.persistence-time` Config `cinnamon.akka.persistence.meta { descriptor { event-persistence-time { key = "persistence-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Command processing time	Recorder	time for processing the command excluding time spent in the mailbox or stash OpenTelemetry `akka.command.processing-time` Prometheus `akka_command_processing_time_ns` Elastic `commands.processing-time` Config `cinnamon.akka.persistence.meta { descriptor { command-processing-time { key = "processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Received commands	Rate	number of received commands OpenTelemetry `akka.command.received-commands` Prometheus `akka_command_received_commands` Elastic `commands.received-commands` Config `cinnamon.akka.persistence.meta { descriptor { received-commands { key = "received-commands" unit-type = "custom" unit-suffix = "command" } } }`
Recovery failed	Event	Event created whenever a message replay fails. Event information: actor-ref — the actor trying to recover failure — the exception that occurred event — the event that failed if available recovery-failure-time — the time in nanoseconds OpenTelemetry `akka.entity.recovery-failed` Prometheus `akka_entity_recovery_failed` Elastic `entities.recovery-failed` Config `cinnamon.akka.persistence.meta { descriptor { recovery-failed { key = "recovery-failed" } } }`
Recovery failure time	Recorder	failure recovery time for a persistent entity (in case of failure) OpenTelemetry `akka.entity.recovery-failure-time` Prometheus `akka_entity_recovery_failure_time_ns` Elastic `entities.recovery-failure-time` Config `cinnamon.akka.persistence.meta { descriptor { recovery-failure-time { key = "recovery-failure-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Recovery time	Recorder	total time for recovery of a persistent entity (includes waiting for a recovery permit, and fetching and replaying events and snapshots from the journal) OpenTelemetry `akka.entity.recovery-time` Prometheus `akka_entity_recovery_time_ns` Elastic `entities.recovery-time` Config `cinnamon.akka.persistence.meta { descriptor { recovery-time { key = "recovery-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
RecoveryPermitter used permits	Recorder	number of used permits OpenTelemetry `akka.recovery-permitter.used-permits` Prometheus `akka_recovery_permitter_used_permits` Elastic `recovery-permitter.used-permits` Config `cinnamon.akka.persistence.meta { descriptor { used-permits { key = "used-permits" unit-type = "custom" unit-suffix = "permit" } } }`

Akka Projection

Metric	Type	Description
Handler Error	Rate	error rate during the envelope processing. If you use a handler strategy including a retry operation this number will increase OpenTelemetry `akka-projection.projection.handler-error` Prometheus `akka_projection_projection_handler_error` Elastic `projections.handler-error` Config `cinnamon.akka.projection.meta.descriptor { projection { handler-error { key = "handler-error" } } }`
Handler Error Warning	Event	handler error event OpenTelemetry `akka-projection.projection.handler-error-event` Prometheus `akka_projection_projection_handler_error_event` Elastic `projections.handler-error-event` Config `cinnamon.akka.projection.meta.descriptor { projection { handler-error-event { key = "handler-error-event" } } }`
Handler Success	Counter	number of processed envelopes or number of times an envelope handler ran successfully OpenTelemetry `akka-projection.projection.handler-success` Prometheus `akka_projection_projection_handler_success` Elastic `projections.handler-success` Config `cinnamon.akka.projection.meta.descriptor { projection { handler-success { key = "handler-success" unit-type = "custom" unit-suffix = "envelope" } } }`
Offset Success	Counter	number of envelopes marked as processed or number of offset positions advanced OpenTelemetry `akka-projection.projection.offset-success` Prometheus `akka_projection_projection_offset_success` Elastic `projections.offset-success` Config `cinnamon.akka.projection.meta.descriptor { projection { offset-success { key = "offset-success" unit-type = "custom" unit-suffix = "envelope" } } }`
Projection Count	Counter	number of running projections. This is a counter of instances so if you run a sharded projection the counter will indicate the number of shards. OpenTelemetry `akka-projection.projection.projection-count` Prometheus `akka_projection_projection_projection_count` Elastic `projections.projection-count` Config `cinnamon.akka.projection.meta.descriptor { projection { projection-count { key = "projection-count" } } }`
Projection Failure	Rate	failure rate for projections. Failures indicate the projection required restarting OpenTelemetry `akka-projection.projection.projection-failure` Prometheus `akka_projection_projection_projection_failure` Elastic `projections.projection-failure` Config `cinnamon.akka.projection.meta.descriptor { projection { projection-failure { key = "projection-failure" } } }`
Projection Failure Warning	Event	projection failure event OpenTelemetry `akka-projection.projection.projection-failure-event` Prometheus `akka_projection_projection_projection_failure_event` Elastic `projections.projection-failure-event` Config `cinnamon.akka.projection.meta.descriptor { projection { projection-failure-event { key = "projection-failure-event" } } }`
Residence Time	Recorder	time (in milliseconds) spent since the creation of the envelope until its offset was committed OpenTelemetry `akka-projection.projection.residence-time` Prometheus `akka_projection_projection_residence_time` Elastic `projections.residence-time` Config `cinnamon.akka.projection.meta.descriptor { projection { residence-time { key = "residence-time" } } }`
Service Time	Recorder	time (in nanoseconds) from the instant the envelope was considered ready until the projected data was available for reading. Note this time doesn’t include the period until data being readable and the offset being committed. As a consequence, when using `at-least-once` semantics, this metric will be reported for every envelope handler completing successfully (even if storing the offset failed and the envelope is retried) OpenTelemetry `akka-projection.projection.service-time` Prometheus `akka_projection_projection_service_time_ns` Elastic `projections.service-time` Config `cinnamon.akka.projection.meta.descriptor { projection { service-time { key = "service-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Wait Time	Recorder	time (in milliseconds) an envelope spent on the queue/broker/journal. This is the time since the envelope was created until it was read, deserialized, and considered ready to be processed by the projection. This metric depends on the type of source since some sources don’t provide the creation timestamp. At the same time, some producers record the creation time in the producing process, while others record the timestamp on the database/broker when accepting the envelope. Note this metric is affected by existing time skew between nodes on your infrastructure OpenTelemetry `akka-projection.projection.wait-time` Prometheus `akka_projection_projection_wait_time` Elastic `projections.wait-time` Config `cinnamon.akka.projection.meta.descriptor { projection { wait-time { key = "wait-time" } } }`

Akka Router

Metric	Type	Description
Router processing time	Recorder	statistics for the processing time of the router logic OpenTelemetry `akka.message.router-processing-time` Prometheus `akka_message_router_processing_time_ns` Elastic `messages.router-processing-time` Config `cinnamon.akka.meta.descriptor { router { router-processing-time { key = "router-processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Router routed messages	Rate	the number of messages that routers have processed in the selected time frame OpenTelemetry `akka.message.router-routed-messages` Prometheus `akka_message_router_routed_messages` Elastic `messages.router-routed-messages` Config `cinnamon.akka.meta.descriptor { router { router-routed-messages { key = "router-routed-messages" unit-type = "custom" unit-suffix = "msg" } } }`

Akka Stream

Metric	Type	Description
Akka Stream boundary demand	Rate	the rate of demand on an internal async boundary (includes batched demand requests) OpenTelemetry `akka-stream.boundary.demand` Prometheus `akka_stream_boundary_demand` Elastic `boundaries.demand` Config `cinnamon.akka.stream.meta.descriptor { boundary { demand { key = "demand" unit-type = "custom" unit-suffix = "element" } } }`
Akka Stream boundary demand processing time	Recorder	the time for processing demand requests on an internal async boundary OpenTelemetry `akka-stream.boundary.demand-processing-time` Prometheus `akka_stream_boundary_demand_processing_time_ns` Elastic `boundaries.demand-processing-time` Config `cinnamon.akka.stream.meta.descriptor { boundary { demand-processing-time { key = "demand-processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream boundary processing time	Recorder	the time for processing elements pushed downstream on an internal async boundary OpenTelemetry `akka-stream.boundary.processing-time` Prometheus `akka_stream_boundary_processing_time_ns` Elastic `boundaries.processing-time` Config `cinnamon.akka.stream.meta.descriptor { boundary { processing-time { key = "processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream boundary throughput	Rate	the rate of events that have been processed on an internal async boundary OpenTelemetry `akka-stream.boundary.throughput` Prometheus `akka_stream_boundary_throughput` Elastic `boundaries.throughput` Config `cinnamon.akka.stream.meta.descriptor { boundary { throughput { key = "throughput" unit-type = "custom" unit-suffix = "element" } } }`
Akka Stream connection demand	Rate	the rate of demand on a connection OpenTelemetry `akka-stream.connection.demand` Prometheus `akka_stream_connection_demand` Elastic `connections.demand` Config `cinnamon.akka.stream.meta.descriptor { connection { demand { key = "demand" unit-type = "custom" unit-suffix = "element" } } }`
Akka Stream connection demand latency	Recorder	the time spent waiting for demand from downstream (the time from providing an element downstream with a push to starting to process demand from a pull) OpenTelemetry `akka-stream.connection.demand-latency` Prometheus `akka_stream_connection_demand_latency_ns` Elastic `connections.demand-latency` Config `cinnamon.akka.stream.meta.descriptor { connection { demand-latency { key = "demand-latency" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream connection demand processing time	Recorder	the time for processing demands on a connection OpenTelemetry `akka-stream.connection.demand-processing-time` Prometheus `akka_stream_connection_demand_processing_time_ns` Elastic `connections.demand-processing-time` Config `cinnamon.akka.stream.meta.descriptor { connection { demand-processing-time { key = "demand-processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream connection latency	Recorder	the time spent waiting for elements from upstream (the time from demanding an element upstream with a pull to starting to process an element from a push) OpenTelemetry `akka-stream.connection.latency` Prometheus `akka_stream_connection_latency_ns` Elastic `connections.latency` Config `cinnamon.akka.stream.meta.descriptor { connection { latency { key = "latency" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream connection processing time	Recorder	the time for processing elements pushed downstream on a connection OpenTelemetry `akka-stream.connection.processing-time` Prometheus `akka_stream_connection_processing_time_ns` Elastic `connections.processing-time` Config `cinnamon.akka.stream.meta.descriptor { connection { processing-time { key = "processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream connection throughput	Rate	the rate of elements that have been processed on a connection OpenTelemetry `akka-stream.connection.throughput` Prometheus `akka_stream_connection_throughput` Elastic `connections.throughput` Config `cinnamon.akka.stream.meta.descriptor { connection { throughput { key = "throughput" unit-type = "custom" unit-suffix = "element" } } }`
Akka Stream flow active time	Recorder	the active time during the flow, provided by the Stopwatch, and allowing flow efficiency to be measured OpenTelemetry `akka-stream.stream.flow-active-time` Prometheus `akka_stream_stream_flow_active_time_ns` Elastic `streams.flow-active-time` Config `cinnamon.akka.stream.meta.descriptor { stream { flow-active-time { key = "flow-active-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream flow efficiency	Recorder	the efficiency of end-to-end flows, based on how much time was spent actively processing as compared with waiting. Recorded as a percentage using active time / total flow time OpenTelemetry `akka-stream.stream.flow-efficiency` Prometheus `akka_stream_stream_flow_efficiency` Elastic `streams.flow-efficiency` Config `cinnamon.akka.stream.meta.descriptor { stream { flow-efficiency { key = "flow-efficiency" unit-type = "custom" unit-suffix = "percent" } } }`
Akka Stream end-to-end flow time	Recorder	the time for elements to flow through the stream from sources to sinks (uses a Stopwatch internally) OpenTelemetry `akka-stream.stream.flow-time` Prometheus `akka_stream_stream_flow_time_ns` Elastic `streams.flow-time` Config `cinnamon.akka.stream.meta.descriptor { stream { flow-time { key = "flow-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operator demand	Rate	the rate of demand on an operator OpenTelemetry `akka-stream.operator.demand` Prometheus `akka_stream_operator_demand` Elastic `operators.demand` Config `cinnamon.akka.stream.meta.descriptor { operator { demand { key = "demand" unit-type = "custom" unit-suffix = "element" } } }`
Akka Stream operator demand latency	Recorder	the time spent waiting for demand from downstream (the time from providing an element downstream with a push to starting to process demand from a pull) OpenTelemetry `akka-stream.operator.demand-latency` Prometheus `akka_stream_operator_demand_latency_ns` Elastic `operators.demand-latency` Config `cinnamon.akka.stream.meta.descriptor { operator { demand-latency { key = "demand-latency" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operator demand processing time	Recorder	the time for processing demands on an operator OpenTelemetry `akka-stream.operator.demand-processing-time` Prometheus `akka_stream_operator_demand_processing_time_ns` Elastic `operators.demand-processing-time` Config `cinnamon.akka.stream.meta.descriptor { operator { demand-processing-time { key = "demand-processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operator flow time	Recorder	the time for elements to flow through the stream from sources to being completed by an operator, providing a progress metric for flows OpenTelemetry `akka-stream.operator.flow-time` Prometheus `akka_stream_operator_flow_time_ns` Elastic `operators.flow-time` Config `cinnamon.akka.stream.meta.descriptor { operator { flow-time { key = "flow-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operator latency	Recorder	the time spent waiting for elements from upstream (the time from demanding an element upstream with a pull to starting to process an element from a push) OpenTelemetry `akka-stream.operator.latency` Prometheus `akka_stream_operator_latency_ns` Elastic `operators.latency` Config `cinnamon.akka.stream.meta.descriptor { operator { latency { key = "latency" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
OperatorFailure	Event	event when an operator fails OpenTelemetry `akka-stream.operator.operator-failure` Prometheus `akka_stream_operator_operator_failure` Elastic `operators.operator-failure` Config `cinnamon.akka.stream.meta.descriptor { operator { operator-failure { key = "operator-failure" } } }`
Akka Stream operator processing time	Recorder	the time for processing elements through an operator OpenTelemetry `akka-stream.operator.processing-time` Prometheus `akka_stream_operator_processing_time_ns` Elastic `operators.processing-time` Config `cinnamon.akka.stream.meta.descriptor { operator { processing-time { key = "processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operators (running)	Counter	the number of operator instances currently running (as identified by reporting configuration) OpenTelemetry `akka-stream.operator.running-operators` Prometheus `akka_stream_operator_running_operators` Elastic `operators.running-operators` Config `cinnamon.akka.stream.meta.descriptor { operator { running-operators { key = "running-operators" unit-type = "custom" unit-suffix = "operator" } } }`
Akka Stream operator throughput	Rate	the rate of elements that have been processed by an operator OpenTelemetry `akka-stream.operator.throughput` Prometheus `akka_stream_operator_throughput` Elastic `operators.throughput` Config `cinnamon.akka.stream.meta.descriptor { operator { throughput { key = "throughput" unit-type = "custom" unit-suffix = "element" } } }`
Akka Stream operators (running)	Counter	the total number of operators currently running per stream (as identified by reporting configuration) OpenTelemetry `akka-stream.stream.running-operators` Prometheus `akka_stream_stream_running_operators` Elastic `streams.running-operators` Config `cinnamon.akka.stream.meta.descriptor { stream { running-operators { key = "running-operators" unit-type = "custom" unit-suffix = "operator" } } }`
Akka Streams (running)	Counter	the number of streams currently running (as identified by reporting configuration) OpenTelemetry `akka-stream.stream.running-streams` Prometheus `akka_stream_stream_running_streams` Elastic `streams.running-streams` Config `cinnamon.akka.stream.meta.descriptor { stream { running-streams { key = "running-streams" unit-type = "custom" unit-suffix = "stream" } } }`
Akka Stream throughput	Rate	the rate of elements being processed by a stream as a whole (measured at the sinks) OpenTelemetry `akka-stream.stream.throughput` Prometheus `akka_stream_stream_throughput` Elastic `streams.throughput` Config `cinnamon.akka.stream.meta.descriptor { stream { throughput { key = "throughput" unit-type = "custom" unit-suffix = "element" } } }`

Ask Pattern

Metric	Type	Description
Asked requests	Rate	the number of messages that actors have been asked in the selected time frame OpenTelemetry `akka.message.asked-requests` Prometheus `akka_message_asked_requests` Elastic `messages.asked-requests` Config `cinnamon.akka.ask-pattern.meta.descriptor { ask-pattern { asked-requests { key = "asked-requests" unit-type = "custom" unit-suffix = "msg" } } }`
Expired request event	Event	when request expires before the response arrived Event information: actor-ref — the actor being asked the message message — the message being sent timeout — the given timeout OpenTelemetry `akka.actor.expired-request-event` Prometheus `akka_actor_expired_request_event` Elastic `actors.expired-request-event` Config `cinnamon.akka.ask-pattern.meta.descriptor { ask-pattern { expired-request-event { key = "expired-request-event" } } }`
Expired requests	Rate	the number of requests that timed out in the selected time frame grouped by specified timeout values OpenTelemetry `akka.message.expired-requests` Prometheus `akka_message_expired_requests` Elastic `messages.expired-requests` Config `cinnamon.akka.ask-pattern.meta.descriptor { ask-pattern { expired-requests { key = "expired-requests" unit-type = "custom" unit-suffix = "msg" } } }`
Expired requests	Rate	the number of requests that timed out in the selected time frame grouped by specified timeout values OpenTelemetry `akka.timeout.expired-requests` Prometheus `akka_timeout_expired_requests` Elastic `timeouts.expired-requests` Config `cinnamon.akka.ask-pattern.meta.descriptor { ask-pattern { expired-requests-by-timeout { key = "expired-requests" unit-type = "custom" unit-suffix = "msg" } } }`
Failed request event	Event	when the actor responses with akka.actor.Status.Failure (only Akka Classic) Event information: actor-ref — the actor being asked the message message — the message being sent cause — the exception returned with a failure response OpenTelemetry `akka.actor.failed-request-event` Prometheus `akka_actor_failed_request_event` Elastic `actors.failed-request-event` Config `cinnamon.akka.ask-pattern.meta.descriptor { ask-pattern { failed-request-event { key = "failed-request-event" } } }`
Late response time	Recorder	the time from the moment the message was sent till the moment when the answer was received, counting only responses received after timeout OpenTelemetry `akka.message.late-response-time` Prometheus `akka_message_late_response_time_ns` Elastic `messages.late-response-time` Config `cinnamon.akka.ask-pattern.meta.descriptor { ask-pattern { late-response-time { key = "late-response-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Success response time	Recorder	the time from the moment the message was sent till the moment when the answer was received and before the specified timeout, counting only successful responses OpenTelemetry `akka.message.success-response-time` Prometheus `akka_message_success_response_time_ns` Elastic `messages.success-response-time` Config `cinnamon.akka.ask-pattern.meta.descriptor { ask-pattern { success-response-time { key = "success-response-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.

Cinnamon Stopwatch

Metric	Type	Description
Stopwatch active time	Recorder	the time spent actively in actor message processing, stream operators, or future callbacks, as compared with the time in actor mailboxes or waiting to be scheduled OpenTelemetry `stopwatch.stopwatch-active-time` Prometheus `stopwatch_stopwatch_active_time_ns` Elastic `stopwatches.stopwatch-active-time` Config `cinnamon.akka.meta.descriptor { stopwatch { stopwatch-active-time { key = "stopwatch-active-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Stopwatch rate	Rate	the rate of stopwatch runs OpenTelemetry `stopwatch.stopwatch-rate` Prometheus `stopwatch_stopwatch_rate` Elastic `stopwatches.stopwatch-rate` Config `cinnamon.akka.meta.descriptor { stopwatch { stopwatch-rate { key = "stopwatch-rate" unit-type = "custom" unit-suffix = "event" } } }`
Stopwatch time	Recorder	the total elapsed time between the start and stop endpoints of a flow OpenTelemetry `stopwatch.stopwatch-time` Prometheus `stopwatch_stopwatch_time_ns` Elastic `stopwatches.stopwatch-time` Config `cinnamon.akka.meta.descriptor { stopwatch { stopwatch-time { key = "stopwatch-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
StopwatchTimeLimit	Event	stopwatch interval threshold exceeded event with the limit and actual time in nanos OpenTelemetry `stopwatch.stopwatch-time-limit` Prometheus `stopwatch_stopwatch_time_limit` Elastic `stopwatches.stopwatch-time-limit` Config `cinnamon.akka.meta.descriptor { stopwatch { stopwatch-time-limit { key = "stopwatch-time-limit" } } }`

Java Future

Metric	Type	Description
Java Future completing time	Recorder	the time from creation to completing the CompletableFuture OpenTelemetry `java-future.completing-time` Prometheus `java_future_completing_time_ns` Elastic `java-futures.completing-time` Config `cinnamon.java.future.meta.descriptor { future { completing-time { key = "completing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Java Futures created	Rate	the rate of CompletableFutures being created (for a given name) OpenTelemetry `java-future.created-futures` Prometheus `java_future_created_futures` Elastic `java-futures.created-futures` Config `cinnamon.java.future.meta.descriptor { future { created-futures { key = "created-futures" unit-type = "custom" unit-suffix = "future" } } }`
Java Futures processed	Rate	the rate of futures/callbacks being processed (marked when the future/callback has finished running) OpenTelemetry `java-future.processed-futures` Prometheus `java_future_processed_futures` Elastic `java-futures.processed-futures` Config `cinnamon.java.future.meta.descriptor { future { processed-futures { key = "processed-futures" unit-type = "custom" unit-suffix = "future" } } }`
Java Future processing time	Recorder	the time that futures/callbacks take to run OpenTelemetry `java-future.processing-time` Prometheus `java_future_processing_time_ns` Elastic `java-futures.processing-time` Config `cinnamon.java.future.meta.descriptor { future { processing-time { key = "processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Java Futures scheduled	Recorder	the number of futures currently scheduled (includes both future async sources and completion callbacks) OpenTelemetry `java-future.scheduled-futures` Prometheus `java_future_scheduled_futures` Elastic `java-futures.scheduled-futures` Config `cinnamon.java.future.meta.descriptor { future { scheduled-futures { key = "scheduled-futures" unit-type = "custom" unit-suffix = "future" } } }`
Java Future scheduling time	Recorder	the time that futures/callbacks are in scheduling (in executor queues waiting to run) OpenTelemetry `java-future.scheduling-time` Prometheus `java_future_scheduling_time_ns` Elastic `java-futures.scheduling-time` Config `cinnamon.java.future.meta.descriptor { future { scheduling-time { key = "scheduling-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.

Scala Future

Metric	Type	Description
Scala Future completing time	Recorder	the time from creation to completing the underlying Promise for this Future OpenTelemetry `scala-future.completing-time` Prometheus `scala_future_completing_time_ns` Elastic `scala-futures.completing-time` Config `cinnamon.scala.future.meta.descriptor { future { completing-time { key = "completing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Scala Futures created	Rate	the rate of Futures being created (for a given name) OpenTelemetry `scala-future.created-futures` Prometheus `scala_future_created_futures` Elastic `scala-futures.created-futures` Config `cinnamon.scala.future.meta.descriptor { future { created-futures { key = "created-futures" unit-type = "custom" unit-suffix = "future" } } }`
Scala Futures processed	Rate	the rate of Futures/callbacks being processed (marked when the Future/callback has finished running) OpenTelemetry `scala-future.processed-futures` Prometheus `scala_future_processed_futures` Elastic `scala-futures.processed-futures` Config `cinnamon.scala.future.meta.descriptor { future { processed-futures { key = "processed-futures" unit-type = "custom" unit-suffix = "future" } } }`
Scala Future processing time	Recorder	the time that Futures/callbacks take to run OpenTelemetry `scala-future.processing-time` Prometheus `scala_future_processing_time_ns` Elastic `scala-futures.processing-time` Config `cinnamon.scala.future.meta.descriptor { future { processing-time { key = "processing-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.
Scala Futures scheduled	Recorder	the number of Futures currently scheduled (includes both Future runnables and Future callbacks) OpenTelemetry `scala-future.scheduled-futures` Prometheus `scala_future_scheduled_futures` Elastic `scala-futures.scheduled-futures` Config `cinnamon.scala.future.meta.descriptor { future { scheduled-futures { key = "scheduled-futures" unit-type = "custom" unit-suffix = "future" } } }`
Scala Future scheduling time	Recorder	the time that Futures/callbacks are in scheduling (in execution context queues waiting to run) OpenTelemetry `scala-future.scheduling-time` Prometheus `scala_future_scheduling_time_ns` Elastic `scala-futures.scheduling-time` Config `cinnamon.scala.future.meta.descriptor { future { scheduling-time { key = "scheduling-time" unit-type = "nanoseconds" } } }` Unit conversion OpenTelemetry duration metrics are converted to seconds by default.