Akka

Akka metrics come from Cinnamon, and describe performance of actors and http endpoints.

akka_inbox_growth

Akka applications have actors, and each actor has a mailbox. An actor mailbox is a kind of queue for work to be done. The akka_inbox_growth monitor alerts if that queue size trends upward over time, meaning the actor is not keeping up with incoming requests.

Cinnamon provides a measure of mailbox size as akka_actor_mailbox_size.

Cinnamon actor level data typically is configured as an aggregate by actor class, in which case the monitor says “some actor in this class has mailbox growth problems” but not which instance exactly.

akka_processing_time

Akka actors take one message at a time from their mailbox and “process” that message before taking the next one. The akka_processing_time monitor warns if that processing time is unusual compared to previous performance. This warning occurs if processing time rises or drops more than two standard deviations from average processing time, which by the empirical rule stands for a 95% chance that something unusual is happening.

Cinnamon provides a timer measuring how much time actors take to process messages as akka_actor_processing_time_ns.

This underlying metric is typically aggregated by actor class.

akka_http_server_response_time

Akka HTTP servers take remote requests, initiate some work, and then respond with an HTTP code to the remote requestor. The akka_http_server_response_time monitor warns if that time is anomalous compared to average response time, in similar manner to the akka_processing_time monitor.

Cinnamon provides a timer measuring time to respond as akka_http_http_server_response_time_ns

akka_http_client_response_time

Akka applications may communicate with remote HTTP servers by making requests and waiting for a response. The akka_http_client_response_time monitor warns if that waiting time is anomalous compared to average wait time, in similar manner to the akka_processing_time monitor.

Cinnamon provides a timer measuring client wait time as akka_http_http_client_http_client_service_response_time_ns

akka_http_server_5xx

Akka HTTP server endpoints may receive requests that cause crashes or otherwise result in 500 class HTTP responses. The akka_http_server_5xx monitor warns if such 5xx errors are observed continuously for 5 minutes.

Cinnamon provides a gauge for 5xx errors per second as akka_http_http_server_responses_5xx_rate