By Henrik Engström (@h3nk3), November 7, 2017
A new version of Lightbend Telemetry, a.k.a. Cinnamon, with support for Netflix Vizceral visualization, has been released.
Vizceral from Netflix is a component for displaying traffic data. Although Netflix created Vizceral to get better insights into how traffic flows between the elements in their systems, the same visualization can be applied to the domain of actors.
Vizceral gives an intuitive view of the components comprising a system. Using flow visualizations in combination with metrics, events, and distributed tracing can provide useful insights into what is going on in an Akka application:
- Flow visualizations: Provides an overview of what components there are in a system, and how they communicate with each other. Vizceral is one example of this.
- Distributed tracing: Enables detailed information about specific traces. For example, Zipkin or Jaeger.
- Metrics: Can be used to look at historical trends for various parts in a system. For example, OpsClarity or Grafana.
- Events: Gives the user possibilities to examine, in detail, individual discrete events. For example, Elasticsearch with Kibana.
The UI is layered into three levels: nodes, actor system, and actor view. Any connection between entities is illustrated as a line.
Messages are represented as dots flowing between entities. Vizceral uses a rate of messages/second to determine the number of messages visualized. It is crucial to understand that there is no 1-1 relationship between a visualized message and the underlying message in Akka.
Regular messages are represented in light blue, warnings are in orange, and errors in red color.
Please note that the Vizceral UI does not represent an accurate representation of the message flow, as the visualization allows for messages to pass each other, something that does not happen with Akka messages.
The node view visualizes communication between “nodes” which in our domain means actor systems running on a
host_port. The example below shows three nodes:
Clicking on a node in this layer will open up a table which contains the underlying entities, which in our case means actors, incoming, and outgoing connections, i.e., nodes, with an error percentage.
The actor system view shows not only inter-actor communication but also actor-node communication. In the example below, we can see that actor
actor.RegionManagerClient sends messages to node
Hovering above an actor will highlight its connections. The hovering feature simplifies identifying relations when there are numerous connections in the actor system.
One powerful feature is that dead letters in the actor system are visualized as connections to the
deadLetters entity. This feature will enable us to quickly identify any dead letters and what actors are causing them.
Also, clicking on an actor will open up a detailed view (not shown in the example below.)
To navigate downwards in the layers, we select an entity and double-click on it. This action will take us the layer below. The breadcrumb, upper left corner, can be used to go back up one or more layers.
Apart from illustrating navigation between layers, the example below also shows the actor view layer. This is the lowest of all layers, and it shows the selected actor in the middle and any entities that communicate with it.
More information about how to get started.
- Producers: Allow for custom metrics and events to be added to the Cinnamon Backends.
- JMX Importer: Enables fetching data out of JMX MBeans and create metrics out of these via configuration.
As always, we are interested in feedback on how we can improve giving you the necessary insight into your applications.