Zookeeper

Introduction

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications.

Please review our support article on how to configure service monitors for instructions on how to initially access the Opsclarity configuration screen and to see which settings there are common to every service.  In this article, we’ll go over all necessary specific details for managing the OpsClarity - ZooKeeper service integration:

 

 

Requirements

ZooKeeper plugin needs the Netcat utility, and on most systems this tool is already present.  It can be installed by executing the following command(s):

  • on Ubuntu:  **sudo apt-get install -y netcat**
  • on CentOS:  **sudo yum install nc.x86_64**

 

 

Configure ZooKeeper Data Sources

There is one default ZooKeeper data source under the Configure Data Sources tab.  You may add additional data sources [1], or edit [2] the default values **[3]** here:

These are the field names (along with their default values) and a detailed description of what should go in each field:

Field Name Default Value Description
Host empty Input the name of the host here.
Port 2181 The particular port that is actually monitored.
Run as Unix user empty To run this operation as another user, populate this field.

  

 

ZooKeeper Default Monitors

For help with adding and configuring new monitors, click here.  There are seven default monitors that are already created for you in a newly added ZooKeeper service:

Monitor Metric Description
Overall Service Health Multiple This is a rollup of the health of all service monitors.
Default Zookeeper Port Monitor  - Reaches out to a specified port with a ping request a specified number of times in a 30 second window. 
ZooKeeper Average Latency latency.average Detects unexpected behavior in Average latency on each ZooKeeper host.
Connection Count connections.service Detects unexpected behavior in Total number of open connections across the ZooKeeper service.
Synced Followers synced.followers.service Detects unexpected behavior in Number of synced followers across the ZooKeeper service.
Pending Syncs pending.syncs.service Detects unexpected behavior in Number of syncs pending across the ZooKeeper service.
Open File Descriptors open.file.descriptor.service Detects unexpected behavior in Percentage open file descriptors across the ZooKeeper service.

 

 

ZooKeeper Metrics

These are the metrics we count for Zookeeper:

Metric Name Units Metric Description
connections.service Connections  Number of connected clients across all Zookeeper nodes. 
connections  Connections Number of connected clients.
followers Nodes Number of follower nodes connected to master.
latency.service Milliseconds Average latency for client requests across all Zookeeper nodes.
latency.average Milliseconds Average latency for client requests.
latency.max Milliseconds Maximum latency for client requests.
local.watches.total Watches Total number of watches on behalf of Zookeeper clients.
local.watches.unique.paths Items Total number of watched items.
max.file.descriptors.service File Descriptors Configured limit on number of file descriptors across all nodes.
max.file.descriptors File Descriptors Configured limit on number of file descriptors.
nodes Nodes Number of znodes in the ZooKeeper namespace.
open.file.descriptor.service File Descriptors Total number of open files across all zk nodes.
open.file.descriptors File Descriptors Total number of open files on the zk node.
requests.outstanding Requests Number of queued requests from clients.
pending.syncs.service Requests Number of queued sync requests between all zk nodes.
pending.syncs Requests Number of queued sync requests from other zk nodes.
server.state State ZooKeeper Mode: leader = 2, follower = 1, unknown = 0
synced.followers.service Nodes Total number of Zookeeper nodes in sync with leaders anywhere in the cluster
synced.followers Nodes Total number of zk nodes in sync with this leader node
if.packets.rx Packets ZooKeeper network reads.
if.packets.tx Packets ZooKeeper network writes.