Troubleshooting
Known Issues
Workload Name Length Limitation
This will lead to a constant scrolling grey bar for workload health, where workloads never show a health status.
This is due to a known issue with Console that restricts the length of workload names. If your deployment name exceeds 57
(47
in 1.1.0 and earlier) characters, Console will be unable to determine the workload name, and be unable to render the health of the workload.
This is because Kubernetes truncates the pod name to keep it under 64 characters, while allowing room for the pod hash - and Console relies on the pod name to determine the workload name.
Until this is resolved in Console, the workaround is to limit the length of deployment names.
lightbend_monitor_api_errors
An alert from lightbend_monitor_api_errors
indicates that the console failed to process one or more monitors. This can occur, for example, when you upgrade the console and it cannot handle an existing monitor. This alert warns when the metric is greater than 0.
If this alert fires, please perform a debug dump with the installer script and open a support ticket with the dump file attached:
./lbc.py debug-dump --namespace=<console namespace>
OOMKilled
Containers and Pods
It is possible that your use of Console requires more resources than those specified in the defaults. If you have containers and pods terminated with an OOMKilled
reason, try to increase their memory limit. For example, you can increase the memory limit for the Console Prometheus container by adding the following setting to your values.yaml
file:
prometheusMemoryRequest: 512Mi
See more details about how to customize Console installation in the install guide.
Make sure to check the memory available for the Kubernetes cluster node. If the sum of all memory limits is higher than node capacity, it can exhaust the memory, causing some pods to terminate to free some memory.