Because pods and their containers are in constant motion and dynamically scheduled, monitoring tools rely on services as their endpoint. Services expose an IP address that can be accessed from outside the pods, allowing services to communicate on a continual basis even as individual pods and containers are created and deleted.

The health of Kubernetes nodes indicates the ability of the node to run pods assigned to it. Kubernetes provides a node problem detector DaemonSet, which aggregates information on problems from node metrics daemons and sends them back to the APIserver reported as node conditions and events.

What is Prometheus?

Prometheus is an open source system by the CNCF for collecting metrics on Kubernetes health. Prometheus installs data exporter pods on each node in the cluster, and its server collects data from nodes, pods, and jobs. The collected time-series data is saved into a database, and alerts can be automatically generated based on predefined conditions.

Prometheus has its own dashboard with limited capabilities, which have been enhanced by the use of external visualization tools like Grafana, which use the Prometheus database to enable sophisticated inquires, debugging, and reporting that can be tailored for dev, test, and production teams.

Prometheus offers support for bridging in data from other tools and can connect to a broad range of third-party databases.

What is the Kubernetes Dashboard?

The Kubernetes dashboard provides a way to manage cluster resources and debug containerized applications using a simple web interface. The Kubernetes dashboard offers a simple overview of resources both cluster-wide and on individual nodes. It also provides a rundown of all the namespaces in the cluster as well as all of the storage classes that have been defined. Some uses for the dashboard include:

• Adminview lists all the nodes and persistent storage volumes along with aggregated metrics for each node.
• Configand storage view identify persistent volume claims for each clustered application and all the Kubernetes resources running in the cluster.
• Workload view lists every application running by namespace, including current pod memory usage and the number of pods currently ready in a Deployment.
• Discoverview lists services that have been exposed to the outside world and have enabled discovery within the cluster.

What is Kubernetes Pod Health? 

There are three measures of the health of a Kubernetes pod: liveness, readiness, and startup condition. They are determined by probes that are managed by the kubelet.

The liveness probe determines if a container within a pod needs to restart. It can help identify when pods have become unresponsive.

The readiness probes indicate to the cluster when containers in the pod are ready to begin processing traffic. The pod is considered ready only when all of the containers in a pod are ready. If a pod is not ready, it will be removed from Service load balancers and will not receive incoming traffic.

The startup probe indicates when the application in the pod has started successfully. If a startup probe is in effect, both liveness and readiness probes will be disabled until it ensures other probes do not interfere with the startup process. 

What is cAdvisor?

cAdvisor collects metrics about resource usage, historical data, and resource isolation at the cluster level and down to the container level. Its data collection is used as the underlying technology for many other metrics collection systems.

What is Kubernetes Application Health?

Since Kubernetes applications always run in application pods, the health of the applications can be measured by the liveness and readiness probes in the respective pods. If applications report they are ready to process new requests, and the node on which they are running is not reporting any errors, the application is likely healthy.