Chapter 6 set requests and limits and glanced at kubectl top. This lesson is about reading it well.

Pods and nodes

kubectl top pods -n wastes
kubectl top nodes

NAME                          CPU(cores)   MEMORY(bytes)
market-db-5f7d8c9b4-m2k8q     12m          130Mi
market-api-7b6c4d5f9-x2k4p    9m           64Mi
market-web-6d9f8c7b5-q8r1z    2m           12Mi

NAME              CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
shallows-node-1   103m         5%     812Mi           21%

The numbers come from metrics-server, a Deployment in kube-system that asks each kubelet what its containers use. No metrics-server, no top: the error is metrics not available yet, also what you see for the first minute after a Pod starts.

Finding the hungry one

kubectl top pods -n wastes --sort-by=memory
kubectl top pods -n wastes --sort-by=cpu
kubectl top pods -A --sort-by=memory | head -5

--sort-by sorts descending. With -A and head it answers "what is the biggest thing in the cluster" in one line.

Per container

kubectl top pods market-db-5f7d8c9b4-m2k8q -n wastes --containers

POD                         NAME       CPU(cores)   MEMORY(bytes)
market-db-5f7d8c9b4-m2k8q   postgres   12m          128Mi
market-db-5f7d8c9b4-m2k8q   exporter   1m           6Mi

A Pod's total hides which container is growing. --containers splits it.

What it is not

A Pod that was OOMKilled a moment ago can show a modest number, because the spike fell between two samples. Compare top with the limit in describe pod to see how much headroom a container really has, and treat anything above eighty percent as a limit about to bite.