Two more families of failure never reach the application: the container's configuration cannot be assembled, or its identity is not allowed to do what it tries.

CreateContainerConfigError

kubectl get pods
kubectl describe pod market-api-7b6c-x2k4p | grep Warning

Warning  Failed  Error: couldn't find key DB_HOST in ConfigMap wrecks/market-config

The image pulled, the Pod was scheduled, and the kubelet stopped while building the container: a ConfigMap or Secret named in env, envFrom or volumes does not exist, or exists without the key asked for. The message names it. Fix either side, add the key or change the reference, and the kubelet retries on its own.

Marking a reference optional: true lets the container start without it, which is right for some settings and a silent failure for others.

A missing ServiceAccount

A Deployment whose template names a ServiceAccount that does not exist produces no Pods at all. The ReplicaSet's Events say why:

kubectl describe rs -l app=market-web | tail -2

Warning  FailedCreate  Error creating: pods "market-web-5f7d-" is forbidden: error looking up service account wrecks/web-runner: serviceaccount "web-runner" not found

Create the account, or point the template at one that exists.

Forbidden

A Pod that runs but logs Forbidden from the API is asking for something its ServiceAccount may not do. Confirm before you change anything:

kubectl auth can-i list pods --as=system:serviceaccount:wrecks:deployer
kubectl auth can-i --list --as=system:serviceaccount:wrecks:deployer

Then bind the smallest Role that covers it, as in Chapter 10. The other common cause is the right Role bound to the wrong account: check serviceAccountName in the Pod spec against the binding's subjects.

Immutable fields

Some fixes are refused with field is immutable: a Deployment's selector, a Service's clusterIP, a PVC's size downwards, a Job's template. The way through is to delete and recreate the object, after checking what depends on it.