peaks-node-3 is reserved for GPU work. A training Pod in peaks-9gokpq is Pending:
sh
kubectl get pods -n peaks-9gokpq -o wide
kubectl describe pod trainer -n peaks-9gokpq
kubectl describe node peaks-node-3 | grep TaintsThe Pod's manifest is in ~/peaks-9gokpq/trainer.yaml. Add a toleration for the taint you found, delete the Pod and apply the file:
yaml
tolerations:
- key: dedicated
operator: Equal
value: gpu
effect: NoScheduleThen take a node out of service the hard way. market-web has a replica on peaks-node-2:
sh
kubectl taint nodes peaks-node-2 maintenance=true:NoExecute
kubectl get pods -n peaks-9gokpq -o wide