Production-grade triage, debugging, and recovery with kubectl, targeting kubectl / Kubernetes v1.31+.
Quick Reference
| Task | Command |
|---|---|
| Wide pod status | kubectl get pods -o wide -n <ns> |
| Full object state and events | kubectl describe pod <pod> |
| Recent warnings | kubectl events -n <ns> --types=Warning |
| Logs from the crashed container | kubectl logs <pod> -c <ctr> --previous |
| Tail logs across a label | kubectl logs -l app=web --prefix --since=15m -f |
| Shell into a container | kubectl exec -it <pod> -c <ctr> -- sh |
| Ephemeral debug container | kubectl debug -it <pod> --image=busybox:1.36 --target=<ctr> |
| Node shell | kubectl debug node/<node> -it --image=busybox:1.36 |
| Resource usage | kubectl top pod -n <ns> --sort-by=memory |
| Rollout progress | kubectl rollout status deploy/<name> --timeout=5m |
| Roll back | kubectl rollout undo deploy/<name> |
| Local tunnel | kubectl port-forward svc/<svc> 8080:80 |
| RBAC check | kubectl auth can-i <verb> <resource> --as=<user> |
| Preview changes | kubectl diff -f <file>.yaml |
| Wait for readiness | kubectl wait --for=condition=Ready pod -l app=web --timeout=120s |
Core Patterns
1. Triage a failing pod
Walk from coarse to fine: status, then conditions and events, then logs from the previous container instance. Most crash causes appear in describe before you read any logs.
NS=prod; POD=web-7d9f8c6b5-x2k4p kubectl get pod $POD -n $NS -o wide # STATUS, RESTARTS, NODE, IP kubectl describe pod $POD -n $NS # Conditions, State/Last State, Events kubectl events -n $NS --for pod/$POD # events scoped to this pod kubectl logs $POD -n $NS --all-containers --previous --tail=200 kubectl get pod $POD -n $NS -o yaml # full spec plus status 2. Decode why a container died
lastState.terminated holds the reason and exit code. Exit code 137 means SIGKILL, usually OOMKilled. Exit code 143 means SIGTERM. Exit code 1 means an application error.
kubectl get pod $POD -n $NS -o jsonpath='{range .status.containerStatuses[*]}{.name}{"\t"}{.restartCount}{"\t"}{.lastState.terminated.reason}{"\t"}{.lastState.terminated.exitCode}{"\n"}{end}' # Waiting reasons: CrashLoopBackOff, ImagePullBackOff, CreateContainerConfigError kubectl get pod $POD -n $NS -o jsonpath='{.status.containerStatuses[*].state.waiting}' # Pods with the most restarts, cluster-wide kubectl get pods -A --sort-by='.status.containerStatuses[0].restartCount' | tail -15 This is the free part. The full kubectl Troubleshooting & Daily Commands Cheat Sheet (production commands, best practices, common pitfalls and a debugging checklist on 2–4 dense pages) is available as a PDF, pay what you want: https://guranova.gumroad.com/l/wktiwb
More sheets: https://vladguran.github.io/dev-cheatsheets/
This article was written with AI assistance.
