Day 49advancedMar 21, 2026

Debug Running Pods with Ephemeral Containers

When your container has no shell, ephemeral containers let you debug it without restarting.

kubernetesdebuggingtroubleshooting

What

Ephemeral containers let you attach a temporary debugging container to a running pod without restarting it or modifying its spec. This is essential for debugging distroless or minimal container images that lack basic tools like a shell, curl, or strace. The debug container shares the pod's process namespace so you can inspect the running application.

Why It Matters

Production containers should be minimal — no shell, no package manager, no extra tools. But when something goes wrong, you need to inspect processes, network connections, and file systems. Ephemeral containers give you full debugging access on demand without rebuilding images or restarting the pod, so you can troubleshoot issues in their live state.

Example

# Attach a debug container to a running pod
kubectl debug -it pod/my-app --image=busybox:1.36 --target=my-app

# Use a more powerful debug image with networking tools
kubectl debug -it pod/my-app --image=nicolaka/netshoot --target=my-app

# Debug with a copy of the pod (non-destructive)
kubectl debug pod/my-app -it --copy-to=debug-pod --container=debug \
  --image=ubuntu:22.04

# Inside the ephemeral container, inspect the app process
ps aux
netstat -tlnp
cat /proc/1/environ

yaml

Common Mistake

Trying to kubectl exec into distroless containers that have no shell. You'll get an error like 'OCI runtime exec failed: exec failed: unable to start container process: exec: sh: executable file not found in $PATH'.

Quick Fix

Use `kubectl debug -it pod/<name> --image=busybox --target=<container>` instead of exec. The --target flag lets you share the process namespace of the original container so you can see its processes and inspect its filesystem via /proc/<pid>/root.