Kubernetes CronJob not running: what the controller skipped, and why

Check these first: the CronJob's own events

The controller records why it did not start a Job. Read that before anything else.

# SUSPEND, ACTIVE and LAST SCHEDULE at a glance
kubectl get cronjob nightly-backup

# The events at the bottom name the reason
kubectl describe cronjob nightly-backup

# Start one run now, outside the schedule
kubectl create job nightly-backup-manual --from=cronjob/nightly-backup

The event names worth knowing: JobAlreadyActive (Forbid is holding the run back), MissSchedule (the starting deadline passed), TooManyMissedTimes, UnknownTimeZone and FailedCreate. Events expire after a while, and by default only the last 3 successful and 1 failed Jobs are kept (successfulJobsHistoryLimit and failedJobsHistoryLimit), so the evidence of an old run may already be gone.

Seven reasons a CronJob does not run

In the order worth checking, quoting the Kubernetes documentation where it is exact.

kubectl patch cronjob nightly-backup -p '{"spec":{"suspend":false}}'

kubectl get jobs
kubectl get pods --selector=batch.kubernetes.io/job-name=<job-name>
kubectl describe pod <pod-name>

Get alerted when a CronJob does not run

Nothing inside the cluster reports a Job that was never created. A ping from the job command, checked against the schedule outside the cluster, reports the silence. No operator, nothing installed.

spec:
  schedule: "0 3 * * *"
  timeZone: "Etc/UTC"
  concurrencyPolicy: Forbid
  startingDeadlineSeconds: 600
  jobTemplate:
    spec:
      activeDeadlineSeconds: 3600
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: backup
              image: your-registry/backup:latest
              env:
                - name: LASTPING_URL
                  valueFrom:
                    secretKeyRef:
                      name: lastping
                      key: ping-url
              command:
                - /bin/sh
                - -c
                - |
                  curl -fsS -m 10 --retry 3 -o /dev/null "$LASTPING_URL/start"
                  if /app/backup.sh; then
                    curl -fsS -m 10 --retry 3 -o /dev/null "$LASTPING_URL"
                  else
                    curl -fsS -m 10 --retry 3 -o /dev/null "$LASTPING_URL/fail"
                    exit 1
                  fi

What this catches

The job reports what it can; the missing ping reports the rest.

• ✓No Job was created: suspended, held back by Forbid, past its starting deadline, over 100 missed schedules or scheduled in the wrong zone. No ping in the window opens an incident.
• ✓The pod never ran: an image that will not pull or a pod stuck Pending sends no ping either, so the same silence raises the alert.
• ✓The run hung: the start ping arrived and the finish did not, so overrun detection opens an incident while it is still running.
• ✓The workload failed: the /fail ping alerts at once, and the non-zero exit still lets Kubernetes retry.
• ✓Recovery: the next successful run closes the incident and sends a recovery notice.

Kubernetes says plainly that a CronJob creates a Job "approximately once per execution time of its schedule", and that in some circumstances two Jobs might be created, or none, so make the job idempotent. The monitor tells you about the "none".

Questions people ask

Front-loaded answers: the most important fact first.

• Why is my Kubernetes CronJob not running?

  Run kubectl describe cronjob and read the events at the bottom. The usual reasons: suspend is true; concurrencyPolicy is Forbid and an earlier Job is still running; the run missed its startingDeadlineSeconds; the controller counted more than 100 missed schedules; or the schedule is read in another timezone than you expect. If a Job was created but nothing ran, look at the Job's pods instead.

• What does too many missed start times mean?

  The CronJob controller counts the schedules it missed since the last scheduled time. The Kubernetes documentation says that with more than 100 it does not start the Job and logs too many missed start times. Set or decrease .spec.startingDeadlineSeconds or check clock skew. With startingDeadlineSeconds set, it counts only the misses inside that window, so setting it lets the Job start again.

• What does startingDeadlineSeconds do?

  It is how many seconds after its scheduled time a Job may still be started. Past it, the controller skips that run and Kubernetes treats it as a failed Job; later runs are still scheduled. Without it, runs have no deadline. Keep it at 10 seconds or more, because the controller checks every 10 seconds.

• Why does concurrencyPolicy Forbid stop my CronJob?

  With Forbid, a new run is skipped while the previous Job is still running, and the skipped run counts as missed. One hung Job therefore blocks every run after it. Set activeDeadlineSeconds on the Job so a stuck run is terminated, and monitor the runs so the first skip alerts you.

• What happens when I unsuspend a CronJob?

  Runs whose scheduled time fell while it was suspended count as missed. If the CronJob has no startingDeadlineSeconds, the missed Jobs are scheduled immediately when suspend changes from true to false. Set a starting deadline first if catch-up runs would do harm.

• What timezone does a Kubernetes CronJob use?

  Without .spec.timeZone, the kube-controller-manager reads the schedule in its own local time zone. Set .spec.timeZone to an IANA name such as Etc/UTC; the field is stable since Kubernetes v1.27, and TZ or CRON_TZ inside the schedule is rejected. The cron expression explainer lists the next runs in any zone.

• How do I get alerted when a CronJob does not run?

  Nothing inside the cluster reports a Job that was never created. Add a ping to the job command and give a LastPing monitor the same schedule, timezone and a grace window: a run that never starts sends no ping and opens an incident, and a /fail ping alerts at once.

• Is this monitoring free?

  Yes. LastPing is free for individuals with no monitor cap. See how it compares to Healthchecks.io and Cronitor.