Skip to content

Portal | Level: L1: Foundations | Topics: Cron & Job Scheduling, Bash / Shell Scripting, systemd | Domain: Linux

Cron & Job Scheduling - Primer

Why This Matters

Scheduled jobs are the backbone of automation. Backups, log rotation, certificate renewal, report generation, database maintenance, cache warming — all cron jobs. They run in the background, silently keeping your infrastructure alive. When they break, things degrade slowly until someone notices the backups have not run in two weeks.

The problem with scheduled jobs is that they fail silently. Nobody watches them. The environment they run in is not what you expect. And when two instances overlap because the first one has not finished, you get corrupted data or deadlocks.

This primer covers classic cron, systemd timers (the modern replacement), and Kubernetes CronJobs — because your jobs need to work everywhere.

Classic Cron

Cron Syntax: The Five Fields

┌───────────── minute (0-59)
│ ┌───────────── hour (0-23)
│ │ ┌───────────── day of month (1-31)
│ │ │ ┌───────────── month (1-12 or JAN-DEC)
│ │ │ │ ┌───────────── day of week (0-7 or SUN-SAT, 0=7=Sunday)
│ │ │ │ │
* * * * *  command

Remember: Cron field order mnemonic: Minute, Hour, Day of month, Month, Day of week — "My Hard Drive Might Die." The first field is always the smallest time unit (minute), increasing left to right.

Common Schedules

# Every minute
* * * * *  /usr/local/bin/health-check.sh

# Every 5 minutes
*/5 * * * *  /usr/local/bin/poll.sh

# Every hour at minute 0
0 * * * *  /usr/local/bin/hourly-job.sh

# Every day at 2:30 AM
30 2 * * *  /usr/local/bin/backup.sh

# Every Monday at 9 AM
0 9 * * 1  /usr/local/bin/weekly-report.sh

# First day of every month at midnight
0 0 1 * *  /usr/local/bin/monthly-cleanup.sh

# Every weekday at 6 PM
0 18 * * 1-5  /usr/local/bin/eod-report.sh

# Every 15 minutes between 8 AM and 6 PM
*/15 8-18 * * *  /usr/local/bin/business-hours-check.sh

# Twice a day (6 AM and 6 PM)
0 6,18 * * *  /usr/local/bin/sync.sh

Special Strings

@reboot    Run once at startup
@hourly    0 * * * *
@daily     0 0 * * *
@weekly    0 0 * * 0
@monthly   0 0 1 * *
@yearly    0 0 1 1 *

Managing Crontabs

User Crontabs

# Edit current user's crontab
crontab -e

# List current user's crontab
crontab -l

# Edit another user's crontab (as root)
crontab -e -u deploy

# Remove current user's crontab (DANGEROUS — no confirmation)
crontab -r

# Remove with confirmation (install this habit)
crontab -ri

User crontabs are stored in /var/spool/cron/crontabs/ (Debian) or /var/spool/cron/ (RHEL). Do not edit these files directly.

System Crontabs

# System crontab (has an extra USER field)
cat /etc/crontab

# Drop-in directory (one file per job)
ls /etc/cron.d/

# Periodic directories (scripts, not cron syntax)
ls /etc/cron.hourly/
ls /etc/cron.daily/
ls /etc/cron.weekly/
ls /etc/cron.monthly/

/etc/cron.d/ Format

# /etc/cron.d/backup
# Note: includes username field (6th field)
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAILTO=ops@example.com

30 2 * * * root /usr/local/bin/backup.sh >> /var/log/backup.log 2>&1

User vs System Crontabs

User crontab (crontab -e):
  - No user field (runs as the editing user)
  - 5 fields + command
  - Survives user profile changes
  - Cannot set SHELL in the schedule line

System crontab (/etc/cron.d/):
  - Has user field (6th field)
  - 5 fields + user + command
  - Managed by config management (Ansible, etc.)
  - Can set environment variables at top

The Cron Environment Trap

Gotcha: Cron uses /bin/sh by default, not /bin/bash. Bashisms like [[ ]], $(( )), and process substitution <() will silently fail. Either set SHELL=/bin/bash at the top of your crontab or write POSIX-compatible scripts.

This is the number-one source of "it works when I run it manually but fails in cron":

Your shell:
  PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/go/bin:/home/user/.local/bin
  HOME=/home/user
  SHELL=/bin/bash
  All your .bashrc exports are loaded

Cron's environment:
  PATH=/usr/bin:/bin
  HOME=/root (or the user's home)
  SHELL=/bin/sh
  NO .bashrc, NO .profile, NO exports

Fixes

# Option 1: Use absolute paths
* * * * * /usr/local/bin/python3 /opt/app/script.py

# Option 2: Set PATH in the crontab
PATH=/usr/local/bin:/usr/bin:/bin
* * * * * python3 /opt/app/script.py

# Option 3: Source your profile in the command
* * * * * . /home/user/.profile && /opt/app/script.sh

# Option 4: Use a wrapper script that sets up the environment
* * * * * /opt/app/run-in-env.sh

Systemd Timers

Systemd timers are the modern replacement for cron. They offer better logging, dependency management, and resource control.

Timer + Service Pair

Every timer needs a corresponding service unit:

# /etc/systemd/system/backup.timer
[Unit]
Description=Daily backup timer

[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true
RandomizedDelaySec=300

[Install]
WantedBy=timers.target

# /etc/systemd/system/backup.service
[Unit]
Description=Daily backup job
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh
User=backup
Group=backup
StandardOutput=journal
StandardError=journal

OnCalendar Syntax

OnCalendar=*-*-* 02:30:00         # daily at 2:30 AM
OnCalendar=Mon *-*-* 09:00:00     # every Monday at 9 AM
OnCalendar=*-*-01 00:00:00        # first of every month
OnCalendar=*-*-* *:00/15:00       # every 15 minutes
OnCalendar=hourly                  # shorthand
OnCalendar=daily                   # shorthand
OnCalendar=weekly                  # shorthand

# Validate your OnCalendar expression
systemd-analyze calendar "*-*-* 02:30:00"
# Shows next trigger times — extremely useful for debugging

# Test complex expressions
systemd-analyze calendar "Mon..Fri *-*-* 08:00:00"

Key Timer Options

Persistent=true      # if the system was off at trigger time, run on boot
RandomizedDelaySec=  # add random delay to spread load across machines
AccuracySec=         # how precise the trigger needs to be (default 1min)

Managing Timers

# Enable and start a timer
systemctl enable --now backup.timer

# List all timers with next trigger time
systemctl list-timers --all

# Check timer status
systemctl status backup.timer

# Check the service status (last run)
systemctl status backup.service

# View logs for the service
journalctl -u backup.service --since "24 hours ago"

# Manually trigger the service (for testing)
systemctl start backup.service

Advantages Over Cron

Feature              cron                    systemd timer
─────────────────    ────────────────────    ──────────────────────
Logging              mail or redirect        journald (structured)
Dependencies         none                    After=, Requires=
Resource limits      none                    CPUQuota=, MemoryMax=
Missed runs          lost                    Persistent=true
Random delay         none                    RandomizedDelaySec=
Status               cron log parsing        systemctl status
Overlap prevention   manual (flock)          built-in (oneshot)
Calendar validation  none                    systemd-analyze calendar

Kubernetes CronJobs

CronJob Spec

apiVersion: batch/v1
kind: CronJob
metadata:
  name: database-backup
  namespace: production
spec:
  schedule: "30 2 * * *"          # same cron syntax
  timeZone: "America/New_York"    # K8s 1.27+
  concurrencyPolicy: Forbid       # do not overlap
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 5
  startingDeadlineSeconds: 600    # give up if 10 min late
  jobTemplate:
    spec:
      backoffLimit: 2             # retry failed job 2 times
      activeDeadlineSeconds: 3600 # kill job after 1 hour
      template:
        spec:
          restartPolicy: OnFailure
          containers:
          - name: backup
            image: backup-tool:latest
            command: ["/bin/sh", "-c", "/backup.sh"]
            env:
            - name: DB_HOST
              valueFrom:
                secretKeyRef:
                  name: db-credentials
                  key: host
            resources:
              requests:
                memory: "256Mi"
                cpu: "100m"
              limits:
                memory: "512Mi"
                cpu: "500m"

Concurrency Policy

Allow     default, multiple jobs can run simultaneously
Forbid    skip new job if previous is still running
Replace   kill the running job and start a new one

Default trap: The default Kubernetes CronJob concurrencyPolicy is Allow, meaning if your job takes 10 minutes and runs every 5 minutes, you will have overlapping instances. Always explicitly set concurrencyPolicy: Forbid for stateful jobs like backups and database maintenance.

Forbid is almost always what you want for data-integrity-sensitive jobs. Allow is appropriate for idempotent, independent tasks.

startingDeadlineSeconds

If the CronJob controller misses a scheduled run (e.g., controller was down), this setting controls how late the job can start. Without it, missed runs are silently skipped.

Overlap Prevention

The most common scheduling disaster: a job that runs longer than its interval.

With cron: Use flock

# /etc/cron.d/long-job
*/5 * * * * root flock -n /var/lock/long-job.lock /usr/local/bin/long-job.sh

# flock -n: non-blocking. If lock exists, exit immediately.
# flock -w 60: wait up to 60 seconds for lock.

With systemd: Built-in

# oneshot services do not overlap by default
[Service]
Type=oneshot
ExecStart=/usr/local/bin/long-job.sh

If the timer fires while the previous run is still going, systemd skips the trigger.

With Kubernetes: concurrencyPolicy

spec:
  concurrencyPolicy: Forbid

Output and Notifications

Cron: Capturing Output

# Redirect stdout and stderr to a log file
30 2 * * * /usr/local/bin/backup.sh >> /var/log/backup.log 2>&1

# Send stdout to log, stderr to email (default MAILTO)
30 2 * * * /usr/local/bin/backup.sh >> /var/log/backup.log

# Set MAILTO for email notifications
MAILTO=ops@example.com
30 2 * * * /usr/local/bin/backup.sh 2>&1

# Suppress all output (use sparingly)
30 2 * * * /usr/local/bin/backup.sh > /dev/null 2>&1

Systemd: journald

# Logs go to journal automatically
journalctl -u backup.service -n 50

# Follow logs in real-time during a run
journalctl -u backup.service -f

Quick Reference

Task                              Tool
─────────────────────────────     ──────────────────────────────
Simple scheduled command          cron (crontab -e)
Managed by config management      /etc/cron.d/ files
Needs logging and dependencies    systemd timer
Needs resource limits             systemd timer
Runs in Kubernetes                CronJob
Needs overlap prevention          flock (cron) / Forbid (K8s)
One-time future job               at
Run when system is idle           batch

Fun fact: The name "cron" comes from the Greek word "chronos" (time). The cron daemon was written by Ken Thompson for Version 7 Unix in 1979. The modern Vixie Cron (written by Paul Vixie in 1987) is the version most Linux distributions use, and it introduced the @reboot and @hourly shorthand syntax.

Scheduled jobs are only as reliable as your monitoring of them. If nobody checks whether the backup ran, the backup did not run. Monitor your cron jobs like you monitor your services.

Wiki Navigation

Prerequisites

Related Content