tar & Compression - Footguns

1. tar extracting over existing files with no warning. You extract an archive into a directory that already has files. tar silently overwrites every file that exists in both the archive and the directory. No prompt, no backup, no undo. If the archive contains an older version of a config file, your current configuration is gone.

Fix: Always extract into an empty or separate directory first, then compare:

mkdir /tmp/restore-check
tar xzf backup.tar.gz -C /tmp/restore-check/
diff -r /var/app/ /tmp/restore-check/var/app/

Only after verifying should you extract into the live directory. Or use --keep-old-files to skip existing files, or --keep-newer-files to keep whichever is newer.

1. Tar bomb — extracting into the current directory without a subdirectory. Someone creates an archive that does not have a top-level directory. When you extract it, hundreds of files scatter into your current directory, mixed with your existing files. Cleanup is painful.

# The tar bomb:
tar czf bomb.tar.gz *.py *.txt *.cfg  # No containing directory
# Victim extracts:
tar xzf bomb.tar.gz  # 200 files dump into cwd

Fix: Always list contents first: tar tf archive.tar.gz | head -20. If there is no common top-level directory, extract into a new directory:

mkdir /tmp/untarred && tar xzf archive.tar.gz -C /tmp/untarred/

When creating archives, always include a top-level directory:

tar czf myapp-v2.tar.gz myapp-v2/   # Good: everything under myapp-v2/

1. Absolute paths in tar archives. If you create a tar with absolute paths and then extract it on another system (or as root), the files go to the same absolute paths, potentially overwriting system files.

tar czf backup.tar.gz /etc/nginx/   # Stores as /etc/nginx/...
tar xzf backup.tar.gz               # Extracts to /etc/nginx/ — overwrites!

Fix: GNU tar strips the leading / by default and warns you. Do not suppress this warning. If you need to extract to a different path:

tar xzf backup.tar.gz --strip-components=1 -C /tmp/nginx-backup/

When creating, use -C to store relative paths:

tar czf backup.tar.gz -C /etc nginx/   # Stores as nginx/...

1. Compression slowing down transfers on fast networks. You add -z (gzip) to every tar command out of habit. On a 10Gbps network or on local disk-to-disk copies, the CPU time for compression exceeds the time saved by smaller data size. Your 2-minute transfer becomes 10 minutes because a single gzip core is the bottleneck.

Fix: Skip compression for local transfers and fast networks. Use compression only when bandwidth is the bottleneck (WAN, slow links):

# Fast network: no compression
tar cf - /var/data/ | ssh remote 'tar xf - -C /dest/'

# Slow network: compress
tar czf - /var/data/ | ssh remote 'tar xzf - -C /dest/'

# Or use parallel compression to reduce CPU bottleneck
tar cf - /var/data/ | pigz | ssh remote 'pigz -d | tar xf - -C /dest/'

1. gzip destroying the original file. gzip file.log compresses the file and deletes the original. If the compression fails mid-way (disk full, interrupted), you may lose both the original and the compressed version. Most people expect compression tools to keep the original by default. gzip does not.

Fix: Use -k to keep the original:

gzip -k file.log    # Keeps file.log, creates file.log.gz

Or use zstd, which keeps the original by default.

Default trap: gzip, bzip2, and xz all delete the original after compression by default. zstd is the exception — it keeps the original. This inconsistency catches people who switch between tools. The -k (keep) flag works on all four, so consider aliasing compression commands with -k in shared environments.

For critical files, always verify the compressed output before removing the original:

gzip -k file.log && gzip -t file.log.gz && rm file.log

1. xz memory usage during compression. xz at its default compression level uses about 100MB of memory. At -9, it uses nearly 700MB. On a shared server with limited memory, running xz on a large file can trigger the OOM killer, killing other processes.

Fix: Limit memory explicitly:

xz --memlimit=256MiB largefile.tar

Or use zstd, which achieves similar compression ratios with much lower memory usage. Monitor memory during compression if you are unsure:

xz largefile.tar &
watch -n1 "ps -p $! -o pid,rss,vsz,comm"

1. Extracting as root preserves ownership — security risk. When root extracts a tar archive, file ownership is preserved from the archive metadata. If someone crafted an archive with files owned by uid 0 (root) and setuid bits, extracting as root creates setuid-root binaries on your system.

Fix: Use --no-same-owner when extracting as root:

tar xzf untrusted-archive.tar.gz --no-same-owner -C /tmp/inspect/

Better: never extract untrusted archives as root. Extract as an unprivileged user, inspect, then move files where needed. Also consider --no-same-permissions to strip special bits.

1. Forgetting -f makes tar read from stdin. If you forget the -f flag, tar waits for input on stdin. Your terminal hangs with no output, no error, no prompt. You sit there wondering what happened.

tar czf /var/data/      # WRONG: creates archive from stdin, "f" takes /var/data as filename? No...
tar cz /var/data/       # WRONG: no -f, tar writes to stdout (binary garbage on terminal)

Fix: Always use -f explicitly:

tar czf archive.tar.gz /var/data/   # Correct

If tar appears to hang, press Ctrl+C and check your flags. If binary garbage appears on your terminal, run reset to fix the terminal state.

1. tar and sparse files. Sparse files (files with holes — common for VM disk images, database files) are stored at their full virtual size by default. A 100GB VM image that is 90% empty becomes a 100GB tar entry.

Fix: Use --sparse (or -S) to detect and handle sparse files:

tar czSf vm-backup.tar.gz /var/lib/libvirt/images/

This stores only the allocated blocks, dramatically reducing archive size for sparse files.

1. Running out of disk during extraction. You start extracting a large archive. Midway through, the disk fills up. tar exits with an error. You now have a partially extracted directory — some files are complete, some are truncated, some are missing. The application starts up with this inconsistent state and does something unpredictable.

   Fix: Check available space before extracting:

   # Estimate uncompressed size
   NEEDED=$(tar tzvf backup.tar.gz | awk '{sum += $3} END {print sum}')
   AVAILABLE=$(df --output=avail -B1 /var/restore/ | tail -1)
   if [ "$NEEDED" -gt "$AVAILABLE" ]; then
       echo "Not enough space: need $(numfmt --to=iec $NEEDED), have $(numfmt --to=iec $AVAILABLE)"
       exit 1
   fi
   

   Always extract to a temporary directory and move atomically on success.

2. Different tar implementations (GNU vs BSD). macOS uses BSD tar. Linux uses GNU tar. They have different flags, different defaults, and different behaviors. Scripts written on Linux break on macOS and vice versa.

   Common differences: - BSD tar does not support --wildcards - BSD tar uses -s for pattern substitution (GNU uses --transform) - --exclude syntax differs slightly - GNU tar has --listed-incremental; BSD tar does not

   Fix: For portable scripts, stick to POSIX-standard flags (-c, -x, -t, -f, -z). Test on both platforms. Or install GNU tar on macOS: brew install gnu-tar (provides gtar).

Pages that link here

• Anti-Primer: Tar And Compression
• tar & Compression