sed — Footguns¶

Mistakes that silently corrupt files, produce wrong output, or waste hours debugging cross-platform differences.

1. sed -i differs on macOS vs Linux¶

On Linux (GNU sed), sed -i 's/old/new/' file works. On macOS (BSD sed), the same command fails with invalid command code. BSD sed requires an argument to -i for the backup extension — even if it is empty.

# Works on Linux, breaks on macOS
sed -i 's/old/new/g' config.txt
# macOS: sed: 1: "config.txt": invalid command code c

# Works on macOS, breaks on Linux (creates config.txt.bak backup)
sed -i '.bak' 's/old/new/g' config.txt

# Works on macOS (empty string = no backup), breaks on Linux
sed -i '' 's/old/new/g' config.txt
# Linux: sed: can't read : No such file or directory

# Portable fix: always use backup extension, then remove it
sed -i.bak 's/old/new/g' config.txt && rm config.txt.bak

# Better portable fix: use perl instead
perl -pi -e 's/old/new/g' config.txt
# Identical behavior on macOS, Linux, and BSDs

Rule: In scripts that must run on both macOS and Linux (CI, developer machines), use perl -pi -e or sed -i.bak ... && rm *.bak.

2. Not escaping / in sed (use a different delimiter)¶

When your search or replacement contains slashes, the command becomes unreadable and error-prone. One missing backslash and you get unknown command errors or wrong substitutions.

# Unreadable and fragile
sed -i 's/\/usr\/local\/bin\/python/\/usr\/bin\/python3/g' script.sh

# Fix: use a different delimiter — ANY character works after s
sed -i 's|/usr/local/bin/python|/usr/bin/python3|g' script.sh
sed -i 's#/usr/local/bin/python#/usr/bin/python3#g' script.sh
sed -i 's@/old/path@/new/path@g' config.txt

# The delimiter just needs to be consistent and not appear in the pattern
# Common choices: | # @ ! ,

3. sed BRE vs ERE (need -E for +, ?, etc.)¶

By default, sed uses Basic Regular Expressions (BRE). In BRE, +, ?, {n}, (), and | are literal characters — not regex metacharacters. You have to escape them with backslash OR use -E for Extended Regular Expressions.

# BUG: + is literal in BRE — matches nothing useful
echo "aabbb" | sed 's/ab+/X/'
# Output: aabbb  (no match — it's looking for literal "ab+")

# Fix: use -E for extended regex
echo "aabbb" | sed -E 's/ab+/X/'
# Output: aX

# BRE requires escaping for these metacharacters:
echo "abc" | sed 's/\(a\)\(b\)/\2\1/'  # BRE: escape parens
echo "abc" | sed -E 's/(a)(b)/\2\1/'    # ERE: clean syntax

# Common mistake: forgetting -E in find-and-replace scripts
# This does nothing (literal parentheses in pattern):
sed 's/(error|warning)/ALERT/g' log.txt
# This works:
sed -E 's/(error|warning)/ALERT/g' log.txt

# GNU sed accepts -E and -r (same thing). BSD sed only accepts -E.
# Always use -E for portability.

4. sed -i '' needed on macOS (BSD sed)¶

This is the flip side of footgun #1, specific enough to deserve its own entry. In Dockerfiles and CI scripts that run on macOS runners, this causes silent failures or cryptic errors.

# In a Makefile or CI script that runs on macOS:
sed -i 's/DEBUG=true/DEBUG=false/' .env
# Error: sed: 1: ".env": invalid command code .

# Developer "fixes" it with:
sed -i '' 's/DEBUG=true/DEBUG=false/' .env
# Works on macOS, but now it breaks on Linux CI runners:
# sed: can't read : No such file or directory

# The real fix for CI: detect the OS
if [[ "$OSTYPE" == "darwin"* ]]; then
    sed -i '' 's/DEBUG=true/DEBUG=false/' .env
else
    sed -i 's/DEBUG=true/DEBUG=false/' .env
fi

# Or just use perl (see footgun #1)
perl -pi -e 's/DEBUG=true/DEBUG=false/' .env

5. sed newlines in replacement text¶

Replacing text with a string that contains a newline is surprisingly hard in sed. The \n escape does not work in the replacement side of s/// in all implementations.

# Does NOT insert a newline (puts literal \n)
echo "hello" | sed 's/hello/line1\nline2/'
# GNU sed: line1\nline2  (some versions: line1<newline>line2)
# BSD sed: line1\nline2  (always literal)

# Fix for GNU sed: use a literal backslash-newline
echo "hello" | sed 's/hello/line1\
line2/'
# Output:
# line1
# line2

# Fix for any sed: use the a (append) or i (insert) commands instead
sed '/MARKER/a\new line after marker' file.txt

# Portable fix: use awk or perl
echo "hello" | awk '{ gsub(/hello/, "line1\nline2"); print }'
echo "hello" | perl -pe 's/hello/line1\nline2/'

6. Modifying a file you are reading from¶

Reading and writing the same file in a pipeline truncates it before the read completes. The file ends up empty.

# DESTROYS the file:
sed 's/old/new/g' config.txt > config.txt

# Fix: use sed -i (in-place editing, handled correctly by sed)
sed -i 's/old/new/g' config.txt

# Fix: use a temp file
sed 's/old/new/g' config.txt > config.tmp && mv config.tmp config.txt

# Fix: use sponge from moreutils (absorbs all input before writing)
sed 's/old/new/g' config.txt | sponge config.txt

Rule: Never redirect output to the same file you are reading in a single pipeline. Use -i, temp files, or sponge.

7. sed & in replacement is special¶

In the replacement part of s///, & represents the entire matched pattern. If you want a literal &, you must escape it. This catches people replacing URLs, company names, or any text containing ampersands.

# BUG: trying to insert a literal &
echo "company" | sed 's/company/Ben & Jerry/'
# Output: Ben company Jerry  (& was replaced with the matched text "company")

# Fix: escape the ampersand
echo "company" | sed 's/company/Ben \& Jerry/'
# Output: Ben & Jerry

# Same problem in URLs
echo "old-url" | sed 's|old-url|https://example.com?a=1&b=2|'
# Output: https://example.com?a=1old-urlb=2

echo "old-url" | sed 's|old-url|https://example.com?a=1\&b=2|'
# Output: https://example.com?a=1&b=2

8. BSD vs GNU differences beyond sed -i¶

Many scripts break when moving between Linux and macOS because of subtle GNU vs BSD behavior differences.

# GNU sed \b (word boundary) — not supported in BSD sed
echo "cat catalog" | sed 's/\bcat\b/dog/g'
# GNU: dog catalog
# BSD: no match or error

# Fix: stick to POSIX features in portable scripts
# - No \b, \w, \d in sed (use [[:alpha:]], [0-9], etc.)
# - Use -E not -r
# - Always use -i.bak for in-place editing

# Check what you're running
sed --version 2>/dev/null || echo "BSD sed (no --version)"