sed — Street-Level Ops¶

Real-world sed workflows for config management, mass replacements, and text transformation in production.

Mass Find-and-Replace Across a Codebase¶

# Replace a string across all Python files
find . -name '*.py' -exec sed -i 's/old_function/new_function/g' {} +

# Same thing, but preview first (no -i)
find . -name '*.py' -exec grep -l 'old_function' {} + | head -20

# Replace in specific file types, excluding vendor dirs
find . -name '*.go' -not -path './vendor/*' -exec sed -i 's/oldpkg/newpkg/g' {} +

# Portable macOS/Linux replace (uses perl instead of sed)
find . -name '*.yaml' -exec perl -pi -e 's/image: v1\.2/image: v1.3/g' {} +

> **Gotcha:** macOS `sed -i` requires an explicit backup suffix (`sed -i '' 's/...'`), while GNU sed allows bare `sed -i 's/...'`. Scripts that work on Linux will fail on macOS with "extra characters after command." Use `perl -pi -e` for portable in-place edits, or always test with `--version` first.

# Replace with a different delimiter (when the replacement contains /)
find . -name '*.conf' -exec sed -i 's|/old/path|/new/path|g' {} +

> **Remember:** sed's `s` command accepts any delimiter character, not just `/`. Use `|`, `#`, or `@` when your pattern contains slashes. `s|/old/path|/new/path|g` is far more readable than `s/\/old\/path/\/new\/path/g`.

# Multi-line config block replacement with sed
sed -i '/\[old-section\]/,/^\[/{/^\[old-section\]/!{/^\[/!d}}' config.ini

Rewriting Config Files¶

# Change a specific setting in a key=value config
sed -i 's/^max_connections=.*/max_connections=200/' postgresql.conf

# Toggle a boolean setting
sed -i 's/^debug_mode=true/debug_mode=false/' app.conf

# Add a setting after a specific line (if it does not already exist)
grep -q '^log_level=' app.conf || sed -i '/^\[logging\]/a log_level=info' app.conf

# Comment out a line
sed -i 's/^dangerous_setting/#&/' config.txt

# Uncomment a line
sed -i 's/^#\(dangerous_setting\)/\1/' config.txt

# Replace a value in an INI section (only within that section)
sed -i '/^\[database\]/,/^\[/ s/^host=.*/host=db-prod.internal/' config.ini

# Update YAML value (simple key: value on its own line)
sed -i 's/^  replicas: .*/  replicas: 5/' deployment.yaml

# Append to end of file if line does not exist
grep -q '^nameserver 10.0.0.2' /etc/resolv.conf || \
  echo 'nameserver 10.0.0.2' >> /etc/resolv.conf

In-Place Config Modification in Docker Builds¶

# Dockerfile: modify config files during build
RUN sed -i 's/^#\(listen_addresses\)/\1/' /etc/postgresql/postgresql.conf && \
    sed -i "s/^max_connections = .*/max_connections = 200/" /etc/postgresql/postgresql.conf && \
    sed -i 's/^#\(log_destination\)/\1/' /etc/postgresql/postgresql.conf

# Dockerfile: enable Apache modules
RUN sed -i 's/^#\(LoadModule rewrite_module\)/\1/' /etc/httpd/conf/httpd.conf

# Dockerfile: set timezone
RUN sed -i "s|^;date.timezone =.*|date.timezone = UTC|" /etc/php/php.ini

# Dockerfile: disable SSH root login in hardened images
RUN sed -i 's/^PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config && \
    sed -i 's/^#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config

CSV/TSV Format Conversion¶

# Convert CSV to TSV
sed 's/,/\t/g' data.csv

# Convert TSV to CSV (simple — does not handle embedded tabs)
sed 's/\t/,/g' data.tsv

Log File Cleanup and Extraction¶

# Strip ANSI color codes from log output
sed -E 's/\x1b\[[0-9;]*m//g' colored.log

# Extract timestamps from structured logs
sed -n 's/.*\(20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9:]*\).*/\1/p' app.log

# Remove blank lines and comment lines from a config
sed '/^$/d; /^#/d' config.txt

# Extract all URLs from a file
sed -nE 's/.*(https?:\/\/[^ "]+).*/\1/p' page.html

# Truncate log lines to 200 characters
sed -E 's/(.{200}).*/\1.../' verbose.log

Systemd Unit and Service File Edits¶

# Change ExecStart in a systemd unit override
sed -i 's|^ExecStart=.*|ExecStart=/usr/bin/myapp --config /etc/myapp.conf|' \
  /etc/systemd/system/myapp.service

# Disable a timer by commenting it out
sed -i 's/^OnCalendar=/#&/' /etc/systemd/system/cleanup.timer

# Adjust resource limits in a unit file
sed -i 's/^LimitNOFILE=.*/LimitNOFILE=65536/' /etc/systemd/system/nginx.service

Combining sed with Other Tools¶

# Extract failed systemd units (sed strips .service suffix)
systemctl --failed --no-legend | awk '{ print $1 }' | sed 's/\.service$//'

# Sanitize filenames (remove special chars)
ls *.log | sed 's/[^a-zA-Z0-9._-]/_/g'

# Generate sed commands from a mapping file
awk -F'\t' '{ printf "s/%s/%s/g\n", $1, $2 }' mappings.tsv > bulk_replace.sed
sed -f bulk_replace.sed input.txt

Power One-Liners¶

Recursive search and replace across files¶

grep -rl 'old_string' . | xargs sed -i 's/old_string/new_string/g'

Breakdown: grep -rl finds files containing the pattern (recursive, files-only). xargs feeds them to sed -i for in-place replacement. Add --include='*.py' to grep for file type filtering.

Default trap: sed -i with no preview is destructive and not undoable. Always run the command without -i first to see what would change, or use sed -i.bak to keep backup files. In git repos, git diff after the edit is your safety net.

[!TIP] When to use: Renaming variables, updating URLs, changing config values across a codebase.

Delete a specific line by number¶

sed -i '8d' ~/.ssh/known_hosts

[!TIP] When to use: Removing a stale SSH host key after server rebuild. Way faster than ssh-keygen -R.

Strip ANSI color codes from output¶

sed 's/\x1b\[[0-9;]*m//g'

Breakdown: Matches ESC (\x1b) followed by [, digits/semicolons, and m (the SGR escape sequence format). Strips all color/formatting codes.

[!TIP] When to use: Cleaning command output before logging, piping colored output to files, processing CI logs.

Print lines between two patterns (like awk range)¶

sed -n '/START/,/END/p' file.txt

[!TIP] When to use: Extracting config blocks, log windows, or function bodies.

Create a visual directory tree¶

ls -R | grep ':$' | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/  /' -e 's/-/|/'

Breakdown: ls -R lists recursively (dirs end with :). Series of sed substitutions: strip colon, replace path components with --, indent, and add pipe character for tree-like visualization.

[!TIP] When to use: Quick directory overview when tree isn't installed.

Reverse line order (tac in pure sed)¶

sed '1!G;h;$!d' file.txt

Breakdown: This is famously cryptic. For each line except the first (1!), append hold space to pattern space (G). Then copy pattern space to hold space (h). For all lines except the last ($!), delete pattern space (d). Effect: builds the file in reverse in the hold space, prints it all at the end. The hold space is sed's only "memory" — this one-liner abuses it brilliantly.

[!TIP] When to use: tac isn't available (macOS, minimal containers). Also a great exercise for understanding sed's two-buffer model (pattern space vs hold space).

Join backslash-continued lines¶

sed -e :a -e '/\\$/N; s/\\\n//; ta' file.txt

Breakdown: Creates a label a. If the line ends with \, append the Next line (N), remove the \ + newline, and branch back to a (loop). Handles multi-level continuations (3+ lines joined). This is sed doing a while-loop with labels and branches.

[!TIP] When to use: Preprocessing Makefiles, nginx configs, or shell scripts before grepping or further processing.

Add commas to numbers (1234567 to 1,234,567)¶

sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta' file.txt
# or with GNU sed:
gsed ':a;s/\B[0-9]\{3\}\>/,&/;ta'

Breakdown: Labels and branches again — sed loops inserting commas from right to left. Each pass places one comma before the rightmost group of 3 digits that doesn't already have one. Loops until no more substitutions are made (ta = branch if substitution succeeded).

[!TIP] When to use: Formatting byte counts, request counts, or financial data for human-readable output in reports.

Print paragraph containing a pattern¶

sed -e '/./{H;$!d;}' -e 'x;/ERROR/!d;' file.txt

Breakdown: Accumulates non-blank lines into hold space (H). On blank lines (paragraph breaks) or EOF, swaps hold to pattern (x) and tests for the pattern. If found, prints the whole paragraph; if not, deletes. This is sed doing paragraph-mode grep.

[!TIP] When to use: Extracting full config blocks, log paragraphs, or multi-line records that contain a keyword. More powerful than grep -A/-B because it respects paragraph boundaries.

Print line before a regex match (context)¶

sed -n '/regex/{g;1!p;};h' file.txt

Breakdown: Every line is copied to hold space (h). When the regex matches, we swap in the hold space (g) — which contains the previous line — and print it (1!p skips if it's line 1, since there's no previous line). This is the sed equivalent of grep -B1.

[!TIP] When to use: Finding what happened right before an error, getting the config key above a value.

Remove most HTML tags¶

sed -e :a -e 's/<[^>]*>//g;/</N;//ba' file.html

Breakdown: Removes all <...> tags. If a < remains (tag split across lines), appends the next line (N) and loops. The // reuses the last regex. Handles multi-line tags that simpler s/<[^>]*>//g would miss.

[!TIP] When to use: Quick-and-dirty HTML-to-text conversion, stripping tags from curl output, cleaning up scraped data.

Caveat: Not a real HTML parser — breaks on <script> content, nested tags, etc. Good enough for simple cases.

Delete duplicate consecutive lines (uniq in pure sed)¶

sed '$!N; /^\(.*\)\n\1$/!P; D' file.txt

Breakdown: Reads pairs of lines (N). If they're identical (\1 backreference matches), only the D fires (deletes first line of pair, loops). If different, P prints the first line, then D loops with the second line becoming the new first. This is uniq without sorting, in pure sed.

[!TIP] When to use: Deduplicating sorted output when uniq isn't available, or embedded in a sed script where you can't pipe to external tools.