Linux Hardening: Closing the Doors
- lesson
- ssh-hardening
- selinux
- apparmor
- sysctl-tuning
- filesystem-hardening
- auditd
- user-management
- cis-benchmarks
- file-integrity-monitoring ---# Linux Hardening — Closing the Doors
Topics: SSH hardening, firewalls (iptables/nftables/ufw/firewalld), SELinux, AppArmor, sysctl tuning, filesystem hardening, auditd, user management, CIS benchmarks, file integrity monitoring Level: L1–L2 (Foundations to Operations) Time: 75–100 minutes Strategy: Build-up + adversarial Prerequisites: None (everything explained from scratch)
The Mission¶
You just got handed a freshly installed Ubuntu 22.04 server. Tomorrow morning it goes into production — a payment processing app behind an Nginx reverse proxy. Your security team says it needs to pass a CIS benchmark scan before it touches customer data.
Right now this server has one user (ubuntu), password authentication enabled, no firewall,
no MAC policy, and every default still in place. It is connected to the internet. The clock
is ticking.
By the end of this lesson you will have: - Locked down SSH so brute-force attacks bounce off - Built a firewall that allows only what's needed - Enabled mandatory access control (AppArmor or SELinux) - Tuned the kernel to resist common network attacks - Set up audit logging that catches intruders - Hardened user accounts, filesystems, and mount points - Understood what CIS benchmarks actually check and why
We will build the hardened server layer by layer — and then think like an attacker to test each layer.
Part 1: SSH — The Front Door Everyone Tries First¶
Before anything else, lock the front door. SSH is the #1 target on any internet-facing Linux box. Automated scanners hit port 22 within minutes of a server going live.
War Story: In 2019, a cloud consultancy spun up an EC2 instance for a client demo. Default
sshd_config: root login enabled, password authentication on, port 22. They planned to harden it "after lunch." The access logs told a different story. Within 14 minutes, brute-force bots from three different countries had attempted over 800 root password combinations. By minute 40, one succeeded — the defaultubuntuuser had a weak password set during provisioning. The attacker installed a cryptocurrency miner, pivoted to the instance metadata service (169.254.169.254), and extracted IAM temporary credentials. The client's S3 buckets were exfiltrated before anyone noticed the CPU spike. Total time from boot to breach: 53 minutes.
That story has three lessons: defaults are dangerous, attackers are fast, and SSH hardening is not optional.
Create a user, set up keys, harden the config¶
# Step 1: non-root user with sudo
adduser deploy && usermod -aG sudo deploy
# Step 2: key-based auth (run on YOUR machine)
ssh-keygen -t ed25519 -C "deploy@payment-prod-01"
ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@payment-prod-01
ssh deploy@payment-prod-01 # test: should work without a password
Name Origin: Ed25519 is named after the Edwards curve over the prime field 2^255 - 19. Designed by Daniel J. Bernstein in 2011, it's faster and more resistant to side-channel attacks than RSA or ECDSA. The keys are also much shorter — 256 bits vs RSA's 4096.
Step 3 — harden sshd_config:
# Back up before editing (always)
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
sudo tee /etc/ssh/sshd_config.d/hardening.conf << 'EOF'
# Authentication
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no
MaxAuthTries 3
AuthenticationMethods publickey
# Session
X11Forwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
LoginGraceTime 30
# Access control
AllowUsers deploy
# Logging
LogLevel VERBOSE
# Ciphers (drop weak ones)
Ciphers aes256-gcm@openssh.com,aes128-gcm@openssh.com,chacha20-poly1305@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com
KexAlgorithms curve25519-sha256,diffie-hellman-group16-sha512
EOF
Key directives: PermitRootLogin no forces sudo audit trails. PasswordAuthentication no
eliminates brute-force entirely. AllowUsers deploy is a whitelist — everything not listed
is denied. ClientAliveInterval 300 kills abandoned sessions after 10 minutes of silence.
Validate and apply:
# ALWAYS validate before restarting — a broken config locks you out
sudo sshd -t
# If no output, the config is valid
# Show effective configuration
sudo sshd -T | grep -E 'permitroot|passwordauth|allowusers'
# Restart sshd
sudo systemctl restart sshd
Gotcha: Do not close your current SSH session until you have verified you can open a new one. Open a second terminal and test:
ssh deploy@payment-prod-01. If it works, you're safe. If not, your original session is still open to fix things.
The port change debate¶
Should you move SSH off port 22? It drops 99% of automated scans but anyone with nmap
finds the new port in seconds. Verdict: a noise reducer, not a security control. Don't
treat it as a substitute for key-only auth or fail2ban.
Install fail2ban¶
sudo apt install fail2ban -y
# Create a local override (don't edit the main config — it gets overwritten on updates)
sudo tee /etc/fail2ban/jail.local << 'EOF'
[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600
EOF
sudo systemctl enable --now fail2ban
# Check status
sudo fail2ban-client status sshd
Under the Hood: fail2ban is a single-threaded Python daemon that tails log files, matches lines against regex filters, and adds iptables/nftables DROP rules for offending IPs. It monitors
/var/log/auth.log(Debian/Ubuntu) or/var/log/secure(RHEL) for failed login patterns. Despite protecting millions of servers worldwide, it processes logs sequentially — on heavily targeted servers (thousands of attempts per minute), fail2ban itself can become a bottleneck.
Flashcard Check: SSH¶
| Question | Answer |
|---|---|
| What two sshd_config directives eliminate brute-force attacks? | PasswordAuthentication no and PubkeyAuthentication yes |
| How do you validate sshd_config without restarting? | sshd -t (syntax check) or sshd -T (show effective config) |
| Why should you test SSH from a new session before closing the old one? | If the config is wrong, the old session is your only way back in |
| What does fail2ban actually do at the packet level? | Adds iptables/nftables DROP rules for IPs that exceed the failure threshold |
Part 2: The Firewall — Only What's Needed, Nothing Else¶
SSH is locked. Now build the wall around everything else.
Name Origin: "iptables" comes from "IP tables" — the kernel data structures holding firewall rules. "nftables" stands for "Netfilter tables" — a complete rewrite of iptables by the Netfilter project, not an evolution of the same code. Despite the similar names, nftables uses a different syntax, a different kernel backend, and different performance characteristics.
The stack: netfilter > iptables/nftables > ufw/firewalld¶
All Linux firewalls talk to netfilter in the kernel. iptables (legacy) and nftables (modern replacement) are the low-level tools. ufw (Ubuntu) and firewalld (RHEL) are friendlier frontends. For our Ubuntu server, we use ufw — then peek underneath.
Build the firewall¶
# Start with default deny
sudo ufw default deny incoming
sudo ufw default allow outgoing
# Allow SSH (do this BEFORE enabling ufw!)
sudo ufw allow 22/tcp comment 'SSH'
# Allow HTTPS (the app's public port)
sudo ufw allow 443/tcp comment 'HTTPS'
# Allow the Prometheus node_exporter for monitoring
sudo ufw allow from 10.0.0.0/8 to any port 9100 proto tcp comment 'node_exporter'
# Enable the firewall
sudo ufw enable
# Verify
sudo ufw status verbose
Gotcha: If you run
sudo ufw enablebefore allowing SSH, you just locked yourself out. Always add the SSH rule first. This is the single most common firewall mistake in existence, and it happens to experienced engineers who are "just going to be quick."
What ufw actually created (the iptables underneath)¶
You will see something like:
Chain ufw-user-input (1 references)
num pkts bytes target prot opt in out source destination
1 423 25K ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
2 1.2K 72K ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:443
3 87 5220 ACCEPT tcp -- * * 10.0.0.0/8 0.0.0.0/0 tcp dpt:9100
Mental Model: A firewall is a bouncer with a clipboard. Packets arrive, the bouncer checks the list top to bottom. First match wins. If a packet doesn't match any rule, the default policy applies (our default: DROP). This is why rule order matters — a misplaced DROP above an ACCEPT silently kills traffic with no error.
Our ufw default allow outgoing keeps DNS (53), NTP (123), and HTTPS (443) working. If
you ever tighten outbound rules, immediately test apt update and timedatectl status —
blocking those ports breaks package updates, time sync, and certificate validation.
Part 3: Mandatory Access Control — The Second Lock¶
The firewall controls what gets into the server. MAC controls what processes can do once they're running. It is your last line of defense when an attacker gets a shell.
Traditional Linux permissions (DAC) answer: "Does this user have rwx?" Problem: root bypasses everything. MAC adds a second gate that even root cannot bypass — both DAC and MAC must pass. A compromised Nginx running as root is still confined to what the MAC policy allows.
Trivia: SELinux was developed by the NSA and released as open source in December 2000 — the first major open-source contribution by a U.S. intelligence agency. It was merged into the mainline kernel in 2003 (Linux 2.6). The irony of the NSA contributing to open-source security was not lost on the community, especially after the Snowden revelations.
SELinux vs AppArmor — The Quick Version¶
| SELinux | AppArmor | |
|---|---|---|
| Ships with | RHEL, CentOS, Fedora, Rocky | Ubuntu, SUSE, Debian |
| Approach | Label-based (labels survive mv) |
Path-based (easier to read) |
| Learning curve | Steep | Moderate |
| Default policy | Comprehensive (100,000+ rules) | Per-application profiles |
| Container support | Excellent (built-in labeling) | Good (Docker integration) |
In practice: you use whatever your distro ships. Fighting the default is rarely worth it. Our Ubuntu server has AppArmor. If you are on RHEL, read the SELinux section below — both are covered.
AppArmor on Ubuntu¶
sudo aa-status # loaded profiles and modes
ls /etc/apparmor.d/ # profile files live here
sudo aa-enforce /etc/apparmor.d/usr.sbin.nginx # enforce mode
sudo aa-complain /etc/apparmor.d/usr.sbin.nginx # complain (log, don't block)
sudo aa-genprof /usr/sbin/myapp # generate a new profile
An AppArmor profile for nginx looks like this — note how readable it is compared to SELinux:
/usr/sbin/nginx {
#include <abstractions/base>
#include <abstractions/nameservice>
/usr/sbin/nginx mr, # r=read, m=mmap exec
/etc/nginx/** r,
/var/www/** r,
/var/log/nginx/** rw,
/run/nginx.pid rw,
network inet stream, # TCP connections
# Everything else: denied (implicit)
}
SELinux Crash Course (RHEL/CentOS)¶
If you're on a RHEL-family system, SELinux is your MAC. Here's the survival guide.
Modes:
# Check current mode
getenforce # Enforcing, Permissive, or Disabled
sestatus # Detailed status
# Switch temporarily (does not survive reboot)
sudo setenforce 1 # Enforcing
sudo setenforce 0 # Permissive
# Permanent: edit /etc/selinux/config
# SELINUX=enforcing
The troubleshooting loop (this is 90% of SELinux work):
# 1. Something broke. Check for denials.
sudo ausearch -m AVC -ts recent
# 2. Get a human-readable explanation
sudo ausearch -m AVC -ts recent | audit2why
# 3. Usually the fix is one of these:
# a) Wrong file label → fix it
sudo restorecon -Rv /var/www/html/
# b) Need a boolean toggle
sudo setsebool -P httpd_can_network_connect on
# c) Need a custom port label
sudo semanage port -a -t http_port_t -p tcp 8080
# d) Need a custom policy (last resort)
sudo ausearch -m AVC -ts recent | audit2allow -M myfix
cat myfix.te # REVIEW THIS FIRST
sudo semodule -i myfix.pp # then install
Remember: Mnemonic for the SELinux troubleshooting order: "R-B-P-C" — Restorecon (fix labels), Boolean (toggle a switch), Port (add a port label), Custom policy (write a module). Try them in that order. Most problems are solved by R or B. If you're writing custom policy, double-check that you didn't miss a simpler fix.
Gotcha: Never pipe
audit2allowoutput directly into policy without reading the.tefile. It might grant far more access than the specific action that was denied. A rule likeallow httpd_t file_type:file { read open getattr }lets Apache read ALL file types, not just web content.
Flashcard Check: MAC¶
| Question | Answer |
|---|---|
| What does MAC protect against that DAC cannot? | A compromised process running as root — MAC confines even root |
| What's the difference between SELinux and AppArmor's approach? | SELinux uses labels (survive mv), AppArmor uses paths (easier to read) |
| What command shows SELinux denials in human-readable form? | ausearch -m AVC -ts recent \| audit2why |
| What's the first thing to try when SELinux blocks a file access? | restorecon -Rv <path> to fix the file label |
Part 4: Kernel Hardening with sysctl¶
The kernel has hundreds of tunable parameters. Many ship with insecure defaults for
backward compatibility. A few lines in /etc/sysctl.d/ close serious attack vectors.
sudo tee /etc/sysctl.d/99-hardening.conf << 'EOF'
# === Network hardening ===
# Disable IP forwarding (this is not a router)
net.ipv4.ip_forward = 0
# Enable reverse path filtering (anti-IP-spoofing)
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Enable SYN flood protection
net.ipv4.tcp_syncookies = 1
# Disable ICMP redirects (prevent route manipulation)
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
# Disable source routing (prevent packet path manipulation)
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Log martian packets (impossible source addresses)
net.ipv4.conf.all.log_martians = 1
# Ignore ICMP broadcasts (Smurf attack prevention)
net.ipv4.icmp_echo_ignore_broadcasts = 1
# === Kernel hardening ===
# Full ASLR (stack + heap + libraries + mmap)
kernel.randomize_va_space = 2
# Restrict kernel pointer exposure
kernel.kptr_restrict = 2
# Restrict dmesg to root
kernel.dmesg_restrict = 1
# Restrict ptrace (prevent process snooping)
kernel.yama.ptrace_scope = 1
# Disable core dumps for SUID binaries
fs.suid_dumpable = 0
EOF
# Apply immediately
sudo sysctl -p /etc/sysctl.d/99-hardening.conf
# Verify critical ones
sysctl net.ipv4.ip_forward kernel.randomize_va_space net.ipv4.tcp_syncookies
Trivia: ASLR was first implemented for Linux by the PaX project in 2001, partially merged in 2005, and not fully effective until kernel 4.x with KASLR. The value
2means full randomization — stack, heap, libraries, mmap, and VDSO are all randomized.Gotcha: If this server ever runs Docker containers, you must set
net.ipv4.ip_forward = 1. Docker and Kubernetes both need IP forwarding for container networking. A hardening script that sets it to 0 on a Docker host will appear to work until the next container starts — then networking silently breaks. Existing connections survive (conntrack), but new ones fail.
Part 5: Filesystem Hardening — Mount Options and Permissions¶
Mount options are free security. Zero performance cost, significant attack surface reduction.
Harden mount points¶
noexec stops binaries from running (blocks malware in /tmp). nosuid prevents SUID
escalation. nodev blocks device file creation. Apply all three to /tmp and /dev/shm:
# Apply immediately
sudo mount -o remount,nodev,nosuid,noexec /tmp
sudo mount -o remount,nodev,nosuid,noexec /dev/shm
# Make permanent in /etc/fstab:
# tmpfs /tmp tmpfs defaults,nodev,nosuid,noexec 0 0
# tmpfs /dev/shm tmpfs defaults,nodev,nosuid,noexec 0 0
Audit dangerous permissions¶
# Find world-writable files (should be minimal outside /tmp and /proc)
sudo find / -type f -perm -0002 \
-not -path "/proc/*" -not -path "/sys/*" -not -path "/tmp/*" 2>/dev/null
# Find SUID binaries (each one is a potential privilege escalation)
sudo find / -type f -perm -4000 -exec ls -la {} \; 2>/dev/null
Normal SUID binaries: sudo, passwd, su, mount, umount, ping. Anything else —
investigate. Remove what you don't need:
sudo chmod u-s /usr/bin/chage /usr/bin/wall
sudo chmod 750 /home/*
sudo chmod 700 /etc/cron.d /etc/cron.daily /etc/cron.hourly /etc/cron.weekly
sudo chmod 600 /etc/crontab && sudo chown root:root /etc/crontab
Part 6: Auditd — Catching the Intruder¶
Write targeted audit rules¶
sudo tee /etc/audit/rules.d/hardening.rules << 'EOF'
# === Identity and access ===
# Watch for changes to user/group databases
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/sudoers -p wa -k actions
-w /etc/sudoers.d/ -p wa -k actions
# === SSH config tampering ===
-w /etc/ssh/sshd_config -p wa -k sshd
-w /etc/ssh/sshd_config.d/ -p wa -k sshd
# === Privilege escalation ===
# Log all commands run as root by non-root users
-a always,exit -F arch=b64 -S execve -F euid=0 -F auid>=1000 -F auid!=4294967295 -k privilege_escalation
# === Kernel module loading ===
-w /sbin/insmod -p x -k modules
-w /sbin/modprobe -p x -k modules
# === Time changes (compliance requirement) ===
-a always,exit -F arch=b64 -S adjtimex -S settimeofday -k time_change
-a always,exit -F arch=b64 -S clock_settime -k time_change
# === Login tracking ===
-w /var/log/lastlog -p wa -k logins
-w /var/log/faillog -p wa -k logins
# === Make rules immutable (requires reboot to change) ===
-e 2
EOF
# Load the rules
sudo augenrules --load
The -e 2 at the end is critical: it makes audit rules immutable at runtime. An attacker
who gets root cannot disable auditing without rebooting the server — which is noisy and
obvious.
Reading audit logs¶
sudo ausearch -k identity -ts recent # events by key
sudo ausearch -k privilege_escalation --interpret # human-readable
sudo aureport --summary # overview
sudo aureport --failed --summary # failed actions
Raw audit lines are gibberish (type=SYSCALL msg=audit(1678234567.123:456)...). The
--interpret flag translates them. The -k flag filters by the key you assigned — that's
why meaningful key names matter.
Gotcha: Do not audit "everything." A rule like
-a always,exit -S allgenerates gigabytes per hour, fills your disk, degrades performance, and buries real events in noise. Audit specific paths, specific syscalls, and specific user actions. Start with the CIS recommended rules and add to them based on your threat model.
Flashcard Check: Auditd and Filesystem¶
| Question | Answer |
|---|---|
What does -e 2 do in audit rules? |
Makes rules immutable — can't be changed without a reboot |
What does noexec on /tmp prevent? |
Execution of binaries dropped in /tmp (common attacker technique) |
| How do you read audit events in human-readable form? | ausearch -k <key> --interpret |
| What three mount options should /tmp have? | nodev,nosuid,noexec |
Part 7: User Management and Password Policy¶
Restrict sudo¶
# Restrict su to the sudo group
sudo dpkg-statoverride --update --add root sudo 4750 /usr/bin/su
# Ensure sudo logs all commands
echo 'Defaults logfile="/var/log/sudo.log"' | sudo tee /etc/sudoers.d/logging
sudo chmod 440 /etc/sudoers.d/logging
Password policy and session controls¶
Even with key-based SSH, local passwords matter (console access, sudo). Set sane defaults:
sudo apt install libpam-pwquality -y
# Password complexity
sudo tee /etc/security/pwquality.conf << 'EOF'
minlen = 14
dcredit = -1
ucredit = -1
lcredit = -1
ocredit = -1
EOF
# Account lockout: 5 failures, 15-minute lockout
sudo tee /etc/security/faillock.conf << 'EOF'
deny = 5
unlock_time = 900
fail_interval = 900
EOF
# Session timeout (idle shells auto-close after 10 minutes)
echo -e 'TMOUT=600\nreadonly TMOUT\nexport TMOUT' | sudo tee /etc/profile.d/timeout.sh
Trivia: NIST SP 800-63B (2017, reaffirmed 2024) explicitly recommends against forced password rotation and complexity rules. Their research shows these requirements increase the rate of weak, predictable passwords (P@ssw0rd2026!). A 16-character passphrase ("correct horse battery staple") is stronger than a 12-character complex password that users write on sticky notes. However, many compliance frameworks (PCI DSS, HIPAA) still require complexity rules, so you may need them even when the science says otherwise.
Disable unused accounts and services¶
# Lock system accounts that should never log in
for u in games lp mail; do sudo usermod -L -s /usr/sbin/nologin $u; done
# Kill unnecessary services
sudo systemctl disable --now cups avahi-daemon 2>/dev/null
sudo systemctl mask ctrl-alt-del.target
# Set expiration on contractor accounts
sudo chage -E 2026-06-30 contractor_user
Part 8: Automatic Updates and File Integrity¶
Automatic security updates¶
File integrity monitoring with AIDE¶
AIDE takes a snapshot of your filesystem and alerts you when files change unexpectedly:
sudo apt install aide -y
sudo aideinit
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db # activate baseline
sudo aide --check # compare current state
A changed /etc/shadow after a password change is expected. A changed /usr/bin/curl with
no recent apt upgrade is suspicious — someone may have replaced a system binary.
Under the Hood: AIDE stores SHA-256/SHA-512 checksums of contents, permissions, ownership, and timestamps. The trade-off: the database itself must be stored securely. An attacker with root could update the AIDE database to hide their changes. Best practice: keep a copy on a separate, read-only system.
Part 9: The CIS Benchmark Checklist¶
CIS publishes hardening benchmarks for every major OS. Top 20 items from CIS Ubuntu, mapped to what we have done:
| # | CIS Control | Status | How |
|---|---|---|---|
| 1 | Separate /tmp partition with noexec,nosuid,nodev | Done | Part 5 |
| 2 | Separate /var/log and /var/log/audit partitions | Plan | LVM on fresh installs |
| 3 | Disable unused filesystems (cramfs, squashfs, udf) | Do now | See below |
| 4 | Enable ASLR (randomize_va_space=2) | Done | Part 4 |
| 5 | Disable core dumps | Done | Part 4 (fs.suid_dumpable=0) |
| 6 | Configure SSH hardening | Done | Part 1 |
| 7 | Disable root login via SSH | Done | Part 1 |
| 8 | Enable firewall with default deny | Done | Part 2 |
| 9 | Enable AppArmor/SELinux | Done | Part 3 |
| 10 | Configure auditd | Done | Part 6 |
| 11 | Set password policy | Done | Part 7 |
| 12 | Configure account lockout (faillock) | Done | Part 7 |
| 13 | Enable automatic security updates | Done | Part 8 |
| 14 | Set session timeout (TMOUT) | Done | Part 7 |
| 15 | Disable unused services | Done | Part 7 |
| 16 | Configure NTP time sync | Do now | See below |
| 17 | Set banner warnings (/etc/issue) | Do now | See below |
| 18 | Enable file integrity monitoring | Done | Part 8 |
| 19 | Restrict cron to authorized users | Done | Part 5 |
| 20 | Disable IP forwarding and source routing | Done | Part 4 |
The remaining items¶
# 3. Disable unused kernel modules (filesystem types)
sudo tee /etc/modprobe.d/cis-hardening.conf << 'EOF'
install cramfs /bin/true
install squashfs /bin/true
install udf /bin/true
install usb-storage /bin/true
EOF
# 16. Ensure NTP is configured
sudo timedatectl set-ntp true
timedatectl status
# 17. Set login banners (legal warning — required by many compliance frameworks)
sudo tee /etc/issue << 'EOF'
Authorized access only. All activity is monitored and recorded.
EOF
sudo tee /etc/issue.net << 'EOF'
Authorized access only. All activity is monitored and recorded.
EOF
Verify with Lynis¶
Lynis gives a hardening index (0-100). Fresh Ubuntu: ~55. After this lesson: 75-85. The remaining points are log forwarding, disk encryption, and advanced MAC policy.
Part 10: Think Like an Attacker¶
Now test your defenses. For each layer, think like an attacker:
| Attack | Your Defense | Quick Verification |
|---|---|---|
| SSH brute force | Key-only auth + fail2ban | fail2ban-client status sshd |
| Port scan | ufw default deny | nmap -sT localhost |
| Malware in /tmp | noexec mount |
cp /bin/ls /tmp/x; /tmp/x → "Permission denied" |
| SUID escalation | Removed unnecessary SUID | find / -perm -4000 2>/dev/null |
| Config tampering | auditd + immutable rules | ausearch -k identity -ts recent |
| IP spoofing | rp_filter=1 | sysctl net.ipv4.conf.all.rp_filter |
| Web shell | AppArmor/SELinux confinement | aa-status or getenforce |
Flashcard Check: Full Review¶
| Question | Answer |
|---|---|
| What CIS benchmark category covers mount options? | Category 1: Filesystem Configuration |
What does kernel.kptr_restrict = 2 prevent? |
Leaking kernel addresses (used for exploit development) |
| Why does AIDE need its database stored off-system? | An attacker with root could update the database to hide changes |
| What sysctl prevents SYN flood attacks? | net.ipv4.tcp_syncookies = 1 |
| What is the correct SELinux troubleshooting order? | Restorecon, Boolean, Port, Custom policy (R-B-P-C) |
| What does fail2ban do when maxretry is exceeded? | Adds a DROP rule (iptables/nftables) for the offending IP |
Exercises¶
Exercise 1: Quick Win (2 minutes)¶
Check your current system's hardening posture:
# Run these and note the output
getenforce 2>/dev/null || aa-status 2>/dev/null | head -5
sysctl kernel.randomize_va_space
sudo ufw status 2>/dev/null || sudo iptables -L -n 2>/dev/null | head -10
What to look for
- MAC: Is SELinux enforcing or AppArmor loaded? - ASLR: Should be `2` (full randomization) - Firewall: Should show active rules, not empty chainsExercise 2: Detect the Weak SSH Config (5 minutes)¶
Given this sshd_config, find all the security problems:
Port 22
PermitRootLogin yes
PasswordAuthentication yes
PermitEmptyPasswords no
X11Forwarding yes
MaxAuthTries 6
PubkeyAuthentication yes
AllowUsers root admin deploy
Answer
1. `PermitRootLogin yes` — root should never log in via SSH 2. `PasswordAuthentication yes` — enables brute-force attacks 3. `X11Forwarding yes` — unnecessary attack surface 4. `MaxAuthTries 6` — too high, use 3 5. `AllowUsers root` — root is in the allow list (redundant with PermitRootLogin, but shows intent to allow root) 6. Missing: `LoginGraceTime`, `ClientAliveInterval`, cipher restrictionsExercise 3: Write an AppArmor Profile (10 minutes)¶
Write an AppArmor profile for a hypothetical app /opt/paymentd/bin/paymentd that needs to:
- Read its config from /opt/paymentd/etc/
- Write logs to /var/log/paymentd/
- Connect to a PostgreSQL database over TCP
- Read TLS certificates from /etc/ssl/certs/
Hint
Start with `#includeSolution
Exercise 4: Adversarial Audit (10 minutes)¶
On a test system, try to break each layer. Document what was blocked and what wasn't:
ssh root@<host>— should be deniedssh -o PubkeyAuthentication=no deploy@<host>— should fail (no password auth)cp /bin/ls /tmp/x; /tmp/x— should get "Permission denied" (noexec)sudo touch /etc/shadow; sudo ausearch -k identity -ts recent— auditd should log itsudo modprobe usb-storage— should fail (disabled module)
Cheat Sheet¶
| Area | Key Commands |
|---|---|
| SSH | sshd -t (validate), sshd -T (show effective), PermitRootLogin no, PasswordAuthentication no |
| Firewall | ufw default deny incoming, ufw allow 443/tcp, ufw status verbose, iptables -L -n -v |
| SELinux | getenforce, ausearch -m AVC -ts recent \| audit2why, restorecon -Rv, setsebool -P |
| AppArmor | aa-status, aa-enforce, aa-complain, aa-genprof |
| sysctl | rp_filter=1 (anti-spoof), tcp_syncookies=1 (SYN flood), randomize_va_space=2 (ASLR), ip_forward=0 |
| Auditd | ausearch -k <key> --interpret, aureport --summary, auditctl -l |
| AIDE | aideinit (baseline), aide --check (compare), store DB off-system |
| Mount opts | noexec,nosuid,nodev on /tmp and /dev/shm |
Takeaways¶
-
SSH hardening is step zero. Key-only auth, no root login, fail2ban. Do this within minutes of provisioning, not "after lunch."
-
Firewalls are whitelists. Default deny incoming, explicitly allow only what the server needs. Always add the SSH rule before enabling the firewall.
-
MAC (SELinux/AppArmor) is your last defense. It confines processes even after a root compromise. Never disable it — learn to troubleshoot it.
-
sysctl tuning is cheap and powerful. A few lines in
/etc/sysctl.d/close network attack vectors that have existed since the 1990s. -
Mount options are free security.
noexec,nosuid,nodevon/tmpand/dev/shmcosts nothing and blocks a major class of attacks. -
Auditd catches what firewalls miss. Immutable rules (
-e 2) prevent attackers from covering their tracks without rebooting. -
Hardening is not a one-time event. CIS benchmarks, AIDE scans, and unattended updates keep the server hardened as it drifts through its lifecycle.
Related Lessons¶
- SSH Is More Than You Think — deep dive into SSH protocol, tunnels, and agent forwarding
- iptables: Following a Packet Through the Chains — how Linux packet filtering actually works
- Permission Denied — differential diagnosis of permission errors across DAC, MAC, and containers
- The Container Escape — when hardening meets containerization
- Compliance as Code — automating CIS benchmarks with OpenSCAP and Ansible
- Secrets Management Without Tears — the next layer after host hardening