Mounts & Filesystems - Street-Level Ops¶
Real-world mount diagnosis and filesystem management workflows for production Linux systems.
Task: Find All Current Mounts and Their Options¶
# Tree view — the best way to see the mount hierarchy
$ findmnt
TARGET SOURCE FSTYPE OPTIONS
/ /dev/sda2 ext4 rw,relatime
|-/boot /dev/sda1 ext4 rw,relatime
|-/mnt/data /dev/sdb1 xfs rw,noatime
|-/tmp tmpfs tmpfs rw,nosuid,noexec
|-/var/lib/docker /dev/sdc1 xfs rw,noatime
# Find what is mounted at a specific path
$ findmnt -T /mnt/data
TARGET SOURCE FSTYPE OPTIONS
/mnt/data /dev/sdb1 xfs rw,noatime
# Show block devices and their mount points
$ lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
|-sda1 ext4 a1b2c3d4-... /boot
|-sda2 ext4 e5f6a7b8-... /
sdb
|-sdb1 xfs c9d0e1f2-... /mnt/data
Task: Mount a New Disk¶
# Identify the new disk
$ lsblk
sdc 8:32 0 100G 0 disk
# Create a filesystem
$ mkfs.ext4 /dev/sdc1
# Or for large file workloads:
$ mkfs.xfs /dev/sdc1
# Create the mount point and mount
$ mkdir -p /mnt/storage
$ mount /dev/sdc1 /mnt/storage
# Get the UUID for fstab (never use /dev/sdX — it can change)
> **Gotcha:** Device names like `/dev/sdb` can change between reboots if disks are added, removed, or probed in a different order. UUIDs are permanently tied to the filesystem and survive reboots, cable swaps, and RAID rebuilds. Always use `UUID=` in fstab.
$ blkid /dev/sdc1
/dev/sdc1: UUID="3e8c4f1a-7b2d-4e9f-a1c3-8d5e6f7a0b1c" TYPE="ext4"
# Add to fstab with nofail
$ cat >> /etc/fstab <<'EOF'
UUID=3e8c4f1a-7b2d-4e9f-a1c3-8d5e6f7a0b1c /mnt/storage ext4 defaults,noatime,nofail 0 2
EOF
# Test fstab BEFORE rebooting
$ umount /mnt/storage
$ mount -a
$ findmnt /mnt/storage # Should show mounted
Task: Fix "Device Is Busy" When Unmounting¶
# Cannot unmount — something is using it
$ umount /mnt/data
umount: /mnt/data: target is busy
# Find what is using the mount
$ fuser -vm /mnt/data
USER PID ACCESS COMMAND
/mnt/data: root 1234 ..c.. bash
app 5678 F.... python3
# c = current directory, F = open file for writing
# e = executable, r = root directory, m = mmap'd file
# Option 1: Ask processes to stop
$ kill 1234 5678
$ umount /mnt/data
# Option 2: Lazy unmount (detaches immediately, cleans up when free)
$ umount -l /mnt/data
# Option 3: Force kill everything using the mount
$ fuser -km /mnt/data # Sends SIGKILL — use as last resort
$ umount /mnt/data
Task: Recover from a Bad fstab Entry¶
# Server hung at boot — bad fstab entry
# From recovery console or single-user mode:
# Remount root as read-write
$ mount -o remount,rw /
# Comment out the bad line
$ vi /etc/fstab
# UUID=bad-entry /mnt/gone nfs defaults 0 0
# Reboot
$ reboot
# Prevent this next time: always use nofail
# UUID=xxx /mnt/data ext4 defaults,nofail 0 2
Task: Diagnose a Hung NFS Mount¶
> **Debug clue:** Processes in `D` state (uninterruptible sleep) are waiting on I/O and cannot be killed, not even with `kill -9`. They are almost always stuck on a hung NFS mount or a failing disk. `D` state processes will clear once the underlying I/O completes or the mount is lazily unmounted.
# Application processes stuck in D state (uninterruptible sleep)
$ ps aux | grep " D "
app 5678 D python3 /opt/myapp/server.py
# Check NFS mount state
$ mount | grep nfs
nfs-server:/export/data on /mnt/nfsdata type nfs (hard,timeo=600)
# Is the NFS server reachable?
$ ping -c 1 nfs-server
# No response — server is down
# Lazy unmount to free stuck processes
$ umount -l /mnt/nfsdata
# For future NFS mounts, use these options:
# hard,timeo=300,retrans=3,bg,_netdev,nofail
# "bg" retries in background, "_netdev" waits for network at boot
Task: Set Up a tmpfs for Fast Temporary Storage¶
# Create a RAM-backed filesystem for build caches
$ mount -t tmpfs -o size=2G,noexec,nosuid tmpfs /tmp/build-cache
# Persist in fstab
$ echo 'tmpfs /tmp/build-cache tmpfs size=2G,noexec,nosuid,nofail 0 0' >> /etc/fstab
# Check usage
$ df -h /tmp/build-cache
Filesystem Size Used Avail Use% Mounted on
tmpfs 2.0G 0 2.0G 0% /tmp/build-cache
Task: Create a Bind Mount¶
# Share host logs inside a chroot
$ mount --bind /var/log /mnt/chroot/var/log
# Make it read-only (two steps required)
$ mount --bind /var/log /mnt/chroot/var/log
$ mount -o remount,bind,ro /mnt/chroot/var/log
# Persist in fstab
$ echo '/var/log /mnt/chroot/var/log none bind,ro 0 0' >> /etc/fstab
Task: Remount Filesystem Read-Only in Emergency¶
# Filesystem errors detected — prevent further writes
$ mount -o remount,ro /mnt/data
# Run filesystem check
$ fsck -y /dev/sdb1
# Remount read-write after repair
$ mount -o remount,rw /mnt/data
# For root filesystem (recovery mode):
$ mount -o remount,rw /
Task: Check Filesystem Health¶
# View filesystem info
$ tune2fs -l /dev/sda2 | grep -E "Filesystem state|Last checked|Mount count"
Filesystem state: clean
Last checked: Sun Mar 01 03:00:00 2026
Mount count: 42
# Force check on next reboot
$ tune2fs -C 100 -c 50 /dev/sda2 # Trigger at mount count 50
# XFS health check (can run on mounted filesystem)
$ xfs_repair -n /dev/sdb1 # -n = dry run, check only
Task: Verify fstab Before Rebooting¶
# ALWAYS test fstab changes
$ mount -a
# If this fails, fix fstab BEFORE rebooting
# Check for common mistakes
$ cat /etc/fstab | grep -v '^#' | grep -v '^$'
UUID=a1b2c3d4 / ext4 defaults 0 1
UUID=e5f6a7b8 /boot ext4 defaults 0 2
UUID=c9d0e1f2 /mnt/data xfs defaults,nofail 0 2
# Verify all UUIDs exist
$ blkid | grep a1b2c3d4
/dev/sda2: UUID="a1b2c3d4-..." TYPE="ext4"
Emergency: Root Filesystem Full¶
# Cannot even run commands — "No space left on device"
# Step 1: Find and delete the largest offenders
$ find / -xdev -type f -size +100M -printf '%s %p\n' | sort -rn | head -10
# Step 2: Check for deleted files held open
$ lsof +L1 | head -10
# Step 3: Clean package manager cache
$ apt-get clean # Debian/Ubuntu
$ dnf clean all # RHEL/Fedora
# Step 4: Truncate large logs
$ > /var/log/syslog
$ journalctl --vacuum-size=100M
Default trap:
lsof +L1finds deleted files still held open by processes. A common culprit is log rotation: logrotate renames the file, but the process keeps writing to the old (now deleted) file descriptor. The space is not freed until the process is restarted or the fd is closed. This is whydfanddudisagree on disk usage.
Power One-Liners¶
Formatted filesystem overview¶
or for disk-centric view:
[!TIP] When to use: Understanding what's mounted where, filesystem types, and options in effect.
Quick Reference¶
- Deep Dive: Linux Filesystem Internals