Linux Recovery

← Back to all decks

10 cards — 🟢 1 easy | 🟡 5 medium | 🔴 4 hard

🟢 Easy (1)

1. Compare locate/updatedb vs find for file searching.

Show answer

`find` searches the filesystem in real-time by traversing directories. It is always current but can be slow on large filesystems.
`locate` queries a pre-built database (updated by `updatedb`, typically via daily cron). It is nearly instant but may return stale results.
Key differences:
- Speed: locate is O(1) lookup vs find's O(n) traversal
- Freshness: find is real-time; locate may be hours old
- Features: find supports `-exec`, `-mtime`, `-size`, `-perm` predicates; locate only matches path patterns
- Update: `sudo updatedb` refreshes the locate database manually
Use `locate` for quick name lookups; `find` when you need current results or complex criteria.

🟡 Medium (5)

1. What is the difference between /dev/random and /dev/urandom?

Show answer

Both provide pseudorandom bytes from the kernel's entropy pool.
`/dev/random` (historically) blocks when the entropy pool is depleted, waiting for more environmental noise. `/dev/urandom` never blocks -- it reuses the internal pool.
As of Linux 5.6+, `/dev/random` behaves like `/dev/urandom` after initial seeding (both are non-blocking). Before 5.6, `/dev/random` blocking could stall applications.
Best practice: **always use /dev/urandom** (or `getrandom()` syscall). It is cryptographically secure after initial boot seeding. The only exception is generating long-lived cryptographic keys on a freshly installed system before enough entropy is gathered.

2. How do ACL commands getfacl and setfacl work?

Show answer

ACLs (Access Control Lists) extend standard Unix permissions to allow per-user or per-group rules.
- `getfacl file` -- display all ACL entries
- `setfacl -m u:alice:rwx file` -- give user alice rwx on file
- `setfacl -m g:devs:rx dir` -- give group devs r-x on dir
- `setfacl -x u:alice file` -- remove alice's ACL entry
- `setfacl -d -m g:devs:rx dir` -- set default ACL for new files in dir
- `setfacl -b file` -- remove all ACLs
A `+` in `ls -l` output indicates ACLs are set. ACLs require the filesystem to be mounted with `acl` option (default on ext4, XFS).

3. What are the key LVM snapshot commands?

Show answer

LVM snapshots capture the state of a logical volume at a point in time using copy-on-write.
- `lvcreate -s -n snap01 -L 5G /dev/vg0/data` -- create a 5GB snapshot of `data`
- `lvs` -- list all LVs including snapshots and their usage %
- `mount /dev/vg0/snap01 /mnt/snap` -- mount snapshot read-only for backup
- `lvconvert --merge /dev/vg0/snap01` -- merge snapshot back into origin (revert)
- `lvremove /dev/vg0/snap01` -- delete the snapshot
Snapshot space is consumed as the origin changes. If the snapshot fills up, it becomes invalid. Always allocate enough space and monitor with `lvs`.

4. How do you manage AppArmor profiles?

Show answer

AppArmor uses per-program profiles to restrict capabilities.
- `aa-status` -- show loaded profiles and their mode (enforce/complain)
- `aa-enforce /etc/apparmor.d/usr.bin.app` -- set profile to enforce mode
- `aa-complain /etc/apparmor.d/usr.bin.app` -- set to complain (log but allow)
- `aa-disable /etc/apparmor.d/usr.bin.app` -- disable a profile
- `aa-genprof /usr/bin/app` -- generate a profile interactively
- `aa-logprof` -- update profiles based on logged violations
Profiles live in `/etc/apparmor.d/`. Reload after editing: `apparmor_parser -r /etc/apparmor.d/profile`. AppArmor is the default MAC on Ubuntu/SUSE; RHEL uses SELinux.

5. What are the key iproute2 IPv6 commands?

Show answer

iproute2 handles IPv6 natively with the `-6` flag or `inet6` family:
- `ip -6 addr show` -- display IPv6 addresses on all interfaces
- `ip -6 route show` -- display IPv6 routing table
- `ip -6 route add 2001:db8::/32 via fe80::1 dev eth0` -- add a static IPv6 route
- `ip -6 neigh show` -- show IPv6 neighbor cache (equivalent to ARP for IPv4)
- `ip -6 addr add 2001:db8::1/64 dev eth0` -- assign an IPv6 address
- `ss -6 -tlnp` -- show IPv6 TCP listening sockets
IPv6 link-local addresses (fe80::/10) are auto-assigned on every interface. The `%dev` suffix specifies the interface for link-local: `ping6 fe80::1%eth0`.

🔴 Hard (4)

1. How do you recover from "chmod -x /bin/chmod"?

Show answer

Several approaches since chmod itself is no longer executable:
1. **ld-linux.so**: `/lib64/ld-linux-x86-64.so.2 /bin/chmod +x /bin/chmod` -- the dynamic linker can execute ELF binaries directly.
2. **Perl**: `perl -e 'chmod 0755, "/bin/chmod"'`
3. **Python**: `python3 -c 'import os; os.chmod("/bin/chmod", 0o755)'`
4. **busybox**: If available, `busybox chmod +x /bin/chmod`
5. **Copy from another system**: `scp` a working chmod binary, or install the coreutils package.
The ld-linux approach is the most universal and does not require any interpreted language to be installed.

2. How do you recover a deleted file still held open by a process?

Show answer

If a process has a file open, the inode is not freed even after deletion. The file descriptor persists in `/proc/PID/fd/`.
Recovery steps:
1. Find the PID: `lsof | grep deleted` or `lsof +D /path/to/dir`
2. Find the fd: `ls -la /proc//fd/` -- look for the symlink pointing to the deleted file
3. Copy it out: `cp /proc//fd/ /path/to/recovered_file`
This works because Linux only frees disk blocks when the reference count (open file handles + hard links) reaches zero. The file remains fully readable through the fd. This technique commonly recovers accidentally deleted log files or database files.

3. What does the fork bomb :(){ :|:& };: do and how do you stop it?

Show answer

It defines a function named `:` that calls itself twice (piped), backgrounded. Each call spawns two more processes exponentially, exhausting the process table.
Breakdown: `:()` defines function `:`. `{ :|:& }` is the body: call `:`, pipe output to another `:`, run in background. `;` ends the definition. `:` invokes it.
To stop: If you can get a shell, `killall -9 :` or `kill -9 -1` (kill all your processes). If not, you need console/SSH access from another session.
Prevention: Set process limits in `/etc/security/limits.conf`:
`* hard nproc 500`
Or use cgroups to limit PIDs per user. `ulimit -u 500` sets a soft limit for the current session.

4. What are the basics of LDAP/Kerberos integration on Linux?

Show answer

LDAP provides directory services (user accounts, groups). Kerberos provides authentication (tickets, SSO). Together they enable centralized identity management.
Setup components:
- `sssd` (System Security Services Daemon) -- the modern client that caches and mediates LDAP/Kerberos
- `/etc/sssd/sssd.conf` -- configure domains, LDAP URI, Kerberos realm
- `realm join EXAMPLE.COM` -- join an Active Directory domain (uses sssd + Kerberos)
- `kinit user@REALM` -- obtain a Kerberos ticket
- `klist` -- show current tickets
- `id username` -- verify LDAP user resolution
Alternative stack: `nslcd` + `pam_ldap` (older). SSSD is preferred as it handles caching, failover, and offline login.