Skip to content

Portal | Level: L0: Entry | Topics: Linux Fundamentals, systemd, Package Management | Domain: Linux

Linux Fundamentals - Skill Check

Mental model (bottom-up)

Linux is a kernel that offers syscalls. Everything else (shell, systemd, tools) sits on top. Most "Linux problems" reduce to: permissions, namespaces, process state, filesystem state, or resource pressure.

Visual stack

[Your command     ]  bash/zsh runs tools, sets env, pipes streams
|
[Userspace tools  ]  ps/ss/journalctl/find/ip/grep
|
[Libraries        ]  glibc, NSS, SSL, PAM, etc
|
[Syscalls         ]  open/read/write/fork/exec/socket
|
[Kernel           ]  scheduler, VFS, netstack, drivers
|
[Hardware         ]  CPU/RAM/disk/NIC

Glossary (jargon explained)

  • kernel - core of OS; controls CPU, memory, devices
  • syscall - userspace asking kernel to do something
  • VFS - virtual filesystem layer that unifies ext4/xfs/btrfs/etc
  • inode - file metadata record; names live in directories
  • FD - file descriptor number (0/1/2 = stdin/out/err)
  • cgroup - resource control for processes (CPU/mem/io)
  • namespace - isolation boundary (pid/net/mount/user/etc)
  • unit - systemd-managed object: service/socket/timer/mount/target
  • iowait - CPU time waiting on disk/network I/O completion
  • alignment - partition boundary matches physical/erase block sizes

Core internals you should actually know

  • Process = PID + memory mappings + open FDs + creds + cgroups/namespaces.
  • Filesystem path lookup = walk directory entries -> inodes -> permissions.
  • Service startup = systemd loads unit -> resolves deps -> spawns process -> monitors exit.

Common failure modes (ops reality)

  • "Permission denied" that's really missing execute bit on a directory.
  • Disk "full" from inodes or a read-only remount, not bytes.
  • Service works in shell but not systemd because env/PATH/working dir differs.
  • Latency spikes from storage queueing (watch await, util%, iowait).

Roadmap core (10, easy -> hard)

  • What's a PID?
  • Unique ID for a running process in the kernel.
  • Where do config files usually live?
  • /etc (system-wide); user-specific usually under ~/.config or dotfiles.
  • Absolute vs relative path?
  • Absolute starts at /; relative starts from current working directory.
  • /tmp vs /var/tmp?
  • /tmp often cleared on reboot/age policy; /var/tmp intended to persist longer.
  • Explain stdin/stdout/stderr.
  • Input stream, normal output, error output (FD 0/1/2).
  • What does chmod 750 file mean?
  • Owner rwx, group r-x, others ---.
  • What is a "mount"?
  • Attaching a filesystem to a directory in the unified tree.
  • Hardlink vs symlink?
  • Hardlink = another name for same inode; symlink = path pointer (can break).
  • What does systemctl actually control?
  • systemd units: services, timers, mounts, sockets, targets.
  • Why align partitions (e.g., 1MiB boundaries)?
  • Avoid misaligned I/O penalties on 4K/RAID/SSD erase blocks; better performance/endurance.

Filesystems & permissions (easy -> hard)

  • What's the difference: file vs directory permissions?
  • Directory x means "traverse"; file x means "execute".
  • Why is umask important?
  • It sets default permission masking for newly created files/dirs.
  • What do setuid/setgid/sticky do (in one line each)?
  • setuid: exec runs as owner; setgid: runs as group / group inheritance; sticky: only owner can delete in dir.
  • What's an inode?
  • Metadata record (mode/owner/times/pointers); name lives in directory entries.
  • What's the point of ACLs?
  • Extra per-user/group permissions beyond classic ugo.
  • How do you safely find "world-writable" stuff?
  • find / -xdev -type f -perm -0002 -ls (and review).
  • How do you explain "permission denied" on a directory you can ls?
  • You might have r but not x on the directory (can list names, can't traverse/open).
  • Why mount options matter (noexec, nosuid, nodev)?
  • They reduce attack surface by disabling exec, suid bits, device nodes on that mount.

Processes, signals, services (easy -> hard)

  • What's a daemon?
  • Long-running background service (usually managed by init/systemd).
  • SIGTERM vs SIGKILL?
  • TERM asks nicely; KILL is immediate and uncatchable.
  • Zombie vs orphan process?
  • Zombie: exited but not reaped; orphan: parent died, adopted by PID 1.
  • What does nice/renice affect?
  • CPU scheduling priority (not I/O priority).
  • Where do you look for listening ports?
  • ss -lntup (or ss -lntu + process lookup).
  • systemd: enable vs start?
  • Enable = on boot; start = now.
  • systemd: mask vs disable?
  • Disable removes from boot; mask prevents any start (even manual) by symlinking to /dev/null.
  • What's a "unit dependency loop" smell?
  • Conflicting Requires/After chains; often fix with Wants or correct ordering.

Storage & performance (easy -> hard)

  • Block vs object storage (Linux view)?
  • Block = attachable device; object = HTTP API (not a block device).
  • What's the fastest sanity check for disk health?
  • smartctl -a summary + reallocated/pending sectors + error log.
  • What does iowait usually imply?
  • CPUs waiting on I/O; often storage bottleneck or blocked queueing.
  • What's the point of fstrim on SSDs?
  • Tells SSD which blocks are free to improve GC and performance.
  • What does "write amplification" mean?
  • SSD writes more internally than host writes; hurts endurance/perf.
  • When does RAID rebuild hurt you most?
  • During heavy I/O + degraded state; latency spikes.
  • What's a quick "latency vs throughput" diagnosis toolset?
  • iostat -xz, pidstat -d, iotop, sar, plus app-level metrics.
  • What does it mean if df shows free space but writes fail?
  • Inodes exhausted, quota, read-only remount, or underlying errors.

Troubleshooting patterns (easy -> hard)

  • "DNS or network?" quickest split test?
  • dig/getent hosts vs ping/curl --resolve vs raw IP.
  • "Service down" minimal triage?
  • systemctl status, journalctl -u svc -b, check ports (ss).
  • What's the value of journalctl -xe?
  • Recent errors with context; great for boot/service failures.
  • How do you catch a flapping service root cause?
  • Inspect restart counters + logs + dependency readiness + resource pressure.
  • Why "it works in shell but not in systemd" happens?
  • Different env, PATH, working dir, permissions, missing dependencies.
  • When do you use strace?
  • Black-box failing binary: syscalls reveal missing files, perms, DNS, etc.
  • When do you use tcpdump?
  • Prove what's actually on the wire (DNS answers, SYNs, TLS failures).
  • What is "blame the last change" but do it correctly?
  • Correlate deploy/config change window with metrics/logs; confirm by rollback or diff.

Security fundamentals (easy -> hard)

  • Why SSH keys beat passwords (ops view)?
  • Stronger, auditable, can be revoked; supports MFA via bastions/SSO layers.
  • What is least privilege in Linux?
  • Minimal sudo + group perms + service accounts + file ownership.
  • Why do secrets leak in logs?
  • Debug prints, env dumps, CLI flags; fix with redaction + careful logging.
  • Why separate /var can save you?
  • Log growth fills /var without bricking root filesystem.
  • What's a "supply chain" risk for Linux nodes?
  • Untrusted repos/images/binaries -> compromise; mitigate with pinning/signing.
  • What's the point of immutable infrastructure?
  • Replace instead of mutate: reduces drift and hidden state.
  • What's "kernel parameter hardening" in one line?
  • sysctls to reduce attack surface (network, memory, ptrace, etc.).
  • Practical security baseline check?
  • Patch cadence + SSH policy + firewall + audit logging + minimal services.

Mini labs

  • Break/restore perms on a test dir; explain why x on dir matters.
  • Make a systemd override; prove env differs from interactive shell.
  • Create a misaligned partition on a spare image; measure effects.

Cleanup / teardown

  • Remove test units: systemctl disable --now <unit> then delete drop-ins.
  • Remove test filesystems: umount <mnt> then delete loop devices if used.

Fast triage chain

systemctl status -> journalctl -u -> ss -> df -h/free -> top/iostat.

Sources

  • https://www.kernel.org/doc/
  • https://www.freedesktop.org/software/systemd/man/systemctl.html

Wiki Navigation

Related Content