Skip to content

/proc Filesystem - Primer

Why This Matters

The /proc filesystem is the kernel's API exposed as files. Every running process has a directory under /proc/PID/ that reveals its state, memory layout, open files, network connections, resource limits, and cgroup membership. System-wide entries expose CPU information, memory statistics, kernel tunables, and network stats. Tools like ps, top, free, lsof, and sysctl are all frontends to /proc. When those tools aren't installed or don't show what you need, reading /proc directly is the fallback that always works.

What /proc Is

/proc is a virtual filesystem (also called a pseudo-filesystem). It occupies no disk space. The "files" are generated on the fly by the kernel when you read them. Writing to certain files under /proc/sys/ changes kernel parameters in real time.

Name origin: The /proc filesystem (procfs) originated in Unix 8th Edition at Bell Labs (1984), designed by Tom Killian. It was initially just for process information (hence "proc"), but Linux extended it to include system-wide kernel data. Plan 9 from Bell Labs later took this "everything is a file" concept much further, and Linux's /proc borrows some of those ideas.

# /proc is mounted automatically at boot
$ mount | grep proc
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)

# Files have zero size but contain data
$ ls -la /proc/meminfo
-r--r--r-- 1 root root 0 Mar 19 10:00 /proc/meminfo
$ cat /proc/meminfo | head -5
MemTotal:       16384000 kB
MemFree:         2048000 kB
MemAvailable:    8192000 kB
Buffers:          512000 kB
Cached:          5120000 kB

Per-Process Entries (/proc/PID/)

Every process gets a directory named by its PID. The entries are owned by the process's UID, so you can inspect your own processes without root.

/proc/PID/status -- Process State Overview

The most readable per-process file. Contains process state, memory, capabilities, and namespace info in human-friendly format.

$ cat /proc/$(pgrep -f "nginx: master")/status
Name:   nginx
Umask:  0022
State:  S (sleeping)
Tgid:   1234
Pid:    1234
PPid:   1
TracerPid:      0
Uid:    0       0       0       0
Gid:    0       0       0       0
FDSize: 64
VmPeak:   128456 kB
VmSize:   128456 kB
VmRSS:     45232 kB
VmSwap:        0 kB
Threads:        1
Cpus_allowed:   f
voluntary_ctxt_switches:    15423
nonvoluntary_ctxt_switches: 892

Key fields: - State: R (running), S (sleeping), D (disk sleep/uninterruptible), Z (zombie), T (stopped) - VmRSS: Resident Set Size -- actual physical memory used - VmSwap: Memory swapped to disk - Threads: Number of threads - voluntary_ctxt_switches: Process yielded the CPU (I/O wait, sleep) - nonvoluntary_ctxt_switches: Kernel preempted the process (CPU-bound, contention)

/proc/PID/cmdline -- How the Process Was Invoked

# Arguments are null-separated
$ cat /proc/1234/cmdline | tr '\0' ' '
nginx: master process /usr/sbin/nginx -g daemon off;

# Useful for finding processes with specific arguments
$ for pid in /proc/[0-9]*/; do
    cmd=$(cat "$pid/cmdline" 2>/dev/null | tr '\0' ' ')
    [[ "$cmd" == *"--config"* ]] && echo "$pid: $cmd"
  done

/proc/PID/environ -- Process Environment Variables

# Environment at exec time (null-separated)
$ cat /proc/1234/environ | tr '\0' '\n' | sort
HOME=/root
LANG=C.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
TERM=xterm

# Search for specific variables
$ cat /proc/1234/environ | tr '\0' '\n' | grep DATABASE
DATABASE_URL=postgresql://db:5432/app

/proc/PID/fd/ -- Open File Descriptors

Every open file, socket, pipe, and device is represented as a symlink in this directory.

$ ls -la /proc/1234/fd/
lrwx------ 1 root root 64 Mar 19 10:00 0 -> /dev/null
lrwx------ 1 root root 64 Mar 19 10:00 1 -> /var/log/nginx/access.log
lrwx------ 1 root root 64 Mar 19 10:00 2 -> /var/log/nginx/error.log
lrwx------ 1 root root 64 Mar 19 10:00 3 -> socket:[45678]
lrwx------ 1 root root 64 Mar 19 10:00 7 -> /var/log/nginx/access.log (deleted)

# Count open file descriptors
$ ls /proc/1234/fd/ | wc -l
47

# Find sockets
$ ls -la /proc/1234/fd/ | grep socket
lrwx------ 1 root root 64 Mar 19 10:00 3 -> socket:[45678]

# Cross-reference socket inode with /proc/net/tcp
$ grep 45678 /proc/net/tcp6

/proc/PID/maps -- Memory Mappings

Shows every memory region mapped into the process's address space: code, data, heap, stack, shared libraries, anonymous mappings, and memory-mapped files.

$ cat /proc/1234/maps | head -10
55a3b4000000-55a3b4020000 r--p 00000000 08:01 1234567  /usr/sbin/nginx
55a3b4020000-55a3b40a0000 r-xp 00020000 08:01 1234567  /usr/sbin/nginx
55a3b40a0000-55a3b40c0000 r--p 000a0000 08:01 1234567  /usr/sbin/nginx
7f8a12000000-7f8a12200000 rw-p 00000000 00:00 0
7f8a14000000-7f8a14180000 r--p 00000000 08:01 2345678  /usr/lib/x86_64-linux-gnu/libc.so.6

# Permission flags: r=read, w=write, x=execute, p=private, s=shared

/proc/PID/smaps -- Detailed Memory Breakdown

Like /proc/PID/maps but with per-region memory statistics: RSS, PSS (Proportional Set Size), shared/private clean/dirty pages, and swap.

# Total private memory (not shared with other processes)
$ awk '/Private_Dirty:/{sum+=$2} END{print sum, "kB"}' /proc/1234/smaps
12456 kB

# PSS gives a fair share of shared memory (shared_pages / num_sharing_processes)
$ awk '/Pss:/{sum+=$2} END{print sum, "kB"}' /proc/1234/smaps
34567 kB

/proc/PID/io -- I/O Statistics

$ cat /proc/1234/io
rchar: 1234567890        # bytes read (includes page cache hits)
wchar: 987654321         # bytes written (includes to page cache)
syscr: 45678             # read syscalls
syscw: 23456             # write syscalls
read_bytes: 567890123    # bytes actually read from storage
write_bytes: 345678901   # bytes actually written to storage
cancelled_write_bytes: 0 # bytes cancelled (truncated files)

The difference between rchar and read_bytes: rchar counts every read syscall (including cache hits), while read_bytes counts only actual disk I/O.

Remember: "rchar = requested, read_bytes = real." When diagnosing I/O problems, read_bytes tells you the actual disk load. If rchar is high but read_bytes is low, the page cache is doing its job. If both are high, the working set exceeds cache and you have genuine disk pressure.

/proc/PID/limits -- Resource Limits

$ cat /proc/1234/limits
Limit                     Soft Limit     Hard Limit     Units
Max cpu time              unlimited      unlimited      seconds
Max file size             unlimited      unlimited      bytes
Max data size             unlimited      unlimited      bytes
Max stack size            8388608        unlimited      bytes
Max core file size        0              unlimited      bytes
Max resident set          unlimited      unlimited      bytes
Max processes             63432          63432          processes
Max open files            1024           1048576        files
Max locked memory         65536          65536          bytes
Max address space         unlimited      unlimited      bytes
Max file locks            unlimited      unlimited      locks
Max pending signals       63432          63432          signals
Max msgqueue size         819200         819200         bytes
Max nice priority         0              0
Max realtime priority     0              0
Max realtime timeout      unlimited      unlimited      us

/proc/PID/cgroup -- Cgroup Membership

# On cgroup v2 (most modern systems):
$ cat /proc/1234/cgroup
0::/system.slice/nginx.service

# On cgroup v1:
$ cat /proc/1234/cgroup
12:memory:/system.slice/nginx.service
11:cpu,cpuacct:/system.slice/nginx.service
...

# For containers, this reveals the container's cgroup path
$ cat /proc/1/cgroup
0::/kubepods/burstable/pod-abc123/container-def456

/proc/PID/ns/ -- Namespaces

# Each file is a symlink to the namespace inode
$ ls -la /proc/1234/ns/
lrwxrwxrwx 1 root root 0 Mar 19 10:00 cgroup -> 'cgroup:[4026531835]'
lrwxrwxrwx 1 root root 0 Mar 19 10:00 ipc -> 'ipc:[4026531839]'
lrwxrwxrwx 1 root root 0 Mar 19 10:00 mnt -> 'mnt:[4026531840]'
lrwxrwxrwx 1 root root 0 Mar 19 10:00 net -> 'net:[4026531992]'
lrwxrwxrwx 1 root root 0 Mar 19 10:00 pid -> 'pid:[4026531836]'
lrwxrwxrwx 1 root root 0 Mar 19 10:00 user -> 'user:[4026531837]'
lrwxrwxrwx 1 root root 0 Mar 19 10:00 uts -> 'uts:[4026531838]'

# Two processes in the same namespace share the same inode number
# Compare container process vs host process to verify isolation

/proc/PID/stat and /proc/PID/statm

Low-level process statistics in compact machine-readable format. Most tools (ps, top, htop) read these.

# /proc/PID/stat -- single line, space-separated fields
# Field 1: PID, 2: comm, 3: state, 14: utime, 15: stime, 22: starttime, 23: vsize, 24: rss
$ cat /proc/1234/stat
1234 (nginx) S 1 1234 1234 0 -1 4194304 ...

# /proc/PID/statm -- memory in pages (usually 4096 bytes)
# Fields: size resident shared text lib data dt
$ cat /proc/1234/statm
32114 11308 2456 160 0 14562 0
# Total: 32114 pages * 4K = ~125MB virtual
# RSS: 11308 pages * 4K = ~44MB resident

System-Wide /proc Entries

/proc/meminfo -- System Memory

$ cat /proc/meminfo | head -15
MemTotal:       16384000 kB
MemFree:         2048000 kB
MemAvailable:    8192000 kB
Buffers:          512000 kB
Cached:          5120000 kB
SwapCached:        12000 kB
Active:          7200000 kB
Inactive:        5800000 kB
SwapTotal:       4096000 kB
SwapFree:        4000000 kB
Dirty:             25000 kB
Writeback:             0 kB
AnonPages:       6100000 kB
Mapped:          1200000 kB
Shmem:            300000 kB

Key distinction: MemFree is truly unused RAM. MemAvailable is what the kernel estimates can be given to applications (includes reclaimable caches and buffers). Use MemAvailable for capacity planning.

Gotcha: MemAvailable was added in Linux 3.14 (2014). On older kernels, it does not exist, and tools that approximate it (like free) use MemFree + Buffers + Cached, which overestimates available memory because not all cached pages are reclaimable. If your monitoring shows "90% memory used" but the system is fine, you are probably looking at MemFree instead of MemAvailable.

/proc/cpuinfo -- CPU Details

# Count physical CPUs, cores, and threads
$ grep "physical id" /proc/cpuinfo | sort -u | wc -l    # sockets
2
$ grep "cpu cores" /proc/cpuinfo | head -1               # cores per socket
cpu cores       : 8
$ grep -c "^processor" /proc/cpuinfo                      # total threads
32

# Check for CPU vulnerabilities (Spectre, Meltdown, etc.)
$ grep . /proc/cpuinfo | grep -i bug
bugs            : spectre_v1 spectre_v2 spec_store_bypass mds

/proc/loadavg -- System Load

$ cat /proc/loadavg
2.15 1.87 1.42 3/456 12345
# 1-min  5-min  15-min  running/total  last-PID

# Rule of thumb: load average > number of CPUs means processes are waiting

/proc/uptime

$ cat /proc/uptime
1234567.89 9876543.21
# seconds since boot    total idle CPU-seconds across all cores

/proc/mounts -- Mounted Filesystems

# Same information as 'mount' command but reads directly from kernel
$ cat /proc/mounts | grep -v "^proc\|^sys\|^dev\|^cgroup"
/dev/sda1 / ext4 rw,relatime 0 0
/dev/sdb1 /data xfs rw,noatime 0 0

/proc/net/ -- Network Statistics

# Active TCP connections (hex format -- use ss or netstat instead for readability)
$ cat /proc/net/tcp | head -3
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid
   0: 00000000:1F90 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0
   1: 0100007F:0CEA 0100007F:1F90 01 00000000:00000000 00:00000000 00000000  1000

# Network device statistics
$ cat /proc/net/dev
Inter-|   Receive                                                |  Transmit
 face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop
  eth0: 1234567890 9876543    0    0    0     0          0     0 987654321  876543    0    0
    lo: 456789012  345678    0    0    0     0          0     0 456789012  345678    0    0

# Socket statistics summary
$ cat /proc/net/sockstat
sockets: used 456
TCP: inuse 123 orphan 2 tw 45 alloc 200 mem 67
UDP: inuse 12 mem 5

/proc/diskstats -- Block Device I/O

$ cat /proc/diskstats | grep sda
   8       0 sda 123456 7890 9876543 45678 234567 8901 8765432 56789 0 34567 102467
# Fields: major minor name reads_completed reads_merged sectors_read ms_reading
#         writes_completed writes_merged sectors_written ms_writing
#         ios_in_progress ms_io weighted_ms_io

/proc/interrupts -- Hardware Interrupts

$ cat /proc/interrupts | head -5
           CPU0       CPU1       CPU2       CPU3
  0:         45          0          0          0   IO-APIC   2-edge      timer
  1:          0          0          3          0   IO-APIC   1-edge      i8042
  8:          0          0          0          1   IO-APIC   8-edge      rtc0
 16:    1234567          0          0          0   IO-APIC  16-fasteoi   ehci_hcd

# Useful for checking IRQ affinity and interrupt storms

Writing to /proc/sys -- Live Kernel Tuning

The /proc/sys/ tree contains kernel parameters that can be read and written at runtime. This is the same mechanism as sysctl.

# Read a value
$ cat /proc/sys/net/ipv4/ip_forward
0

# Write a value (takes effect immediately, lost on reboot)
$ echo 1 > /proc/sys/net/ipv4/ip_forward

# Equivalent sysctl commands
$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0
$ sysctl -w net.ipv4.ip_forward=1

# Common tunables:
$ cat /proc/sys/vm/swappiness                      # swap aggressiveness (0-100)
$ cat /proc/sys/fs/file-max                        # system-wide file descriptor limit
$ cat /proc/sys/net/core/somaxconn                 # TCP listen backlog max
$ cat /proc/sys/net/ipv4/tcp_max_syn_backlog       # SYN queue size
$ cat /proc/sys/net/ipv4/tcp_fin_timeout           # TIME_WAIT duration
$ cat /proc/sys/kernel/pid_max                     # maximum PID value
$ cat /proc/sys/vm/overcommit_memory               # memory overcommit policy (0,1,2)

# To persist across reboots, add to /etc/sysctl.conf or /etc/sysctl.d/*.conf

# Default trap: sysctl changes via /proc/sys are immediate but lost on reboot.
# The #1 production surprise: you tuned a parameter, rebooted for a kernel update,
# and the tuning vanished because nobody added it to sysctl.d/.
$ echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.d/99-custom.conf
$ sysctl --system   # reload all sysctl configs

Wiki Navigation

Prerequisites

Related Content