--- tags: - linux - l1 - flashcard-deck - linux-kernel --- [Portal](../../../../library/portal/index.md) | **Level:** [L1: Foundations](../../../../library/portal/levels.md) | **Topics:** [Linux Kernel Tuning](../../../../library/portal/topics.md) | **Domain:** Linux id category difficulty tags question answer source_path linux-kernel/02e76845737d linux-kernel medium bash, kernel, linux, shell, syscalls Are wildcards implemented in user or kernel space? User space - specifically in the shell.\n\nHow it works:\n1. You type: ls *.txt\n2. Shell (bash) sees the wildcard\n3. Shell expands *.txt to matching files\n4. Shell calls exec with expanded list\n5. ls receives: ls file1.txt file2.txt file3.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/426-wildcards-are-implemented-on-user-or-kernel-space.txt linux-kernel/03a4b795c1a1 linux-kernel easy distributions, fundamentals, linux, memory-management What is Linux and what distinguishes it from other operating systems? "[Wikipedia](https://en.wikipedia.org/wiki/Linux): ""Linux is a family of open-source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991, by Linus Torvalds. Linux is typically packaged in a Linux distribution.""\n\n[Red Hat](https://www.redhat.com/en/topics/linux/what-is-linux): ""Linux® is an open source operating system (OS)."\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/130-what-is-linux.txt linux-kernel/08c31c94051f linux-kernel hard fundamentals, linux, system What is the use of ulimit in Unix-like systems? "Most Unix-like operating systems, including Linux and BSD, provide ways to limit and control the usage of system resources such as threads, files, and network connections on a per-process and per-user basis. These ""**ulimits**"" prevent single users from using too many system resources."\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/063-what-is-the-use-of-ulimit-in-unix-like-systems.txt linux-kernel/0cfbf9cd53f2 linux-kernel medium linux, namespaces, tuning Are the changes you make to kernel parameters in a container, affects also the kernel parameters of the host on which the container runs? No. Containers have their own /proc filesystem so any change to kernel parameters inside a container, are not affecting the host or other containers running on that host.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/196-are-the-changes-you-make-to-kernel-parameters-in-a.txt linux-kernel/11156bbd5110 linux-kernel hard linux, troubleshooting, debugging, kernel, sysrq "If a server is ""unresponsive"" but still pings, what do you suspect?" The kernel's network stack is alive, but userspace is frozen or resource-starved.\n\nLikely causes:\n- OOM (Out of Memory): Kernel killing processes, system thrashing\n- CPU lockup: Runaway process or kernel bug consuming all CPU\n- Kernel oops/soft lockup: Partial kernel failure, userspace frozen\n- Resource exhaustion: PID limit, file descriptor limit, memory\n- D state processes: Mass I/O wait\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/533-unresponsive-server-pings-diagnosis.txt linux-kernel/111aa6d2b573 linux-kernel medium boot, fundamentals, linux, tuning In what phases of kernel lifecycle, can you change its configuration? * Build time (when it's compiled)\n * Boot time (when it starts)\n * Runtime (once it's already running)\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/188-in-what-phases-of-kernel-lifecycle-can-you-change-.txt linux-kernel/1c0c5ed29e61 linux-kernel easy linux, memory-management, scheduling What is a kernel, and what does it do? The kernel is part of the operating system and is responsible for tasks like:\n\n * Allocating memory\n * Schedule processes\n * Control CPU\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/184-what-is-a-kernel-and-what-does-it-do.txt linux-kernel/1c93dd74dd2e linux-kernel medium fundamentals, linux, memory-management What virtualization solutions are available for Linux? * [KVM](https://www.linux-kvm.org/page/Main_Page)\n * [XEN](http://www.xen.org/)\n * [VirtualBox](https://www.virtualbox.org/)\n * [Linux-VServer](http://linux-vserver.org/Welcome_to_Linux-VServer.org)\n * [User-mode Linux](http://user-mode-linux.sourceforge.net/)\n * ...\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/286-what-virtualization-solutions-are-available-for-li.txt linux-kernel/2024ef004883 linux-kernel easy linux, kernel, grub, boot How do you pass kernel boot parameters? Temporarily: edit GRUB (`e`) and modify `linux` line.\n\nPersistently: `/etc/default/grub` → `GRUB_CMDLINE_LINUX` → regenerate config.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/550-kernel-boot-parameters.txt linux-kernel/221511d60c2d linux-kernel hard boot, fundamentals, linux, system How do you debug boot failures remotely? Without physical access, you need out-of-band management:\n\n1. **Serial/IPMI console**: Connect via iLO, iDRAC, IPMI for console access during boot\n\n2. **Recovery options**:\n - Boot to previous kernel via GRUB\n - Drop to initramfs shell (`rd.break` kernel param)\n - Boot to rescue mode (`systemd.unit=rescue.target`)\n\n3.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/453-how-do-you-debug-boot-failures-remotely.txt linux-kernel/2367477f0f21 linux-kernel medium boot, fundamentals, linux Where can you find the file that contains the command passed to the boot loader to run the kernel? `/proc/cmdline`\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/190-where-can-you-find-the-file-that-contains-the-comm.txt linux-kernel/26086d563e3e linux-kernel easy fundamentals, linux, system What are the differences between Unix, Linux, BSD, and GNU? **GNU** isn't really an OS. It's more of a set of rules or philosophies that govern free software, that at the same time gave birth to a bunch of tools while trying to create an OS. So **GNU** tools are basically open versions of tools that already existed, but were reimplemented to conform to principals of open software.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/002-what-are-the-differences-between-unix-linux-bsd-an.txt linux-kernel/2e43d95373b3 linux-kernel medium boot, fundamentals, linux Where can you find kernel's configuration? Usually it will reside in `/boot/config-..`\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/189-where-can-you-find-kernels-configuration.txt linux-kernel/35c53aef5eda linux-kernel easy boot, linux, modules, syscalls What is a Linux kernel module and how do you load a new module? A Linux kernel module is a piece of code that can be dynamically loaded into the kernel to extend its functionality. These modules are typically used to add support for hardware devices, filesystems, or system calls. The kernel itself is monolithic, but with modules, its capabilities can be extended without having to reboot the system or recompile the entire kernel.\n\nRemember: lsmod=list, modprobe=load+deps, rmmod=remove, modinfo=details.\n\nExample: `modprobe br_netfilter` — needed for K8s networking. projects/knowledge/interview/linux/186-what-is-a-linux-kernel-module-and-how-do-you-load-.txt linux-kernel/36e1d9ddb397 linux-kernel medium boot, linux, tuning How changes to kernel runtime parameters persist? (applied even after reboot to the system for example) There is a service called `systemd-sysctl` that takes the content of /etc/sysctl.conf and applies it. This is how changes persist, even after reboot, when they are written in /etc/sysctl.conf\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/195-how-changes-to-kernel-runtime-parameters-persist-a.txt linux-kernel/38747af38022 linux-kernel medium distributions, fundamentals, linux What are the components of a Linux distribution? * Kernel\n* Utilities\n* Services\n* Software/Packages Management\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/316-what-are-the-components-of-a-linux-distribution.txt linux-kernel/3fa829ba94de linux-kernel medium fundamentals, linux, tuning How to list kernel's runtime parameters? `sysctl -a` — lists all kernel runtime parameters (from the /proc/sys tree) and their current values. Use `sysctl ` to read one parameter or `sysctl -w =` to change one at runtime.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/191-how-to-list-kernels-runtime-parameters.txt linux-kernel/45225a62663e linux-kernel easy linux, memory-management, namespaces What is User-mode Linux? In Linux, user mode is a restricted operating mode in which a user's application or process runs. User mode is a non-privileged mode that prevents user-level processes from accessing sensitive system resources directly.\n\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/327-what-is-user-mode-linux.txt linux-kernel/4750cd460cc9 linux-kernel medium fundamentals, linux Under which license Linux is distributed? GPL v2 (GNU General Public License version 2). This means anyone can use, modify, and distribute the kernel, but modifications must also be released under GPL v2.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/328-under-which-license-linux-is-distributed.txt linux-kernel/4cf0c9f17b32 linux-kernel medium boot, linux, system, tuning How to change the kernel parameters? What kernel options might you need to tune? To set the kernel parameters in Unix-like, first edit the file `/etc/sysctl.conf` after making the changes save the file and run the command `sysctl -p`, this command will make the changes permanently without rebooting the machine.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/051-how-to-change-the-kernel-parameters-what-kernel-op.txt linux-kernel/4efb14e595ae linux-kernel hard linux, kernel, debugging, panic, crash-analysis "What is a ""Kernel Panic"" and how do you debug it post-mortem?" A kernel panic is a fatal kernel error that forces the system to halt.\n\nWhat causes kernel panics:\n- Critical hardware failures\n- Kernel bugs or driver issues\n- Memory corruption\n- Null pointer dereference in kernel code\n- Unrecoverable filesystem errors\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/527-kernel-panic-postmortem.txt linux-kernel/54e7d8bdbc2a linux-kernel medium boot, distributions, linux, namespaces What Linux distributions are you familiar with? Major distribution families:\n\nRed Hat family:\n- RHEL (enterprise, paid support)\n- CentOS/Rocky/Alma (RHEL clones)\n- Fedora (cutting edge)\n- Uses: rpm, dnf/yum, systemd\n\nDebian family:\n- Debian (stable, slow)\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/422-what-linux-distributions-are-you-familiar-with.txt linux-kernel/58c6afeb91a9 linux-kernel hard linux, kernel, production, deployment, safety How do you safely update the kernel in production with minimal risk? Use a staged rollout approach with multiple safety mechanisms.\n\nDeployment strategy:\n1. Test on non-production environment first\n2. Canary deployment to small subset of prod (1-5%)\n3. Monitor for issues, gradually expand rollout\n4.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/515-safe-kernel-update-production.txt linux-kernel/5ba1aef3b118 linux-kernel easy distributions, fundamentals, linux What is a Linux distribution? * A collection of packages - kernel, GNU, third party apps, ...\n* Sometimes distributions store some information on the distribution in `/etc/*-release` file\n * For example for Red Hat distribution it will be `/etc/redhat-release` and for Amazon it will be `/etc/os-release`\n * `lsb_release` is a common command you can use in multiple different distributions\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/318-what-is-a-linux-distribution.txt linux-kernel/616191a4e80f linux-kernel medium linux, memory-management, syscalls Explain user space vs. kernel space "The operating system executes the kernel in protected memory to prevent anyone from changing (and risking it crashing). This is what is known as ""Kernel space"".\n""User space"" is where users executes their commands or applications. It's important to create this separation since we can't rely on user applications to not tamper with the kernel, causing it to crash.\n\nApplications can access system resources and indirectly the kernel space by making what is called ""system calls""."\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/187-explain-user-space-vs-kernel-space.txt linux-kernel/64f1c09bf2d8 linux-kernel hard boot, linux, modules, system `grub>` vs `grub-rescue>`. Explain. - `grub>` - this is the mode to which it passes if you find everything you need to run the system in addition to the configuration file. With this mode, we have access to most (if not all) modules and commands.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/120-codegrubcode-vs-codegrub-rescuecode-explain.txt linux-kernel/65807ceff2dd linux-kernel hard linux, system Describe start-up configuration files and directory in BSD systems. In BSD the primary start-up configuration file is `/etc/defaults/rc.conf`. System startup scripts such as `/etc/rc` and `/etc/rc.d` just include this file.\n\nIf you want to add other programs to system startup you need to change `/etc/rc.conf` file instead of `/etc/defaults/rc.conf`.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/085-describe-start-up-configuration-files-and-director.txt linux-kernel/71881ffd6ad4 linux-kernel medium boot, distributions, linux True or False? both /tmp and /var/tmp cleared upon system boot False. /tmp is cleared upon system boot while /var/tmp is cleared every a couple of days or not cleared at all (depends on distro).\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/217-true-or-false-both-tmp-and-vartmp-cleared-upon-sys.txt linux-kernel/79ada490c145 linux-kernel medium fundamentals, linux How do you find out which Kernel version your system is using? `uname -a` command\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/185-how-do-you-find-out-which-kernel-version-your-syst.txt linux-kernel/7c6eefa0beca linux-kernel medium linux, kernel, syscalls What is a system call? What system calls are you familiar with? System calls are the interface between user programs and the kernel.\n\nPurpose:\n- Request kernel services\n- Access hardware resources\n- Perform privileged operations\n\nCommon system calls:\n- File: open, read, write, close, stat\n- Process: fork, exec, exit, wait, kill\n\nRemember: Syscalls = user↔kernel interface. `strace` traces them. Common: open, read, write, fork. projects/knowledge/interview/linux/404-what-is-a-system-call-what-system-calls-are-you-fa.txt linux-kernel/86a299f15a82 linux-kernel easy linux, boot, grub What is GRUB2 and what role does it play in the Linux boot process? GRUB2 (GRand Unified Bootloader 2) is the default Linux bootloader.\n\nFunctions:\n- Loads the kernel and initramfs\n- Provides boot menu for multiple OS/kernels\n- Supports various filesystems\n- Chain-loading other bootloaders\n\nKey files:\n- /boot/grub/grub.cfg - Generated config (don't edit)\n- /etc/default/grub - User settings\n- /etc/grub.d/ - Menu entry scripts\n\nCommands:\n- grub-install - Install bootloader\n- grub-mkconfig - Regenerate config\n- update-grub - Debian/Ubuntu wrapper\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/344-what-is-grub2.txt linux-kernel/96ade4407f39 linux-kernel easy linux, kernel, fundamentals What is the Linux kernel? The Linux kernel is a free, open-source Unix-like kernel – the core of the operating system – which manages hardware and system resources.\n\nFun fact: Linus Torvalds released it on August 25, 1991: 'I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu).'\n\nUnder the hood: the kernel manages CPU scheduling, memory, device drivers, filesystems, and networking. Runs in ring 0 (privileged mode).\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/554-what-is-linux-kernel.txt linux-kernel/98c9b5142dc3 linux-kernel medium boot, fundamentals, linux, system How to permanently set `$PATH` on Linux/Unix? Why is this variable so important? Your console has two types of messages:\n\n- **generated by the kernel** (via printk)\n- **generated by userspace** (usually your init system)\n\nKernel messages are always stored in the **kmsg** buffer, visible via `dmesg` command. They're also often copied to your **syslog**. This also applies to userspace messages written to `/dev/kmsg`, but those are fairly rare.\n\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/041-how-to-permanently-set-codepathcode-on-linuxunix-w.txt linux-kernel/a20d96a01188 linux-kernel medium linux, syscalls, tuning How sysctl applies the changes to kernel's runtime parameters the moment you run sysctl command? If you `strace` the sysctl command you can see it does it by changing the file under /proc/sys/...\n\nIn the past it was done with sysctl system call, but it was deprecated at some point.\n\nExample: `sysctl -w net.ipv4.ip_forward=1` — runtime. Persist: `/etc/sysctl.d/99-custom.conf`.\n\nGotcha: `sysctl -w` is temporary. Always persist and run `sysctl --system`. projects/knowledge/interview/linux/194-how-sysctl-applies-the-changes-to-kernels-runtime-.txt linux-kernel/a65456a1bce9 linux-kernel easy distributions, fundamentals, linux, system Give some examples of Linux distribution. What is your favorite distro and why? - Red Hat Enterprise Linux\n- Fedora\n- CentOS\n- Debian\n- Ubuntu\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/001-give-some-examples-of-linux-distribution-what-is-y.txt linux-kernel/b5d43b9615a9 linux-kernel medium linux, udev, devices, kernel If I plug a new device into a Linux machine, where and how does the detection process start? Device detection involves kernel, udev, and hardware:\n\n1. Hardware detection:\n - Kernel driver detects device (USB, PCI, etc.)\n - Creates device in kernel data structures\n\n2. Kernel notification:\n - Kernel creates kobject in /sys\n - Sends uevent to userspace\n\n3. udev processing:\n - udevd receives uevent\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/427-if-i-plug-a-new-device-into-a-linux-machine-where-.txt linux-kernel/c576aa7f2a8b linux-kernel medium linux, tuning Will running sysctl -a as a regular user vs. root, produce different result? Yes, you might notice that in most systems, when running `systctl -a` with root, you'll get more runtime parameters compared to executing the same command with a regular user.\n\nExample: `sysctl -w net.ipv4.ip_forward=1` — runtime. Persist: `/etc/sysctl.d/99-custom.conf`.\n\nGotcha: `sysctl -w` is temporary. Always persist and run `sysctl --system`. projects/knowledge/interview/linux/192-will-running-sysctl-a-as-a-regular-user-vs-root-pr.txt linux-kernel/c7c206c1ad24 linux-kernel hard linux, kernel, troubleshooting, sysrq, debugging A server suddenly becomes unresponsive but you can still ping it. How do you debug without rebooting? Use Magic SysRq keys - they communicate directly with the kernel, bypassing userspace.\n\nEnable SysRq if not already:\n- `echo 1 > /proc/sys/kernel/sysrq`\n\nUseful SysRq triggers:\n- `echo t > /proc/sysrq-trigger` - show task states (stack traces)\n- `echo w > /proc/sysrq-trigger` - show blocked (D state) tasks\n- `echo m > /proc/sysrq-trigger` - show memory info\n- `echo l > /proc/sysrq-trigger\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/511-debug-unresponsive-server-still-pings.txt linux-kernel/e2fd96183a10 linux-kernel hard boot, linux, memory-management, system How does Linux boot, end to end? Firmware → bootloader (GRUB) → kernel → initramfs → kernel mounts root FS → `systemd` PID 1 → targets/services. Most boot issues live in initramfs, fstab, or broken units.\n\nThe detailed sequence:\n1. **BIOS/UEFI**: Hardware initialization, POST, loads bootloader\n2. **GRUB**: Loads kernel and initramfs into memory\n3.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/446-how-does-linux-boot-end-to-end.txt linux-kernel/e67d84e8bff3 linux-kernel hard linux, memory-management, modules, syscalls, system What are the different types of kernels? Explain. **Monolithic Kernels**\n\nEarlier in this type of kernel architecture, all the basic system services like a process and memory management, interrupt handling etc were packaged into a single module in kernel space.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/082-what-are-the-different-types-of-kernels-explain.txt linux-kernel/e7996df05e74 linux-kernel medium boot, linux How to make sure a Service starts automatically after a reboot or crash? Depends on the init system.\n\nSystemd: ` systemctl enable [service_name] `\nSystem V: ` update-rc.d [service_name] ` and add this line ` id:5678:respawn:/bin/sh /path/to/app ` to /etc/inittab\nUpstart: add Upstart init script at /etc/init/service.conf\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/267-how-to-make-sure-a-service-starts-automatically-af.txt linux-kernel/e7b5d85cfb84 linux-kernel hard linux, kernel, debugging, kdump, crash-analysis You need to debug a kernel panic that only happens under heavy load. What do you configure? Set up kdump to capture kernel crash dumps for post-mortem analysis.\n\nConfiguration steps:\n1. Install kdump: `yum install kexec-tools` or `apt install kdump-tools`\n2. Configure GRUB - add to kernel cmdline:\n - `crashkernel=auto` (or specific size like `crashkernel=256M`)\n - Edit /etc/default/grub, run grub2-mkconfig\n3. Configure /etc/kdump.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/522-kernel-panic-under-load-debugging.txt linux-kernel/e8ae36028799 linux-kernel easy linux, distribution, fundamentals What is a Linux distribution (distro)? A complete operating system built around the Linux kernel, bundling system libraries, tools, and often a package manager (examples: Debian, Red Hat).\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/555-what-is-linux-distribution.txt linux-kernel/fca5e9a413c5 linux-kernel easy linux, text-processing What is Libvirt and how does it manage virtual machines? It's an open source collection of software used to manage virtual machines. It can be used with: KVM, Xen, LXC and others. It's also called Libvirt Virtualization API.\n\nFrom the official [docs](https://libvirt.org/)\nHypervisor supported [docs](https://libvirt.org/drivers.html)\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/288-what-is-libvirt.txt linux-kernel/fcdda4ec3a47 linux-kernel easy distributions, fundamentals, linux, memory-management What is KVM (Kernel-based Virtual Machine) and how does it work? Is an open source virtualization technology used to operate on x86 hardware. \n\nFrom the official [docs](https://www.linux-kvm.org/page/Main_Page)\nRecommended read:\n * [Red Hat Article - What is KVM?](https://www.redhat.com/en/topics/virtualization/what-is-KVM)\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/287-what-is-kvm.txt linux-kernel/650d3cbd72e6 linux-kernel medium miscellaneous, boot, control-flow, functions How do you perform server OS installation and configuration? Server OS installation and configuration involve the following steps: **Prepare Installation Media:* • Create a bootable installation media, such as a USB drive or DVD, containing the server OS. **Boot from Installation Media:* • • Insert the installation media into the server. • Boot the server from the installation media. **Follow Installation Wizard:* • Follow the prompts of the installation wizard, which typically involves selecting language, time zone, and keyboard layout. **Partitioning and Disk Setup:* • • Choose the disk or partition where the OS will be installed. linux-kernel/a1b2c3d4e5f6 linux-kernel easy kernel-troubleshooting, dmesg, logs How do you view kernel messages with human-readable timestamps and filter for errors? Use dmesg -T for human-readable timestamps. Filter for errors: dmesg -l err,crit,alert,emerg. Combine: dmesg -T -l err,crit,alert,emerg. Use dmesg -w to follow new messages in real-time. This is your first diagnostic command for system-level issues.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/b2c3d4e5f6a7 linux-kernel easy kernel-troubleshooting, dmesg, patterns What patterns should you grep for in dmesg when troubleshooting hardware or system issues? Hardware: "error|fault|fail|warn". Memory: "oom|out of memory|page allocation failure". Disk: "i/o error|medium error|sector|ata|scsi". Network: "link down|link up|carrier|dropped|reset". CPU: "mce|machine check|thermal|throttl". Filesystem: "ext4|xfs|corrupt|mount|remount".\n\nRemember: dmesg = kernel ring buffer. Hardware, drivers, errors. `dmesg -T` for timestamps.\n\nExample: `dmesg | grep -i error` — quick kernel error scan. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/c3d4e5f6a7b8 linux-kernel easy kernel-troubleshooting, oops-vs-panic, basics What is the difference between a kernel oops and a kernel panic? An oops is a kernel bug that kills the offending process but the system usually continues running (degraded, marked tainted). A panic is fatal -- the kernel cannot continue and the system halts or reboots. An oops can escalate to a panic if panic_on_oops=1 is set.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/d4e5f6a7b8c9 linux-kernel medium kernel-troubleshooting, kdump, crash-dumps What is kdump and how does it capture crash dumps during a kernel panic? kdump reserves a small amount of memory at boot for a second (crash) kernel. During a panic, the crash kernel activates and writes the contents of memory (vmcore) to disk at /var/crash/. Setup: install kexec-tools, enable kdump service, ensure crashkernel=256M is in the kernel command line. Without kdump, crash forensic evidence is lost.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/e5f6a7b8c9d0 linux-kernel medium kernel-troubleshooting, sysrq, reisub What is the SysRq REISUB sequence, and when would you use it? REISUB is a safe emergency reboot sequence when the system is hung: R (un-Raw keyboard), E (tErminate all, SIGTERM), I (kIll all, SIGKILL), S (Sync disks), U (Unmount/remount read-only), B (reBoot). Wait 2-5 seconds between each key. This is the cleanest reboot when nothing else works. Enable with kernel.sysrq=1 in /etc/sysctl.d/.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/f6a7b8c9d0e1 linux-kernel medium kernel-troubleshooting, tainted, modules What does it mean when a kernel is "tainted," and why does it matter? A tainted kernel has been modified from its pristine state. Common taint flags: P (proprietary module like nvidia), F (module force-loaded), W (warning/oops occurred), E (unsigned module). Check with cat /proc/sys/kernel/tainted (0=clean). Tainted kernels may affect vendor support and bug report acceptance.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/a7b8c9d0e1f2 linux-kernel medium kernel-troubleshooting, oom-killer, memory How does the OOM killer work, and how do you investigate OOM kill events? When the system runs out of memory, the OOM killer selects and kills a process to free memory. Check for OOM kills: dmesg | grep -i "out of memory\|oom-killer\|killed process". Adjust OOM priority: echo -1000 > /proc//oom_score_adj (never kill) or echo 1000 (kill first). The OOM killer is a symptom -- fix the memory pressure, do not just restart the killed process.\n\nRemember: OOM killer picks by oom_score. Protect: `echo -1000 > /proc//oom_score_adj`.\n\nUnder the hood: Activates when RAM+swap exhausted. Kills highest score to free most memory. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/b8c9d0e1f2a3 linux-kernel medium kernel-troubleshooting, sysrq, debugging What SysRq keys help debug hung systems without rebooting? t = dump task states (debug hung processes), m = dump memory info (debug memory issues), w = dump blocked D-state tasks (debug I/O hangs), s = sync all filesystems, e = send SIGTERM to all processes. Access via Alt+SysRq+ on console, or echo > /proc/sysrq-trigger remotely. These work even when the system is mostly unresponsive.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/c9d0e1f2a3b4 linux-kernel hard kernel-troubleshooting, crash, backtrace How do you analyze a kernel crash dump using the crash utility, and how do you read a backtrace? Install crash and kernel-debuginfo. Open dump: crash /usr/lib/debug/lib/modules/$(uname -r)/vmlinux /var/crash/*/vmcore. Key commands: bt (backtrace), log (kernel log at crash time), ps (process list), sys (system info). Read backtraces bottom-up: the lowest frame is where the problem started, the root cause is usually in the middle frames.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/d0e1f2a3b4c5 linux-kernel hard kernel-troubleshooting, mce, hardware What are Machine Check Exceptions (MCEs), and how do you diagnose them? MCEs are hardware errors reported by the CPU. Common causes: faulty RAM (diagnose with memtest86+), overheating CPU (check thermal sensors), or failing CPU (needs replacement). Check with dmesg | grep -i "machine check\|mce". Install mcelog for detailed analysis. MCEs indicate real hardware problems that cannot be fixed with software.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/e1f2a3b4c5d6 linux-kernel hard kernel-troubleshooting, kdump, configuration How do you configure kdump for remote crash dump storage, and what does the dump level (-d flag) control? Configure in /etc/kdump.conf. For NFS: nfs server:/crash-dumps. For SSH: ssh user@server with sshkey path. The core_collector makedumpfile -d 31 flag controls what to exclude from the dump: 1=zero pages, 2=cache pages, 4=cache private, 8=user pages, 16=free pages. -d 31 excludes all (smallest dump). Lower values produce larger but more complete dumps.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. training/library/topics/kernel-troubleshooting/primer.md linux-kernel/13bd316200bd linux-kernel easy linux, system What is your favorite shell and why? "**BASH** is my favorite. It’s really a preferential kind of thing, where I love the syntax and it just ""clicks"" for me. The input/output redirection syntax (`>>`, `<< 2>&1`, `2>`, `1>`, etc) is similar to C++ which makes it easier for me to recognize.\n\nI also like the **ZSH** shell, because is much more customizable than **BASH**. It has the Oh-My-Zsh framework, powerful context based tab completion, pattern matching/globbing on steroids, loadable modules and more."\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/004-what-is-your-favorite-shell-and-why.txt linux-kernel/68ec08715642 linux-kernel medium linux, kernel How to generate a random string? One way is to run the following: `cat /proc/sys/kernel/random/uuid`\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/169-how-to-generate-a-random-string.txt linux-kernel/9d0a747411da linux-kernel hard linux, system How to log all commands run by root on production servers? `auditd` is the correct tool for the job here:\n\n1. Add these 2 lines to `/etc/audit/audit.rules`:\n\n```bash\n-a exit,always -F arch=b64 -F euid=0 -S execve\n-a exit,always -F arch=b32 -F euid=0 -S execve\n```\n\nThese will track all commands run by root (euid=0). Why two rules? The execve syscall must be tracked in both 32 and 64 bit code.\n\n2. To get rid of `auid=4294967295` messages in logs, add `audit=1` to the kernel's cmdline (by editing `/etc/default/grub`)\n\n3. Place the line\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/114-how-to-log-all-commands-run-by-root-on-production-.txt linux-kernel/a2436157ecd7 linux-kernel easy linux, logging, syslog What is stored in each of the following logs: /var/log/messages, /var/log/syslog, /var/log/auth.log? These are key system log files:\n\n/var/log/messages (RHEL/CentOS):\n- General system messages\n- Kernel messages\n- Service start/stop\n- Hardware events\n\n/var/log/syslog (Debian/Ubuntu):\n- Similar to messages\n- General system log\n- Most daemon output\n\n/var/log/auth.log (or secure):\n- Authentication events\n- SSH logins/failures\n- sudo usage\n- PAM messages\n\nWith systemd, also use:\n- journalctl - All logs\n- journalctl -u service - Specific service\n- journalctl -p err - By priority\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/358-what-is-stored-in-each-of-the-following-logs-varlo.txt linux-kernel/b29a11e31cb3 linux-kernel medium linux, kernel True or False? only root can create files in /proc False. No one can create file in /proc directly (certain operations can lead to files being created in /proc by the kernel).\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/156-true-or-false-only-root-can-create-files-in-proc.txt linux-kernel/c962b4f8134a linux-kernel medium linux, logging Where are Linux system logs located, and what are the key log files? /var/log is the standard log directory. Key files: `/var/log/syslog` or `/var/log/messages` (general system logs), `/var/log/auth.log` or `/var/log/secure` (authentication events), `/var/log/kern.log` (kernel messages), `/var/log/dmesg` (boot messages). On systemd systems, use `journalctl` to query the journal. Log rotation is managed by `logrotate`.\n\nRemember: Kernel manages hardware, memory, processes, I/O. User programs interact via syscalls.\n\nUnder the hood: Monolithic kernel with loadable modules. Extend without rebooting. projects/knowledge/interview/linux/176-where-system-logs-are-located.txt --- ## Wiki Navigation ### Related Content - [Linux Kernel Tuning](../../../../library/topics/linux-kernel-tuning/index.md) (Topic Pack, L2) — Linux Kernel Tuning