Skip to content

Portal | Level: L1: Foundations | Topics: IPMI / ipmitool, BMC (Baseboard Management Controller), Out-of-Band Management, Server Hardware | Domain: Datacenter & Hardware

IPMI and ipmitool

Why This Matters

Timeline: IPMI v1.0 was published by Intel in 1998. IPMI v2.0 (with RMCP+ encryption) followed in 2004. Intel effectively abandoned IPMI development after v2.0, and the DMTF's Redfish standard (2015) is its intended successor. Despite being "legacy," IPMI remains on virtually every server shipped today.

IPMI (Intelligent Platform Management Interface) is the vendor-neutral protocol that lets you manage servers when the OS is dead, the network is misconfigured, or the machine won't boot. Every BMC speaks IPMI. Every datacenter operator uses ipmitool. When you understand IPMI at the protocol level -- not just as "that thing iDRAC uses" -- you can troubleshoot any server from any vendor without reaching for a vendor-specific GUI.

This topic owns the protocol, the CLI, and the hardware management plane. Vendor-specific tools (iDRAC/RACADM, iLO, Redfish) are covered in their own topics. Provisioning workflows (PXE, kickstart) live in the OOB and Provisioning topic.

Scope

  • IPMI protocol: Architecture, message types, transport, authentication
  • ipmitool CLI: The universal interface to any BMC
  • Sensor Data Repository (SDR): How sensors work, thresholds, reading them
  • System Event Log (SEL): Hardware event history, interpretation, lifecycle
  • Serial-over-LAN (SOL): Remote console via IPMI
  • BMC management: Network config, users, firmware, cold reset
  • Linux IPMI kernel interface: In-band access via kernel modules
  • Security: Protocol vulnerabilities, hardening, credential management

The Mental Model

Every modern server has two independent computers sharing one chassis:

┌─────────────────────────────────────────────┐
│  Main System                                │
│  ┌───────┐  ┌───────┐  ┌──────┐  ┌──────┐  │
│  │ CPU   │  │ RAM   │  │ NIC  │  │ Disk │  │
│  └───┬───┘  └───┬───┘  └──┬───┘  └──┬───┘  │
│      │          │         │         │       │
│  ════╪══════════╪═════════╪═════════╪═══    │
│      │    System Bus / PCIe              │  │
│  ════╪══════════════════════════════════    │
│      │                                      │
│  ┌───┴──────────────────────────────────┐   │
│  │  BMC (Baseboard Management Controller)│  │
│  │  - Own CPU (ARM)                      │  │
│  │  - Own RAM (256MB-1GB)                │  │
│  │  - Own NIC (dedicated or shared)      │  │
│  │  - Own flash (firmware, config, SEL)  │  │
│  │  - Always on (standby power)          │  │
│  └───────────────────────────────────────┘  │
└─────────────────────────────────────────────┘

The BMC runs its own OS (typically Linux-based), has its own IP address, and is always powered on as long as the server has AC power -- even when the main system is completely off. It communicates with the main system over IPMB (a low-speed I2C/SMBus) and with operators over the network via IPMI-over-LAN.

IPMI vs Vendor Names

IPMI is the protocol. Vendors build products on top of it:

Vendor BMC Product Extras Beyond IPMI
Dell iDRAC (6/7/8/9) Lifecycle Controller, RACADM, virtual console
HP/HPE iLO (2/3/4/5/6) Active Health System, Federation, virtual media
Supermicro IPMI BMC Web UI, basic virtual media
Lenovo XClarity / IMM REST API, firmware management
Generic / whitebox IPMI BMC Minimal — ipmitool is your primary interface

All of them speak IPMI. ipmitool works with all of them. The vendor UIs add features on top, but IPMI is the common denominator.

IPMI Protocol Architecture

Layers

Application (ipmitool commands)
    │
    ▼
IPMI Message Layer (NetFn + Cmd + Data)
    │
    ▼
Session Layer (authentication, encryption)
    │
    ▼
Transport (RMCP / RMCP+ over UDP 623)
    │
    ▼
Network (IP to BMC interface)

Transport: RMCP and RMCP+

IPMI messages travel over RMCP (Remote Management Control Protocol) on UDP port 623.

  • IPMI 1.5: RMCP with basic authentication (MD5/MD2/none). Passwords sent in weak hashes.
  • IPMI 2.0: RMCP+ with RAKP (Remote Authenticated Key Exchange Protocol). Supports AES-128-CBC encryption and HMAC-SHA1/SHA256 integrity. This is what lanplus means in ipmitool.
# IPMI 1.5 (insecure, legacy)
ipmitool -I lan -H <bmc-ip> -U admin -P pass power status

# IPMI 2.0 (use this — always)
ipmitool -I lanplus -H <bmc-ip> -U admin -P pass power status

Message Structure

Every IPMI command has: - NetFn (Network Function): Category of command (Chassis, Sensor, App, Storage, etc.) - Command: Specific operation within the NetFn - Data: Command-specific payload

You don't usually think about this unless you're debugging raw IPMI traffic, but it explains why ipmitool subcommands are grouped the way they are (chassis, sensor, sel, etc.).

Interfaces

ipmitool supports multiple transport interfaces:

Interface Flag When to Use
lanplus -I lanplus Remote access over network (IPMI 2.0)
lan -I lan Remote access (IPMI 1.5 — avoid)
open -I open Local access via /dev/ipmi0 kernel driver
ipmb -I ipmb Direct bus access (rare, embedded systems)

Linux IPMI Kernel Interface

When you run ipmitool locally (without -H), it talks to the BMC through the kernel's IPMI device driver, not over the network.

Kernel Modules

# Load the IPMI driver stack
modprobe ipmi_devintf      # creates /dev/ipmi0
modprobe ipmi_si           # system interface driver (KCS, SMIC, or BT)

# Verify it loaded
lsmod | grep ipmi
# ipmi_devintf          20480  0
# ipmi_si               57344  0
# ipmi_msghandler       53248  2 ipmi_devintf,ipmi_si

# Check the device exists
ls -l /dev/ipmi0
# crw-------. 1 root root 238, 0 Mar 14 10:00 /dev/ipmi0

# If modules don't load, check BIOS
# Some systems need "IPMI over KCS" enabled in BIOS/UEFI settings
dmesg | grep -i ipmi
# ipmi_si: Adding KCS-specified kcs state machine
# ipmi_si: Trying KCS-specified kcs state machine at i/o address 0xca2

System Interface Types

The BMC connects to the host CPU through one of these interfaces:

  • KCS (Keyboard Controller Style): Most common. Uses I/O ports. Slow but universal.
  • SMIC (Server Management Interface Chip): Older, rarely seen.
  • BT (Block Transfer): Faster than KCS, supports larger messages. Used by newer systems.

The ipmi_si driver auto-detects which interface is available.

In-Band vs Over-LAN

# In-band (local, no network, uses /dev/ipmi0)
ipmitool sensor list
ipmitool sel elist

# Over-LAN (remote, uses UDP 623)
ipmitool -I lanplus -H 10.0.10.5 -U admin -P secret sensor list

In-band is faster and doesn't require BMC network connectivity. Use it when you're already SSH'd into the server. Over-LAN is for when the OS is unreachable.

Sensor Data Repository (SDR)

The BMC maintains a database of all sensors on the platform: temperature probes, fan tachometers, voltage regulators, power supplies, intrusion switches, and more.

Sensor Types

Type What It Monitors Typical Readings
Temperature CPU, inlet, exhaust, DIMM Degrees C
Fan Fan RPM RPM (0 = dead fan)
Voltage CPU Vcore, 3.3V, 5V, 12V rails Volts
Power Supply PSU presence, status, wattage Watts, discrete states
Current System current draw Amps
Intrusion Chassis cover open/closed Discrete (open/closed)
Processor CPU presence, thermal trip Discrete states
Memory DIMM ECC errors, presence Discrete states

Sensor Thresholds

Each analog sensor has up to six thresholds:

            ┌── Upper Non-Recoverable (UNR)
            │   ┌── Upper Critical (UC)
            │   │   ┌── Upper Non-Critical (UNC)
            │   │   │
Reading ────┤   │   │   Normal operating range
            │   │   │
            │   │   └── Lower Non-Critical (LNC)
            │   └── Lower Critical (LC)
            └── Lower Non-Recoverable (LNR)

When a reading crosses a threshold, the BMC logs an event to the SEL and (if configured) sends an SNMP trap or PET alert.

# View sensor thresholds
ipmitool sensor get "Inlet Temp"
# Sensor ID              : Inlet Temp (0x4)
# Entity ID              : 64.1
# Sensor Type (Analog)   : Temperature
# Sensor Reading          : 23 (+/- 0) degrees C
# Status                  : ok
# Lower Non-Recoverable  : na
# Lower Critical          : 3.000
# Lower Non-Critical      : 8.000
# Upper Non-Critical      : 42.000
# Upper Critical          : 47.000
# Upper Non-Recoverable  : na

# Set a threshold (use with caution)
ipmitool sensor thresh "Inlet Temp" upper 40 45 50
# Sets UNC=40, UC=45, UNR=50

SDR vs Sensor List

# sdr list — compact, shows reading + status
ipmitool sdr list
# Inlet Temp       | 23 degrees C      | ok
# Exhaust Temp     | 35 degrees C      | ok
# Fan1             | 8400 RPM          | ok

# sensor list — verbose, shows thresholds + units
ipmitool sensor list
# Inlet Temp       | 23.000     | degrees C  | ok    | na  | 3.0 | 8.0 | 42.0 | 47.0 | na

# sdr type — filter by sensor type
ipmitool sdr type Temperature
ipmitool sdr type Fan
ipmitool sdr type "Power Supply"

System Event Log (SEL)

The SEL is a circular buffer in the BMC's non-volatile storage that records hardware events: threshold crossings, discrete state changes, boot events, and errors.

Reading the SEL

# List events (human-readable)
ipmitool sel elist
#    1 | 03/14/2024 | 10:15:33 | Temperature #0x04 | Upper Critical going high | Asserted
#    2 | 03/14/2024 | 10:15:33 | Fan #0x30 | State Deasserted | Predictive Failure Asserted
#    3 | 03/14/2024 | 10:20:01 | Power Supply #0x51 | Failure detected | Asserted

# SEL info (capacity, usage)
ipmitool sel info
# Version          : 1.5
# Entries          : 47
# Free Space       : 8832 bytes
# Last Add Time    : 03/14/2024 10:20:01

# Export before clearing (always do this)
ipmitool sel elist > /var/log/sel-$(hostname)-$(date +%F).log

# Clear the SEL (frees space — SEL is finite)
ipmitool sel clear

SEL Capacity

The SEL is small — typically 512 to 2048 entries. When full, behavior depends on the BMC: - Some drop new events (you lose recent data) - Some overwrite oldest events (you lose historical data) - Some stop logging entirely

Rule: Export and clear the SEL regularly. Set up automated collection via cron or monitoring.

Interpreting SEL Events

Common patterns and what they mean:

SEL Event Likely Cause Action
Temperature Upper Critical Cooling failure, blocked airflow Check fans, ambient temp, airflow
Fan Predictive Failure Fan bearing wearing out Schedule fan replacement
Power Supply Failure PSU hardware failure or AC loss Check PDU, replace PSU
Memory Correctable ECC DIMM degrading Monitor frequency, plan replacement
Memory Uncorrectable ECC DIMM failure Replace DIMM immediately
Processor Thermal Trip CPU overheating, fan failure Emergency — check cooling
Drive Fault Disk hardware failure Replace drive, check RAID
Chassis Intrusion Cover opened Verify authorized maintenance

Serial-over-LAN (SOL)

SOL redirects the server's serial console output through the BMC to your terminal. It's your eyes on the screen when the server is 2,000 miles away.

End-to-End Setup

For SOL to work, four things must be configured:

1. BMC SOL must be enabled: 

ipmitool sol set enabled true 1
ipmitool sol set volatile-bit-rate 115.2 1
ipmitool sol set non-volatile-bit-rate 115.2 1

2. BIOS must redirect console to serial port: - In BIOS/UEFI setup: Console Redirection → Serial Port → COM1/COM2 - Redirection type: VT100/VT-UTF8

3. GRUB must output to serial: 

# /etc/default/grub
GRUB_TERMINAL="serial console"
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"
GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0,115200n8"

# Regenerate GRUB config
grub2-mkconfig -o /boot/grub2/grub.cfg    # RHEL/CentOS
update-grub                                 # Debian/Ubuntu

4. Login prompt on serial: 

# systemd (most modern distros)
systemctl enable serial-getty@ttyS0.service
systemctl start serial-getty@ttyS0.service

Using SOL

# Connect
ipmitool -I lanplus -H 10.0.10.5 -U admin -P secret sol activate

# You now see the server's serial console.
# Kernel boot messages, GRUB menu, login prompt — all visible.

# Disconnect
# Type: ~.
# If connected through SSH: ~~.  (escape the SSH escape)

# If SOL session is stuck/stale, deactivate first
ipmitool -I lanplus -H 10.0.10.5 -U admin -P secret sol deactivate
ipmitool -I lanplus -H 10.0.10.5 -U admin -P secret sol activate

SOL vs KVM-over-IP

Feature SOL KVM-over-IP
Output Text only Graphical (full screen)
Bandwidth Very low (serial) High (video stream)
Scriptable Yes (expect, screen) No
Licensing Always free Sometimes requires license (iLO)
Boot visibility POST, GRUB, kernel panic POST, GRUB, kernel panic, GUI
Reliability Very high Sometimes needs Java/HTML5 plugin

Use SOL when: you need to see boot output, capture kernel panics in a log, or automate console interactions. Use KVM when: you need to interact with a graphical installer or BIOS setup screens.

BMC User Management

# List users
ipmitool user list 1
# ID  Name             Callin  Link Auth  IPMI Msg  Channel Priv Limit
# 1                    false   false      false     Unknown (0x00)
# 2   admin            false   false      true      ADMINISTRATOR

# Create a user (ID 3)
ipmitool user set name 3 opsuser
ipmitool user set password 3
# (prompts for password interactively)

# Set privilege level
# 1=Callback, 2=User, 3=Operator, 4=Administrator
ipmitool channel setaccess 1 3 callin=on link=on ipmi=on privilege=4

# Enable the user
ipmitool user enable 3

# Disable a user (e.g., disable default user after creating a replacement)
ipmitool user disable 2

BMC Network Configuration

# View current BMC network config
ipmitool lan print 1
# IP Address Source       : Static Address
# IP Address              : 10.0.10.5
# Subnet Mask             : 255.255.255.0
# Default Gateway IP      : 10.0.10.1
# MAC Address             : 00:11:22:33:44:55

# Set static IP
ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr 10.0.10.5
ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 defgw ipaddr 10.0.10.1

# Set to DHCP
ipmitool lan set 1 ipsrc dhcp

# Set VLAN (management VLAN)
ipmitool lan set 1 vlan id 100

# Disable VLAN tagging
ipmitool lan set 1 vlan id off

BMC Cold Reset and Recovery

# Soft reset the BMC (restarts BMC firmware, does not affect host)
ipmitool mc reset cold

# BMC firmware info
ipmitool mc info
# Device ID                 : 32
# Firmware Revision         : 2.82
# IPMI Version              : 2.0
# Manufacturer ID           : 10876 (Dell)

# BMC self-test
ipmitool mc selftest
# Selftest: passed

# Watchdog timer (BMC auto-reboots host if OS stops responding)
ipmitool mc watchdog get
ipmitool mc watchdog off   # disable

When Cold Reset is Needed

  • BMC web UI is unresponsive but IPMI still works
  • Sensor readings are stale or wrong
  • SOL sessions won't connect
  • After BMC firmware update (some require it)

When Cold Reset Won't Help

  • BMC is completely unresponsive (no IPMI reply)
  • BMC NIC is physically down
  • In these cases: AC power cycle (pull and reseat power cables, wait 30 seconds)

IPMI Security

Known Vulnerabilities

War story: The RAKP hash leak vulnerability is so fundamental to the IPMI 2.0 protocol that it cannot be patched — it is how the authentication handshake works by design. HD Moore (Metasploit creator) and Dan Farmer published extensive research on this in 2013, estimating that over 230,000 IPMI-enabled BMCs were directly exposed to the internet. Network isolation is the only mitigation.

RAKP Authentication Bypass (CVE-2013-4786): IPMI 2.0's RAKP handshake returns a salted hash of the password to any unauthenticated client. An attacker can request this hash and crack it offline.

# An attacker does this:
# 1. Send RMCP+ Open Session Request (no auth needed)
# 2. Receive session ID
# 3. Send RAKP Message 1 with target username
# 4. Receive RAKP Message 2 containing HMAC(password) — crackable offline

# Metasploit module: auxiliary/scanner/ipmi/ipmi_dumphashes
# Hashcat mode: 7300 (IPMI2 RAKP HMAC-SHA1)

This is a protocol design flaw — it cannot be patched. It can only be mitigated by network isolation.

Cipher Suite 0 (no authentication): Some BMCs accept cipher suite 0, which means no authentication at all.

# Check if cipher 0 is enabled (should return error, not success)
ipmitool -I lanplus -H 10.0.10.5 -C 0 -U "" -P "" chassis status
# If this works, your BMC is wide open

# Disable cipher 0 (vendor-specific commands)
# Dell: racadm set iDRAC.IPMILan.CipherSuiteSelections 1,2,3,6,7,8,11,12

Hardening Checklist

  1. Isolate on dedicated management VLAN — no user/workload access to BMC network
  2. Change default credentials — automate during provisioning, never leave root/calvin
  3. Disable IPMI 1.5 (lan interface) — force lanplus (IPMI 2.0) only
  4. Disable cipher suite 0 — verify with the test above
  5. Use strong passwords — RAKP makes password cracking feasible, so use 16+ char passwords
  6. Restrict access via ACLs — limit source IPs that can reach UDP 623
  7. Keep BMC firmware updated — BMC CVEs are frequent and often critical
  8. Monitor BMC access — log authentication events, alert on failures
  9. Disable unused services — turn off web UI, SNMP, SSH on BMC if not needed
  10. Consider Redfish — modern replacement with TLS, token auth, and proper API design

Credential Rotation at Scale

For fleets of servers, manual credential rotation is impossible. Approaches:

# Script pattern: rotate BMC password across fleet
for bmc in $(cat bmc_inventory.txt); do
    ipmitool -I lanplus -H "$bmc" -U admin -P oldpass \
        user set password 2 "$(vault read -field=password secret/bmc/new)"
done

# Better: use Redfish API (supports TLS, token auth)
curl -k -u admin:oldpass \
    -X PATCH https://$bmc/redfish/v1/AccountService/Accounts/2 \
    -H "Content-Type: application/json" \
    -d '{"Password": "newpass"}'

# Best: configuration management (Ansible, Puppet)
# ansible-playbook -i bmc_inventory rotate_bmc_creds.yml

IPMI and Monitoring Integration

Collecting IPMI Data

# Prometheus: use ipmi_exporter
# Exposes sensor readings, SEL events, power consumption as Prometheus metrics

# Nagios/Icinga: check_ipmi_sensor plugin
check_ipmi_sensor -H 10.0.10.5 -U monitor -P pass -T temperature

# Zabbix: built-in IPMI monitoring
# Configure IPMI interfaces per host in Zabbix UI

# Custom: cron + ipmitool + push to TSDB
*/5 * * * * ipmitool -I lanplus -H $BMC -U mon -P pass sdr list \
    | parse_and_push_to_influxdb.sh

DCMI (Data Center Management Interface)

DCMI extends IPMI for datacenter power management:

# Read system power consumption
ipmitool dcmi power reading
# Instantaneous power reading:   287 Watts
# Minimum during sampling period: 245 Watts
# Maximum during sampling period: 342 Watts
# Average power reading over sample period: 278 Watts

# Set power cap (limit maximum power draw)
ipmitool dcmi power set_limit action 1 limit 350
ipmitool dcmi power activate

# Get power cap
ipmitool dcmi power get_limit

Multivendor Notes

Supermicro

  • Default creds: ADMIN/ADMIN
  • Web UI on port 80/443
  • ipmitool is the primary CLI — no vendor CLI like RACADM
  • Fan speed control: ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x64 (raw commands, poorly documented)
  • BMC firmware updates via web UI or socflash tool

Dell (iDRAC)

  • Default creds: root/calvin
  • racadm CLI alongside ipmitool
  • Lifecycle Controller for firmware and config
  • Virtual console via iDRAC web UI (HTML5 on iDRAC9+)
  • ipmitool works for all standard IPMI operations

HP/HPE (iLO)

  • Default creds: Administrator/
  • hponcfg for local BMC config
  • iLO has its own REST API (predates Redfish)
  • Some features require iLO Advanced license
  • ipmitool works for standard IPMI; advanced features need iLO CLI/API

Lenovo (XClarity/IMM)

  • Default creds: USERID/PASSW0RD
  • XClarity Controller (XCC) on ThinkSystem
  • REST API + web UI
  • ipmitool works for standard operations

How to Use This Pack

  1. primer.md (this file): Protocol architecture, component model, security context
  2. street_ops.md: The commands and workflows you use in production
  3. footguns.md: Mistakes that brick BMCs, lock out operators, and miss critical events
  • Redfish: The modern REST-based replacement for IPMI — TLS, JSON, richer data model. Start here for new automation.
  • Datacenter OOB and Provisioning: PXE, DHCP, kickstart — the provisioning pipeline that IPMI feeds into
  • Server Hardware: Physical component diagnostics (DIMM, NIC, disk) — uses ipmitool for sensor data
  • Dell PowerEdge: Dell-specific tooling (iDRAC, RACADM, Lifecycle Controller)
  • Firmware: BIOS/UEFI, BMC firmware lifecycle, update workflows

Wiki Navigation

Prerequisites

Next Steps

Related Content