Skip to content

Solution: Firmware Update Boot Loop

Triage

  1. Connect to iLO/BMC remote console immediately. Observe the POST cycle and note:
  2. Does POST complete fully? Any error messages?
  3. Is the boot disk visible in the POST device list?
  4. What boot mode is displayed (UEFI / Legacy)?

  5. Review the BMC Integrated Management Log (IML):

    iLO web UI -> Information -> Integrated Management Log
    
    Look for: firmware flash status, memory training errors, boot device missing.

  6. Check if the BIOS settings were reset to defaults. Many firmware updates reset all BIOS configuration. The most common cause of this symptom is boot mode changed from UEFI to Legacy (or vice versa) after a settings reset.

Root Cause

The BIOS update reset settings to factory defaults. The server was installed with UEFI boot mode, but factory default is Legacy BIOS. With the wrong boot mode, the firmware cannot locate the UEFI boot partition (EFI System Partition), so it cycles through all boot devices and restarts.

This is confirmed by the serial console showing "Discovering boot devices..." with no UEFI boot entries listed.

Fix

  1. Enter BIOS Setup during POST (F9 on HPE):
  2. Navigate to Boot Options
  3. Change Boot Mode from "Legacy BIOS" to "UEFI Mode"
  4. Save and exit

  5. If boot order was also reset, re-add the correct boot entry:

  6. In UEFI Boot Order, ensure the OS disk (e.g., RAID logical drive or NVMe) is first
  7. Remove PXE/network boot from first position if it was moved there

  8. If Secure Boot keys were cleared:

  9. Re-enroll default Secure Boot keys (Reset to manufacturer defaults in Secure Boot config)
  10. Or temporarily disable Secure Boot to get the OS booting, then re-enable

  11. If BIOS settings are complex (custom power profiles, SR-IOV, etc.):

  12. Restore from the pre-update BIOS configuration backup via iLO:

    iLO -> Administration -> Firmware -> BIOS Settings Restore
    

  13. After the OS boots, verify all services are running and the system is healthy:

    systemctl --failed
    dmesg | grep -i error
    

Rollback / Safety

  • HPE servers have a redundant ROM feature. If the new BIOS is actually corrupted:

    iLO -> Administration -> Firmware -> Select "Switch to backup ROM"
    
    This reverts to the pre-update BIOS image stored in the backup flash chip.

  • If the OS disk is suspected damaged, boot from iLO virtual media (mount an ISO) and inspect the disk.

  • Before making BIOS changes, screenshot/export the current (default) settings so you can compare against the desired configuration.

Common Traps

  • Trap: Assuming the firmware flash failed or is corrupted. In most cases, the flash succeeded -- it just reset settings to defaults.
  • Trap: Not exporting BIOS settings before the update. Always use conrep (HPE) or syscfg / BIOS config backup before flashing.
  • Trap: Flashing firmware without checking compatibility matrices. BIOS updates sometimes require matching CPLD, ME, or RAID controller firmware versions.
  • Trap: Panicking and re-flashing the BIOS again without checking settings first, which wastes time and adds risk.
  • Trap: Not testing the boot mode change on a single server before rolling out to a fleet.