DHCP & IP Address Management - Street-Level Ops¶
Quick Diagnosis Commands¶
# Release and renew DHCP lease (Linux)
sudo dhclient -r eth0 && sudo dhclient -v eth0
# Show current lease details
cat /var/lib/dhcp/dhclient.eth0.leases | tail -30
# Watch DHCP traffic in real-time
sudo tcpdump -i eth0 -n port 67 or port 68 -vv
# Capture full DHCP conversation to file
sudo tcpdump -i eth0 -n port 67 or port 68 -w /tmp/dhcp.pcap -c 20
# Check current interface configuration from DHCP
ip addr show eth0
cat /etc/resolv.conf
# Find DHCP server on the network (nmap)
sudo nmap --script broadcast-dhcp-discover -e eth0
# Check lease file on ISC DHCP server
cat /var/lib/dhcp/dhcpd.leases | grep -A 6 "lease 10.1.1."
# Check Kea DHCP leases (database query)
sudo kea-admin lease-dump dhcp4 -o /tmp/leases.csv
# Verify DHCP relay is forwarding (from relay device)
# Cisco: show ip dhcp relay statistics
# Linux: journalctl -u isc-dhcp-relay -f
# Test DHCP from a specific interface
sudo dhctest -i eth0
# Find all active IPs on a subnet (when DHCP is suspect)
nmap -sn 10.1.1.0/24 | grep "Nmap scan report"
# Check for IP conflicts
arping -D -I eth0 10.1.1.50 -c 3
# If you get a reply, someone else has that IP
Gotcha: DHCP Discover Goes Unanswered¶
Client broadcasts DISCOVER, nothing comes back. The interface stays at 0.0.0.0.
Diagnosis path:
# Step 1: Verify the client is actually sending
sudo tcpdump -i eth0 -n 'udp port 67 or udp port 68' -c 10
# You should see:
# 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from aa:bb:cc:dd:ee:ff
# Step 2: If sending but no reply, the problem is between client and server
# Check if there is a relay agent on this VLAN
# On the router/L3 switch: show ip helper-address (Cisco)
# Or: show running-config interface VlanXXX | include helper
# Step 3: Check if the server is receiving
# On the DHCP server:
sudo tcpdump -i eth0 -n 'udp port 67' -c 10
# If nothing arrives, the relay is not forwarding
# Step 4: Check DHCP server logs
journalctl -u isc-dhcp-server --since "5 minutes ago"
# Common errors:
# "no free leases" — pool exhausted
# "not authoritative for subnet" — missing subnet declaration
Pattern: IP Conflict Detection and Resolution¶
Two devices have the same IP. One or both experience intermittent connectivity.
# Step 1: Detect the conflict
arping -D -I eth0 10.1.1.50 -c 5
# If duplicate: "Unicast reply from 10.1.1.50 [aa:bb:cc:dd:ee:ff]"
# Step 2: Find both MAC addresses
arp -a | grep 10.1.1.50
# Or from the switch:
# show mac address-table | include aaaa.bbbb.cccc
# Step 3: Identify the devices
# Look up MAC OUI (first 3 octets identify manufacturer)
# https://maclookup.app/ or:
grep -i "aa:bb:cc" /usr/share/nmap/nmap-mac-prefixes
# Step 4: Check DHCP leases for this IP
grep "10.1.1.50" /var/lib/dhcp/dhcpd.leases
# Step 5: Determine which is legitimate
# - Check DHCP reservation: the one with a reservation is correct
# - Check lease timestamps: the newer lease may be a rogue
# - Check if one is statically configured (bypassing DHCP)
# Step 6: Resolve
# If static conflict: change the static device to use DHCP or a different IP
# If DHCP conflict: clear the conflicting lease and restart
sudo dhcpd -t # Test config
sudo systemctl restart isc-dhcp-server
Gotcha: Relay Agent Not Forwarding¶
You configured ip helper-address but DHCP still does not work on remote VLANs.
Common causes:
- Helper points to wrong IP: It should point to the DHCP server's IP, not a VIP or broadcast address
- ACL blocking UDP 67/68: Check firewall rules between relay and server
- Server not listening on the right interface: ISC DHCP only listens on interfaces listed at startup
- Missing subnet declaration: The server needs a
subnetblock for the remote network, even if the pool is empty
# Verify on the server — does it have a subnet for the remote VLAN?
grep -A 5 "subnet 10.1.0" /etc/dhcp/dhcpd.conf
# If missing, add:
# subnet 10.1.0.0 netmask 255.255.255.0 {
# range 10.1.0.11 10.1.0.200;
# option routers 10.1.0.1;
# option domain-name-servers 10.0.0.53;
# }
# Verify server is listening on the correct interface
ss -ulnp | grep :67
# Should show dhcpd or kea-dhcp4 listening
Pattern: DHCP Lease File Inspection¶
The lease file is the source of truth for current assignments:
# ISC DHCP lease file format
cat /var/lib/dhcp/dhcpd.leases
# lease 10.1.1.50 {
# starts 2026/03/15 08:30:00;
# ends 2026/03/16 08:30:00;
# cltt 2026/03/15 08:30:00;
# binding state active;
# next binding state free;
# hardware ethernet aa:bb:cc:dd:ee:ff;
# client-hostname "web-server-01";
# }
# Find all active leases
grep -B1 "binding state active" /var/lib/dhcp/dhcpd.leases | grep "^lease"
# Count active leases per subnet
grep "^lease" /var/lib/dhcp/dhcpd.leases | \
awk -F. '{print $1"."$2"."$3".0"}' | sort | uniq -c | sort -rn
# Find leases expiring soon (within 1 hour)
# Parse lease end times and compare to current time
awk '/^lease/{ip=$2} /ends/{print ip, $2, $3}' /var/lib/dhcp/dhcpd.leases | \
sort -k2,3
Gotcha: Pool Exhaustion Creeping Up Silently¶
DHCP pool runs out of addresses. New clients cannot get IPs. Existing clients renew fine (they already have leases), so nobody notices until a new device tries to join.
# Monitor pool utilization
# ISC DHCP: use dhcpd-pools
dhcpd-pools -c /etc/dhcp/dhcpd.conf -l /var/lib/dhcp/dhcpd.leases
# Output:
# Ranges:
# shared net name first ip last ip max cur percent
# production 10.1.1.11 10.1.1.200 190 172 90.5%
# Alert when above 80%
dhcpd-pools -c /etc/dhcp/dhcpd.conf -l /var/lib/dhcp/dhcpd.leases -f c | \
awk -F, '$6 > 80 {print "WARNING: "$1" at "$6"%"}'
# Kea: query via REST API
curl -s -X POST -H "Content-Type: application/json" \
-d '{"command":"statistic-get-all","service":["dhcp4"]}' \
http://localhost:8000/ | jq '.arguments'
Pattern: Rogue DHCP Server Detection¶
A rogue DHCP server hands out wrong IPs, gateways, or DNS. Clients get misconfigured. Connectivity breaks randomly.
# Method 1: Listen for DHCP offers from unexpected sources
sudo tcpdump -i eth0 -n 'udp src port 67' -c 20
# Look for source IPs that are NOT your DHCP servers
# Method 2: Active probe with nmap
sudo nmap --script broadcast-dhcp-discover -e eth0
# Shows all responding DHCP servers
# Method 3: Use dhclient in verbose mode
sudo dhclient -v -1 eth0 2>&1 | grep "DHCPOFFER"
# "DHCPOFFER of 10.1.1.50 from 10.1.1.99"
# If 10.1.1.99 is not your server — rogue detected
# Mitigation: Enable DHCP snooping on switches
# Cisco:
# ip dhcp snooping
# ip dhcp snooping vlan 100
# interface GigabitEthernet0/1
# ip dhcp snooping trust ! Only on ports facing legit DHCP servers
DHCP snooping is the proper fix. It tells the switch which ports are allowed to send DHCP server responses. All other ports that try to send OFFERS get blocked.
Gotcha: Short Lease Times in Stable Networks¶
You set a 30-minute lease time on a server VLAN because "shorter is safer." Now 200 servers are renewing every 15 minutes (at T1 = 50%). That is 800 DHCP transactions per hour — per VLAN. Multiply by VLANs and you have a DHCP server under constant load. If the server goes down for 30 minutes, every server loses its IP.
Right-sizing:
# Check current renewal rate on the server
tcpdump -i eth0 -n 'udp port 67' -c 100 2>/dev/null | \
grep "DHCP-Request" | wc -l
# High count = leases are too short
# Check current lease times per scope
grep "default-lease-time" /etc/dhcp/dhcpd.conf
grep "max-lease-time" /etc/dhcp/dhcpd.conf
Rule of thumb: lease time should be at least 2x your maximum expected DHCP server downtime. If you can tolerate 4 hours of server downtime, set leases to 8 hours minimum.
Pattern: DHCP Option Debugging¶
Client gets an IP but wrong gateway, DNS, or domain:
# Check what options the server is sending
sudo tcpdump -i eth0 -n -vv 'udp port 67 or udp port 68' 2>&1 | \
grep -A 20 "DHCP-ACK"
# Look for:
# Option 3 (Router): 10.1.1.1
# Option 6 (Domain Name Server): 10.0.0.53
# Option 15 (Domain Name): prod.example.com
# On the client side, check what was received
cat /var/lib/dhcp/dhclient.eth0.leases | tail -20
# Compare against server config
grep -A 10 "subnet 10.1.1" /etc/dhcp/dhcpd.conf
Common issue: options defined at the global level in dhcpd.conf get overridden by scope-level options. Or vice versa — scope-level options are missing and global defaults do not match the VLAN.