Binary and Floating Point — Street-Level Ops¶
Quick Diagnosis Commands¶
# Inspect binary data in hex (two most common tools)
xxd /path/to/binary | head -20
hexdump -C /path/to/binary | head -20
# Check byte order of a binary file
file /path/to/binary
# ELF 64-bit LSB executable → LSB = Little-endian
# View raw bytes of a network capture (see byte order in action)
tcpdump -XX -r capture.pcap | head -30
# Check endianness of the running system
python3 -c "import sys; print(sys.byteorder)"
# Output: little (on x86/ARM)
# Examine how a float is stored in memory (Python)
python3 -c "
import struct
val = 0.1
raw = struct.pack('d', val)
print('Bytes:', raw.hex())
print('Roundtrip:', struct.unpack('d', raw)[0])
print('Exact repr:', repr(val))
"
# Check if a 32-bit timestamp will overflow (Y2038 check)
python3 -c "
import datetime
max_32 = 2**31 - 1
print('32-bit time_t overflows at:', datetime.datetime.fromtimestamp(max_32, tz=datetime.timezone.utc))
"
# Output: 2038-01-19 03:14:07
# Show file permissions as octal bitmask
stat -c '%a %n' /etc/passwd
# Output: 644 /etc/passwd
# Convert between decimal, hex, octal, binary in bash
printf 'Dec: %d Hex: 0x%x Oct: %o\n' 255 255 255
# Dec: 255 Hex: 0xff Oct: 377
# Quick subnet mask to CIDR conversion (bitwise)
python3 -c "
mask = '255.255.252.0'
cidr = sum(bin(int(x)).count('1') for x in mask.split('.'))
print(f'{mask} = /{cidr}')
"
Gotcha: Floating Point Comparison in Monitoring¶
Symptom: Alerting rule fires intermittently on a metric that should be exactly 1.0. The metric oscillates between 0.9999999999999998 and 1.0000000000000002.
Rule: Never compare floats for exact equality in alerting, SLO calculations, or threshold checks.
Under the hood: IEEE 754 floats represent 0.1 as
0.1000000000000000055511151231257827021181583404541015625. The extra digits are because 0.1 in binary is a repeating fraction (like 1/3 in decimal). This is not a bug — it is a fundamental property of binary floating point.
# BAD: Prometheus alert rule
# alert: SLOBreached
# expr: slo_ratio == 1.0
# This will flap because float arithmetic rarely produces exact 1.0
# GOOD: Use epsilon comparison
# expr: abs(slo_ratio - 1.0) > 0.001
# Python equivalent
python3 -c "
a = 0.1 + 0.2
b = 0.3
print(f'a == b: {a == b}') # False
print(f'close: {abs(a - b) < 1e-9}') # True
import math
print(f'isclose: {math.isclose(a, b)}') # True (default rel_tol=1e-9)
"
Gotcha: Counter Overflow in Monitoring¶
Symptom: Prometheus rate() returns a massive spike, then drops to zero. The underlying counter wrapped from its maximum value back to zero.
# Check the width of a counter
# Prometheus uses float64 for all values, but source counters vary:
# - Linux /proc counters: usually 64-bit unsigned
# - SNMP ifInOctets: 32-bit (wraps at ~4.29 GB)
# - SNMP ifHCInOctets: 64-bit (use this instead)
# Calculate when a 32-bit counter wraps at a given rate
python3 -c "
rate_bytes_sec = 1_000_000_000 # 1 GB/s
max_32 = 2**32
seconds = max_32 / rate_bytes_sec
print(f'32-bit counter wraps in {seconds:.0f} seconds ({seconds/60:.1f} minutes)')
"
# Output: wraps in 4 seconds (0.1 minutes) at 1 GB/s
> **Gotcha:** Prometheus `rate()` handles counter resets (wraps) automatically, but only if the counter resets to zero. If a 32-bit counter wraps from 4294967295 to 0, `rate()` detects the discontinuity and computes the correct rate. But if the counter wraps and the new value is non-zero before the next scrape, the rate will spike.
# For SNMP — always prefer HC (high capacity / 64-bit) counters
snmpget -v2c -c public switch1 IF-MIB::ifHCInOctets.1
Pattern: Inspecting Binary Protocols on the Wire¶
When debugging binary protocols (DNS, TLS, custom), you need to read hex dumps.
# Capture and display DNS query bytes
tcpdump -i eth0 port 53 -XX -c 5
# Example hex dump of a DNS query:
# 0x0000: 4500 003c 1234 0000 4011 ...
# ^IP header ^UDP
# Bytes are in network byte order (big-endian)
# Parse a specific byte pattern with Python
python3 -c "
import struct
# Simulating reading a 4-byte big-endian int from network data
raw = bytes([0x00, 0x00, 0x00, 0x50]) # port 80 in network byte order
port = struct.unpack('!H', raw[2:4])[0] # '!' = network (big-endian)
print(f'Port: {port}')
"
# Convert between host and network byte order in C (conceptual)
# ntohs(0x5000) → 80 (network-to-host short)
# htonl(0x0A000132) → depends on host endianness
Pattern: Debugging Floating Point in Financial / Billing Systems¶
Rule: Never use float for money. If you find it, fix it immediately.
Remember: For money, use integer cents (store $19.99 as 1999) or language-specific decimal types (Python
Decimal, JavaBigDecimal, PostgreSQLnumeric). Mnemonic: Floats lie about money. Integers and decimals tell the truth.
# Demonstrate the problem
python3 -c "
# Simulating billing calculation with floats
price = 19.99
qty = 3
tax_rate = 0.0825
subtotal = price * qty
tax = subtotal * tax_rate
total = subtotal + tax
print(f'Total: {total}') # 64.8675750000...01 (extra digits)
# Correct approach: use Decimal
from decimal import Decimal
price = Decimal('19.99')
qty = 3
tax_rate = Decimal('0.0825')
subtotal = price * qty
tax = subtotal * tax_rate
total = subtotal + tax
print(f'Total: {total}') # 64.867575 (exact)
"
# Check PostgreSQL column types for money-related tables
psql -c "SELECT column_name, data_type
FROM information_schema.columns
WHERE table_name = 'invoices'
AND data_type IN ('real', 'double precision');"
# If this returns rows, those columns are using floats for money — fix them
Pattern: Bit Manipulation for IP Networking¶
# Calculate network address from IP and mask
python3 -c "
import ipaddress
ip = ipaddress.ip_interface('10.0.1.50/22')
print(f'Network: {ip.network.network_address}') # 10.0.0.0
print(f'Broadcast: {ip.network.broadcast_address}') # 10.0.3.255
print(f'Hosts: {ip.network.num_addresses - 2}') # 1022
print(f'Mask bits: {bin(int(ip.netmask))}')
"
# Manual bitwise AND to get network address
python3 -c "
ip = (10 << 24) | (0 << 16) | (1 << 8) | 50 # 10.0.1.50
mask = (255 << 24) | (255 << 16) | (252 << 8) | 0 # /22
net = ip & mask
print(f'Network: {(net>>24)&0xff}.{(net>>16)&0xff}.{(net>>8)&0xff}.{net&0xff}')
"
Pattern: Detecting Y2038 Vulnerable Timestamps¶
# Find files that might store 32-bit timestamps
# Look for time_t usage in C headers
grep -r 'time_t' /usr/include/ 2>/dev/null | head -5
# Check if a running system uses 64-bit time_t
python3 -c "
import struct, time
# On 64-bit systems, time.time() returns float but struct.pack uses native size
print(f'time_t size: {struct.calcsize(\"l\")} bytes')
# 8 = safe (64-bit), 4 = Y2038 vulnerable (32-bit)
"
# Check database timestamp column sizes
# MySQL: TIMESTAMP type is 32-bit (Y2038 vulnerable), DATETIME is not
mysql -e "SELECT TABLE_NAME, COLUMN_NAME, DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE DATA_TYPE = 'timestamp'
AND TABLE_SCHEMA = 'production';" 2>/dev/null
Useful One-Liners¶
# Convert decimal to binary representation
python3 -c "print(f'{255:08b}')" # 11111111
# Show NaN behavior (debugging metric anomalies)
python3 -c "
import math
x = float('nan')
print(f'nan == nan: {x == x}') # False (IEEE 754 spec)
# This is the ONLY value in IEEE 754 that is not equal to itself
print(f'nan in list: {x in [x]}') # True (Python identity check)
print(f'isnan: {math.isnan(x)}') # True (correct way)
"
# Verify file is not stripped (has debug symbols for profiling)
file /usr/bin/python3 | grep -o 'not stripped' || echo 'stripped (no symbols)'
# Quick endianness test from shell
python3 -c "import sys; print(sys.byteorder)"
# Decode a hex string to ASCII
echo "48656c6c6f" | xxd -r -p
# Output: Hello
# Check integer overflow risk in a shell counter
python3 -c "
import sys
print(f'Python int: unlimited')
print(f'C int32 max: {2**31 - 1}')
print(f'C uint32 max: {2**32 - 1}')
print(f'C int64 max: {2**63 - 1}')
print(f'C uint64 max: {2**64 - 1}')
"