Skip to content

Solution: Asymmetric Routing / One-Direction Failure

Summary

Traffic from B to A fails because B's SYN packets traverse fw-02, but A's SYN-ACK replies are routed back through fw-01. Since fw-01 never saw the original SYN, its stateful inspection drops the SYN-ACK as an invalid packet. A-to-B works because both directions of that flow happen to traverse the same firewall.

Senior Workflow

Step 1: Map the network path from both directions

# From Host A (10.1.10.25):
traceroute -n 10.2.20.40
# Expected: 10.1.10.25 -> 10.1.0.1 (fw-01) -> 10.2.20.40

# From Host B (10.2.20.40):
traceroute -n 10.1.10.25
# Expected: 10.2.20.40 -> 10.2.0.1 (fw-02) -> 10.1.10.25

This confirms the two hosts use different firewalls as their gateway.

Step 2: Check routing tables

# Host A
ip route show
# Look for: default via 10.1.0.1 (fw-01)

# Host B
ip route show
# Look for: default via 10.2.0.1 (fw-02)

Step 3: Trace the B-to-A connection failure

When B sends SYN to A: 1. B -> fw-02 (SYN) -> A [fw-02 creates state entry] 2. A -> fw-01 (SYN-ACK) -> DROPPED [fw-01 has no state for this flow]

Step 4: Verify on firewall logs

# On fw-02: check for the SYN from B
grep "10.2.20.40.*10.1.10.25" /var/log/firewall.log

# On fw-01: check for dropped SYN-ACK from A
grep "DROP.*10.1.10.25.*10.2.20.40" /var/log/firewall.log

Step 5: Confirm with state tables

# On fw-01:
conntrack -L | grep 10.2.20.40
# No entries for B-initiated connections

# On fw-02:
conntrack -L | grep 10.1.10.25
# Shows SYN_SENT entries that never complete

Step 6: Apply the fix

Option A -- Fix routing symmetry (preferred) Use VRRP/HSRP so both subnets use a single virtual gateway IP. The active firewall handles all traffic; failover is automatic.

Option B -- Enable state synchronization 

# Configure conntrackd on both firewalls to share state
# /etc/conntrackd/conntrackd.conf on both fw-01 and fw-02

Option C -- Correct static routes Ensure both hosts route to the same firewall, or add specific routes so cross-subnet traffic always traverses a single firewall.

Step 7: Verify

# From Host B:
ssh 10.1.10.25
curl http://10.1.10.25:8080/health

Common Pitfalls

  • Assuming identical rules = identical behavior: Stateful firewalls require seeing the full flow. Identical rules do not help if the firewall never saw the SYN.
  • Only testing one direction: Always test connectivity from both sides when troubleshooting.
  • Overlooking return path: The most common mistake is focusing only on where the initial packet goes, not where the reply returns.
  • Disabling state tracking: This removes security benefits. Fix the routing instead.
  • Not considering future changes: A VRRP/HSRP solution handles failover properly; static route fixes are fragile.