Skip to content

Remediation: SSH Timeout, MTU Mismatch, Fix Is Terraform Variable

Immediate Fix (Cloud — Domain C)

The fix requires updating the Terraform VPC module to allow ICMP type 3 in subnet NACLs.

Step 1: Immediate — add NACL rule via AWS CLI

$ aws ec2 create-network-acl-entry \
    --network-acl-id acl-0abc123def456 \
    --rule-number 25 \
    --protocol 1 \
    --rule-action allow \
    --ingress \
    --icmp-type-code Type=3,Code=-1 \
    --cidr-block 0.0.0.0/0

Step 2: Verify SSH works immediately

$ ssh ec2-user@10.0.12.45
Last login: Thu Mar 19 15:30:12 2026
[ec2-user@ip-10-0-12-45 ~]$
# (connected instantly)

Step 3: Fix the Terraform module permanently

In devops/terraform/modules/vpc/nacl.tf:

resource "aws_network_acl_rule" "private_inbound_icmp_unreachable" {
  network_acl_id = aws_network_acl.private.id
  rule_number    = 25
  egress         = false
  protocol       = "icmp"
  rule_action    = "allow"
  cidr_block     = "0.0.0.0/0"
  icmp_type      = 3    # Destination Unreachable (includes Fragmentation Needed)
  icmp_code      = -1   # All codes
}

resource "aws_network_acl_rule" "private_inbound_icmp_time_exceeded" {
  network_acl_id = aws_network_acl.private.id
  rule_number    = 26
  egress         = false
  protocol       = "icmp"
  rule_action    = "allow"
  cidr_block     = "0.0.0.0/0"
  icmp_type      = 11   # Time Exceeded (needed for traceroute)
  icmp_code      = -1
}

Step 4: Apply Terraform

$ cd devops/terraform/modules/vpc
$ terraform plan -out=plan.tfplan
Plan: 2 to add, 0 to change, 0 to destroy.

$ terraform apply plan.tfplan
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Verification

Domain A (Linux Ops) — SSH works, large payloads transfer

$ ssh ec2-user@10.0.12.45 "dd if=/dev/urandom bs=1M count=1 | base64 | wc -c"
1398104

$ scp ec2-user@10.0.12.45:/var/log/messages /tmp/test-transfer
messages                                      100% 2.4MB   1.2MB/s   00:02

Domain B (Networking) — PMTUD working

$ ping -M do -s 1472 10.0.12.45
PING 10.0.12.45 (10.0.12.45) 1472(1500) bytes of data.
From 10.0.1.1 icmp_seq=1 Frag needed and DF set (mtu = 1400)
# ICMP type 3 is now received — PMTUD works

Domain C (Cloud) — NACL rules in Terraform state

$ terraform state list | grep nacl
module.vpc.aws_network_acl_rule.private_inbound_icmp_unreachable
module.vpc.aws_network_acl_rule.private_inbound_icmp_time_exceeded

$ aws ec2 describe-network-acls --network-acl-ids acl-0abc123def456 \
    --query 'NetworkAcls[].Entries[?IcmpTypeCode.Type==`3`]' --output table
| Egress | Protocol | IcmpType | RuleAction | RuleNumber |
| False  | 1        | 3        | allow      | 25         |

Prevention

  • Monitoring: Add a synthetic MTU check that sends 1500-byte packets with DF bit set to instances in every subnet. Alert when PMTUD fails.

  • Runbook: Every new subnet must allow ICMP types 3 (Destination Unreachable) and 11 (Time Exceeded) in its NACL. These are not optional — they are required for PMTUD and traceroute.

  • Architecture: Update the Terraform VPC module to include ICMP types 3 and 11 by default in all subnet NACLs. Add a tflint rule or OPA policy that flags NACLs missing these ICMP types.