Skip to content

Portal | Level: L1: Foundations | Topics: AWS Networking, Cloud Deep Dive, TCP/IP | Domain: Cloud

AWS Networking - Primer

Why This Matters

Every production system on AWS sits inside a VPC. If the network is wrong, nothing works — your application cannot reach its database, your users cannot reach your load balancer, and your pods cannot pull container images. Network misconfigurations are the #1 category of "it was working yesterday" incidents on AWS. Understanding VPC architecture, security groups, route tables, and DNS resolution is not optional for anyone operating workloads on AWS.

Core Concepts

1. VPC Architecture

A Virtual Private Cloud (VPC) is an isolated network within an AWS region. It is the foundation everything else builds on.

Region (us-east-1)
└── VPC (10.0.0.0/16) — 65,536 IPs
    ├── Availability Zone: us-east-1a
    │   ├── Public Subnet  (10.0.1.0/24) — 251 usable IPs
    │   └── Private Subnet (10.0.10.0/24)
    ├── Availability Zone: us-east-1b
    │   ├── Public Subnet  (10.0.2.0/24)
    │   └── Private Subnet (10.0.20.0/24)
    ├── Internet Gateway (IGW) — one per VPC
    ├── NAT Gateway — one per AZ for HA
    └── Route Tables — one per subnet

Key facts: - A VPC spans a region but subnets are AZ-specific - AWS reserves 5 IPs per subnet (first 4 + last 1): network, VPC router, DNS, future, broadcast

Remember: Mnemonic for AWS's 5 reserved IPs: "Never Very Dull, Future Broadcast" -- Network (.0), VPC router (.1), DNS (.2), Future use (.3), Broadcast (.255). This means a /28 subnet (16 IPs) gives you only 11 usable addresses. - A /24 subnet gives you 251 usable IPs, not 254 - VPC CIDR range: /16 (largest) to /28 (smallest), from RFC 1918 space - You can add secondary CIDR blocks to expand a VPC

# Create a VPC
aws ec2 create-vpc --cidr-block 10.0.0.0/16 --tag-specifications \
  'ResourceType=vpc,Tags=[{Key=Name,Value=prod-vpc}]'

# Enable DNS resolution and hostnames (required for most services)
aws ec2 modify-vpc-attribute --vpc-id vpc-abc123 --enable-dns-support '{"Value":true}'
aws ec2 modify-vpc-attribute --vpc-id vpc-abc123 --enable-dns-hostnames '{"Value":true}'

# Create subnets
aws ec2 create-subnet --vpc-id vpc-abc123 --cidr-block 10.0.1.0/24 \
  --availability-zone us-east-1a --tag-specifications \
  'ResourceType=subnet,Tags=[{Key=Name,Value=public-1a}]'

2. Public vs Private Subnets

The difference between public and private is entirely about routing — there is no "public subnet" checkbox.

Public subnet: route table has a route to an Internet Gateway (0.0.0.0/0 → igw-xxx). Instances with public IPs can be reached from the internet.

Private subnet: route table has NO route to an IGW. Outbound internet access goes through a NAT Gateway (if any). Instances are not directly reachable from outside.

# Public route table
aws ec2 create-route-table --vpc-id vpc-abc123
aws ec2 create-route --route-table-id rtb-pub \
  --destination-cidr-block 0.0.0.0/0 \
  --gateway-id igw-abc123

# Private route table (outbound through NAT)
aws ec2 create-route --route-table-id rtb-priv \
  --destination-cidr-block 0.0.0.0/0 \
  --nat-gateway-id nat-abc123

# Associate subnet with route table
aws ec2 associate-route-table --route-table-id rtb-pub --subnet-id subnet-pub1a

3. Security Groups vs NACLs

These are your two firewall layers. They work differently and you need both.

Feature Security Group NACL
Level Instance/ENI Subnet
State Stateful (return traffic auto-allowed) Stateless (must allow return traffic explicitly)
Rules Allow only Allow and Deny
Evaluation All rules evaluated Rules evaluated in order (first match wins)
Default Deny all inbound, allow all outbound Allow all inbound and outbound 
# Security group: allow HTTPS from anywhere, SSH from office
aws ec2 create-security-group --group-name web-sg \
  --description "Web server SG" --vpc-id vpc-abc123

aws ec2 authorize-security-group-ingress --group-id sg-web \
  --protocol tcp --port 443 --cidr 0.0.0.0/0

aws ec2 authorize-security-group-ingress --group-id sg-web \
  --protocol tcp --port 22 --cidr 203.0.113.0/24

# NACL: remember to allow ephemeral ports for return traffic
aws ec2 create-network-acl-entry --network-acl-id acl-abc123 \
  --rule-number 100 --protocol tcp --port-range From=443,To=443 \
  --cidr-block 0.0.0.0/0 --rule-action allow --ingress

aws ec2 create-network-acl-entry --network-acl-id acl-abc123 \
  --rule-number 100 --protocol tcp --port-range From=1024,To=65535 \
  --cidr-block 0.0.0.0/0 --rule-action allow --egress

Default trap: New VPCs come with a default security group that allows all inbound traffic from itself (the same SG) and all outbound. This means instances in the default SG can talk to each other freely. Many teams accidentally leave instances in the default SG, creating an overly permissive blast radius. Always create purpose-specific security groups.

Security groups can reference other security groups — this is the recommended pattern for intra-VPC communication:

# Allow the app tier to talk to the database tier
aws ec2 authorize-security-group-ingress --group-id sg-db \
  --protocol tcp --port 5432 --source-group sg-app

4. Internet Gateway and NAT Gateway

Internet Gateway (IGW): provides bidirectional internet access for public subnets. One per VPC. Free (no hourly charge).

NAT Gateway: provides outbound-only internet access for private subnets. Instances in private subnets can reach the internet (for package updates, API calls) but cannot be reached from outside.

# Create NAT gateway (must be in a public subnet, needs an Elastic IP)
aws ec2 allocate-address --domain vpc
# Returns: AllocationId: eipalloc-abc123

aws ec2 create-nat-gateway \
  --subnet-id subnet-pub1a \
  --allocation-id eipalloc-abc123

# NAT gateway costs: ~$0.045/hour + $0.045/GB processed
# That is ~$32/month just for existing, plus data costs

NAT Gateway is single-AZ. For high availability, deploy one per AZ:

AZ-a: NAT-a in public-1a → private-1a routes through NAT-a
AZ-b: NAT-b in public-1b → private-1b routes through NAT-b

5. VPC Peering

VPC peering creates a direct network connection between two VPCs. Traffic stays on AWS's backbone — it does not traverse the public internet.

# Create peering connection
aws ec2 create-vpc-peering-connection \
  --vpc-id vpc-requester \
  --peer-vpc-id vpc-accepter \
  --peer-owner-id 999888777666  # for cross-account

# Accept the peering (in the accepter account)
aws ec2 accept-vpc-peering-connection \
  --vpc-peering-connection-id pcx-abc123

# Add routes in BOTH VPCs (peering is not transitive)
aws ec2 create-route --route-table-id rtb-requester \
  --destination-cidr-block 10.1.0.0/16 \
  --vpc-peering-connection-id pcx-abc123

aws ec2 create-route --route-table-id rtb-accepter \
  --destination-cidr-block 10.0.0.0/16 \
  --vpc-peering-connection-id pcx-abc123

Limitations:

Gotcha: VPC peering requires routes in both VPCs, and the CIDR ranges cannot overlap. If you created VPC-A as 10.0.0.0/16 and VPC-B also as 10.0.0.0/16, you cannot peer them. Plan your CIDR allocation across all VPCs and accounts before you build. AWS IPAM (IP Address Manager) can help govern this centrally.

  • Not transitive: VPC-A peers with VPC-B, VPC-B peers with VPC-C — A cannot reach C through B
  • CIDRs cannot overlap
  • Max 125 peering connections per VPC
  • Cross-region peering is supported but has higher latency and costs

6. Transit Gateway

Transit Gateway (TGW) solves the peering scalability problem. It acts as a central hub — all VPCs connect to the TGW, and routes flow through it.

Before (mesh):          After (hub):
VPC-A ←→ VPC-B         VPC-A ──┐
VPC-A ←→ VPC-C         VPC-B ──┤── Transit Gateway
VPC-B ←→ VPC-C         VPC-C ──┘
(3 connections)         (3 connections, but scales to thousands)

TGW also connects VPNs, Direct Connect, and other TGWs (inter-region peering).

# Create transit gateway
aws ec2 create-transit-gateway \
  --description "Central hub" \
  --options DefaultRouteTableAssociation=enable,DefaultRouteTablePropagation=enable

# Attach a VPC
aws ec2 create-transit-gateway-vpc-attachment \
  --transit-gateway-id tgw-abc123 \
  --vpc-id vpc-abc123 \
  --subnet-ids subnet-1a subnet-1b

7. VPC Endpoints

VPC endpoints let you access AWS services without going through the internet. Two types:

Gateway endpoints (S3 and DynamoDB only): free, added to route tables.

# Gateway endpoint for S3
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-abc123 \
  --service-name com.amazonaws.us-east-1.s3 \
  --route-table-ids rtb-priv

Interface endpoints (all other services): creates an ENI in your subnet. Costs ~$0.01/hour + $0.01/GB.

# Interface endpoint for Secrets Manager
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-abc123 \
  --vpc-endpoint-type Interface \
  --service-name com.amazonaws.us-east-1.secretsmanager \
  --subnet-ids subnet-priv1a subnet-priv1b \
  --security-group-ids sg-endpoint

# Enable private DNS so the default service hostname resolves to the endpoint
# --private-dns-enabled (default for interface endpoints)

8. Elastic Network Interfaces (ENI) and Elastic IPs

ENI is a virtual network card. Every instance has at least one. You can attach additional ENIs for multi-homed instances, management networks, or failover.

Elastic IP (EIP) is a static public IPv4 address. It persists until you release it. Charged when NOT attached to a running instance ($0.005/hour).

# Allocate and associate
aws ec2 allocate-address --domain vpc
aws ec2 associate-address --instance-id i-abc123 --allocation-id eipalloc-abc

# List unattached EIPs (you are being charged for these)
aws ec2 describe-addresses --query 'Addresses[?AssociationId==null]'

9. DNS Resolution in VPC

Every VPC gets a Route 53 Resolver at the VPC CIDR base + 2 (e.g., 10.0.0.2). This resolves: - Public DNS names - Private hosted zone records - VPC endpoint DNS names - EC2 instance hostnames (if DNS hostnames are enabled)

# Enable DNS support (needed for VPC endpoints, private hosted zones)
aws ec2 modify-vpc-attribute --vpc-id vpc-abc123 \
  --enable-dns-support '{"Value":true}'
aws ec2 modify-vpc-attribute --vpc-id vpc-abc123 \
  --enable-dns-hostnames '{"Value":true}'

# Route 53 Resolver endpoints for hybrid DNS (on-prem ↔ AWS)
# Inbound endpoint: on-prem DNS forwards queries to AWS
# Outbound endpoint: AWS forwards queries to on-prem DNS

10. VPC Flow Logs

Flow logs capture IP traffic metadata (not payload) for network debugging:

# Enable flow logs on a VPC (to CloudWatch Logs)
aws ec2 create-flow-logs \
  --resource-ids vpc-abc123 \
  --resource-type VPC \
  --traffic-type ALL \
  --log-destination-type cloud-watch-logs \
  --log-group-name /vpc/flow-logs \
  --deliver-logs-permission-arn arn:aws:iam::123456789012:role/flow-logs-role

# Flow log record format:
# version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status
# 2 123456789012 eni-abc123 10.0.1.5 10.0.2.10 49152 3306 6 20 4000 1620000000 1620000060 ACCEPT OK
# 2 123456789012 eni-abc123 203.0.113.5 10.0.1.5 12345 22 6 3 180 1620000000 1620000060 REJECT OK

11. Load Balancers: ALB vs NLB

Application Load Balancer (ALB): Layer 7, HTTP/HTTPS. Path-based routing, host-based routing, WebSocket support, WAF integration.

Network Load Balancer (NLB): Layer 4, TCP/UDP/TLS. Ultra-low latency, static IPs, preserves source IP, millions of requests per second.

# ALB with target group
aws elbv2 create-load-balancer \
  --name web-alb \
  --type application \
  --subnets subnet-pub1a subnet-pub1b \
  --security-groups sg-alb

aws elbv2 create-target-group \
  --name web-targets \
  --protocol HTTP --port 8080 \
  --vpc-id vpc-abc123 \
  --health-check-path /health \
  --health-check-interval-seconds 15

# NLB (no security groups — NLB is transparent)
aws elbv2 create-load-balancer \
  --name api-nlb \
  --type network \
  --subnets subnet-pub1a subnet-pub1b

When to use which: - ALB: web applications, REST APIs, gRPC, anything HTTP-based - NLB: TCP/UDP services, extreme performance needs, static IP requirements, PrivateLink

12. Cross-AZ Traffic Costs

Traffic between AZs costs $0.01/GB in each direction ($0.02 round-trip). This adds up fast for high-throughput services.

Same AZ:  instance-A → instance-B = free
Cross AZ: instance-A (us-east-1a) → instance-B (us-east-1b) = $0.01/GB each way

Mitigation strategies: - Use AZ-aware load balancing (ALB cross-zone enabled by default — consider disabling for cost) - For NLB: cross-zone is disabled by default (enable only if needed) - Keep tightly coupled services in the same AZ when possible - Monitor with Cost Explorer, filter by usage type DataTransfer-Regional-Bytes

PrivateLink exposes a service to other VPCs through an interface endpoint — without VPC peering, public IPs, or route changes. Traffic stays on AWS's network.

Provider VPC (NLB + PrivateLink service)
    ↕ AWS backbone
Consumer VPC (Interface endpoint → ENI)

This is how third-party SaaS vendors give you private access to their services, and how you can expose internal services to other teams' VPCs.

14. Direct Connect

Direct Connect provides a dedicated physical network connection from your data center to AWS. Not over the internet — it is a literal cross-connect in a colocation facility.

Use cases: - Hybrid cloud with consistent latency requirements - Large data transfers (cheaper than internet data transfer at scale) - Regulatory requirements for private connectivity - Bandwidth-intensive workloads

Two flavors: Dedicated (1 Gbps or 10 Gbps physical port) and Hosted (sub-1Gbps through a partner).

Key Takeaways

  • Public vs private subnet is determined by route table entries, not a setting
  • Security groups are stateful; NACLs are stateless — you need explicit return traffic rules for NACLs
  • NAT Gateway is single-AZ by default — deploy one per AZ for production
  • VPC peering is not transitive — Transit Gateway solves hub-and-spoke at scale
  • Gateway VPC endpoints (S3, DynamoDB) are free; interface endpoints cost money
  • Cross-AZ traffic is not free — $0.01/GB each direction adds up
  • Flow logs are your first tool for debugging traffic problems

Interview tip: A classic AWS networking interview question: "A private instance cannot reach the internet for package updates. Walk me through what you would check." Answer sequence: (1) route table has 0.0.0.0/0 pointing to a NAT Gateway, (2) NAT Gateway is in a public subnet with a route to the IGW, (3) NAT Gateway has an Elastic IP, (4) security group allows outbound, (5) NACL allows outbound AND return ephemeral ports inbound. - DNS resolution requires both EnableDnsSupport and EnableDnsHostnames on the VPC

Wiki Navigation

Prerequisites

Next Steps

Related Content

  • AWS CloudWatch (Topic Pack, L2) — Cloud Deep Dive
  • AWS Devops Flashcards (CLI) (flashcard_deck, L1) — Cloud Deep Dive
  • AWS EC2 (Topic Pack, L1) — Cloud Deep Dive
  • AWS ECS (Topic Pack, L2) — Cloud Deep Dive
  • AWS General Flashcards (CLI) (flashcard_deck, L1) — Cloud Deep Dive
  • AWS IAM (Topic Pack, L1) — Cloud Deep Dive
  • AWS Lambda (Topic Pack, L2) — Cloud Deep Dive
  • AWS Networking Flashcards (CLI) (flashcard_deck, L1) — AWS Networking
  • AWS Route 53 (Topic Pack, L2) — Cloud Deep Dive
  • AWS S3 Deep Dive (Topic Pack, L1) — Cloud Deep Dive