Skip to content

Anti-Primer: AWS IAM

Everything that can go wrong, will — and in this story, it does.

The Setup

A startup's sole DevOps engineer is setting up IAM for a growing team. Under pressure to unblock developers, they start granting broad permissions with the intention of tightening them 'later.'

The Timeline

Hour 0: Wildcard Resource Permissions

Grants s3:* on Resource: * to unblock a developer quickly. The deadline was looming, and this seemed like the fastest path forward. But the result is an intern's script accidentally deletes objects from the billing bucket.

Footgun #1: Wildcard Resource Permissions — grants s3:* on Resource: * to unblock a developer quickly, leading to an intern's script accidentally deletes objects from the billing bucket.

Nobody notices yet. The engineer moves on to the next task.

Hour 1: Long-lived Access Keys

Creates access keys for a CI pipeline and never rotates them. Under time pressure, the team chose speed over caution. But the result is keys leak in a public repo; attacker mines crypto on 50 instances before detection.

Footgun #2: Long-lived Access Keys — creates access keys for a CI pipeline and never rotates them, leading to keys leak in a public repo; attacker mines crypto on 50 instances before detection.

The first mistake is still invisible, making the next shortcut feel justified.

Hour 2: No MFA on Root Account

Skips MFA setup on the root account because 'nobody uses it'. Nobody pushed back because the shortcut looked harmless in the moment. But the result is phished credentials give attacker full account control with no second factor.

Footgun #3: No MFA on Root Account — skips MFA setup on the root account because 'nobody uses it', leading to phished credentials give attacker full account control with no second factor.

Pressure is mounting. The team is behind schedule and cutting more corners.

Hour 3: Inline Policies Everywhere

Attaches inline policies to individual users instead of using groups and managed policies. The team had gotten away with similar shortcuts before, so nobody raised a flag. But the result is 20 users each have unique permissions; auditing who can do what takes days.

Footgun #4: Inline Policies Everywhere — attaches inline policies to individual users instead of using groups and managed policies, leading to 20 users each have unique permissions; auditing who can do what takes days.

By hour 3, the compounding failures have reached critical mass. Pages fire. The war room fills up. The team scrambles to understand what went wrong while the system burns.

The Postmortem

Root Cause Chain

# Mistake Consequence Could Have Been Prevented By
1 Wildcard Resource Permissions An intern's script accidentally deletes objects from the billing bucket Primer: Least privilege: scope to specific ARNs and actions
2 Long-lived Access Keys Keys leak in a public repo; attacker mines crypto on 50 instances before detection Primer: Use IAM roles for services; rotate keys on a schedule
3 No MFA on Root Account Phished credentials give attacker full account control with no second factor Primer: MFA on root from day one; lock root keys in a vault
4 Inline Policies Everywhere 20 users each have unique permissions; auditing who can do what takes days Primer: Managed policies attached to groups; no inline policies on users

Damage Report

  • Downtime: 3-6 hours of degraded or unavailable cloud services
  • Data loss: Possible if storage or database resources were affected
  • Customer impact: API errors, failed transactions, or service unavailability for end users
  • Engineering time to remediate: 12-24 engineer-hours across incident response, root cause analysis, and remediation
  • Reputation cost: Internal trust erosion; potential AWS billing surprises; customer-facing impact report required

What the Primer Teaches

  • Footgun #1: If the engineer had read the primer, section on wildcard resource permissions, they would have learned: Least privilege: scope to specific ARNs and actions.
  • Footgun #2: If the engineer had read the primer, section on long-lived access keys, they would have learned: Use IAM roles for services; rotate keys on a schedule.
  • Footgun #3: If the engineer had read the primer, section on no mfa on root account, they would have learned: MFA on root from day one; lock root keys in a vault.
  • Footgun #4: If the engineer had read the primer, section on inline policies everywhere, they would have learned: Managed policies attached to groups; no inline policies on users.

Cross-References