Skip to content

Portal | Level: L2: Operations | Topics: RBAC | Domain: Kubernetes

Scenario: RBAC Forbidden Error During Deploy

The Prompt

"A new engineer joined the team and can't deploy to our Kubernetes cluster. They get Error from server (Forbidden): deployments.apps is forbidden: User 'jane' cannot create resource 'deployments' in API group 'apps' in namespace 'grokdevops'. How do you resolve this?"

Initial Report

New engineer on Slack: "Hey, I'm trying to deploy the hotfix but I keep getting a Forbidden error. I followed the onboarding doc. Can someone unblock me? The customer is waiting."

Constraints

  • Time pressure: You have 15 minutes before the next escalation. A P2 bugfix is blocked because the assigned engineer cannot deploy.
  • Limited access: You can view RBAC resources but creating RoleBindings requires cluster-admin approval. You do not have access to the identity provider (OIDC/LDAP) to check group memberships.

Observable Evidence

  • Terminal output: Error from server (Forbidden): deployments.apps is forbidden: User 'jane' cannot create resource 'deployments' in API group 'apps' in namespace 'grokdevops'.
  • RBAC check: kubectl auth can-i create deployments --as=jane -n grokdevops returns no.
  • Existing bindings: kubectl get rolebindings -n grokdevops shows a dev-team binding to a developer role, but jane is not listed as a subject.

Expected Investigation Path

# 1. Check what the user can do
kubectl auth can-i --list --as=jane -n grokdevops

# 2. Check existing roles and bindings
kubectl get roles,rolebindings -n grokdevops
kubectl get clusterroles,clusterrolebindings | grep -E "jane|deploy"

# 3. Check if there's a team role we should bind to
kubectl get roles -n grokdevops -o name

# 4. Create appropriate binding
kubectl create rolebinding jane-deploy \
  --role=<existing-deploy-role> \
  --user=jane \
  -n grokdevops

Strong Answer

"The error is clear: the user 'jane' lacks permission to create deployments in the grokdevops namespace. I'd first check what roles already exist — most teams have predefined roles like 'developer' or 'deployer'. I'd use kubectl auth can-i --list --as=jane to see their current permissions. Then I'd create a RoleBinding associating jane with the appropriate existing role. I would NOT create a ClusterRoleBinding or give cluster-admin — principle of least privilege. For a longer-term solution, we should manage RBAC through our IaC pipeline (Helm values or Terraform), add the binding to a team role rather than individual user, and use groups (from OIDC/LDAP) so new team members automatically get the right permissions."

Common Traps

  • Giving cluster-admin — this is the "just make it work" anti-pattern
  • Creating ClusterRoleBinding when a RoleBinding suffices — scope matters
  • Not using groups — individual bindings don't scale
  • Not mentioning IaC for RBAC — RBAC should be version-controlled
  • Runbook: training/library/runbooks/kubernetes/rbac_forbidden.md
  • Quest: training/interactive/exercises/levels/level-41/k8s-rbac-permission/

Wiki Navigation

Related Content