Kubernetes Secrets Access and Service Account Token Theft

TL;DR. Credential theft in Kubernetes leaves two kinds of traces. On the secrets side: a list or watch on secrets without a namespace (every secret in the cluster, contents included), or one identity reading many different secrets in a short window. On the token side: a service account used from a public IP or with kubectl/curl as user agent, a burst of kubectl auth can-i checks, 403 bursts, and new tokens minted through TokenRequest. Every secret read is a credential to rotate, and on GKE the reads are only logged if Data Access logs are on.

Secrets are the reason most attackers bother with a cluster in the first place: database passwords, cloud keys, registry credentials, CI tokens, signing keys. ATT&CK files this under T1552.007, Container API, and the token side under T1528, Steal Application Access Token.

How secrets leak through the API

The Kubernetes RBAC good practices make a point many teams miss: list and watch on secrets reveal their contents, just like get. A role that "only lists" secrets is a role that reads them all.

The last row matters: permission to create workloads in a namespace implicitly grants access to its secrets and service account tokens. That read does not show up as a get secrets event.

At the default levels of EKS and GKE, and in any sane policy, secrets are logged at Metadata: you see who read which secret, never the value. That is enough to scope rotation.

The GKE caveat

On GKE, get and list go to the Data Access audit log, which is off by default (see the export guide). Without it, secret reads leave no trace at all. On AKS, the cheaper kube-audit-admin category also drops get and list.

Detections for secrets

The audit log analyzer has two rules:

• Secrets listed across all namespaces (high): list or watch on secrets without a namespace, allowed, by a non-system identity. Controllers that legitimately watch all secrets (for example ingress or certificate operators) must be allow-listed after checking.
• Burst of secret reads (medium): one identity reading 10 or more different secrets within 10 minutes. Counting distinct names avoids flagging an application that re-reads its own secret.

Kubelets (system:node:*) read the secrets of the pods scheduled on their node all day; that is normal and excluded.

How service account tokens get stolen

Pods get a token for their service account, mounted by default at /var/run/secrets/kubernetes.io/serviceaccount/token. Since Kubernetes 1.22 these are short-lived, auto-rotated tokens obtained through the TokenRequest API; since 1.24, long-lived tokens stored in Secrets are no longer created automatically, according to the service accounts documentation. Legacy token Secrets created by hand or before an upgrade are still around in many clusters.

Typical theft paths:

• reading the mounted token after exploiting an application (RCE, SSRF, path traversal);
• kubectl exec … cat /var/run/secrets/kubernetes.io/serviceaccount/token;
• reading a legacy token Secret, a kubeconfig in a Secret, or a CI variable;
• an exposed dashboard or tool running with an over-privileged service account;
• minting a new token with kubectl create token (TokenRequest) when RBAC allows create on serviceaccounts/token.

What a replayed token looks like

A token used by its workload has a stable pattern: pod IP addresses, the application's own user agent, the same few API calls. A replayed token breaks that pattern:

kubectl auth can-i --list produces a SelfSubjectRulesReview; kubectl auth can-i create pods a SelfSubjectAccessReview. Workloads rarely call either, so a burst of them from a service account is someone discovering what their stolen token can do (ATT&CK T1613).

Caveats you should keep in mind:

• The user agent is client-controlled. Its absence proves nothing; its presence is still a good lead.
• Source IP comes from the last entry of sourceIPs. On clusters behind a proxy or with a private endpoint, a "public" address may not appear at all.
• Some Go-based operators use a generic Go-http-client agent. Check before escalating.

Following one token

Recent Kubernetes versions add a credential identifier (the token's JTI) to user.extra for service account tokens. When it is present, it lets you tell apart two tokens of the same service account: the one the pod uses and the one the attacker copied or minted. Pivot on it the same way as on an IP.

For a TokenRequest, the request body (logged at Request level on EKS and in the recommended policy) shows expirationSeconds and the audiences. A token requested for a year (31536000) by an interactive client is a persistence step, not a workload refresh. The API server can cap token lifetimes with --service-account-max-token-expiration.

Scoping and remediation

1. List every secret read by the compromised identities, including those in cluster-wide list responses: for a cluster-wide list, assume every secret was exposed.
2. Rotate them at the source: the database password, the cloud access key, the registry credential, not just the Kubernetes Secret object.
3. Invalidate tokens. Bound tokens die with the object they are bound to; tokens minted with TokenRequest remain valid until they expire unless you delete the service account. Deleting and recreating the service account, then restarting its workloads, is the reliable way out. Delete legacy token Secrets.
4. Find the entry point: the pod that leaked the token, the dashboard, the CI job.
5. Cut the path: automountServiceAccountToken: false where the API is not needed, least-privilege roles with no list on secrets, and no create on serviceaccounts/token outside controllers.

Check the cloud side too: a stolen cloud key found in a Secret is used against the cloud API, not Kubernetes. The sibling sites AWS Forensics, GCP Forensics and Azure Forensics cover those logs.

Related articles

• How to detect kubectl exec, attach and port-forward
• RBAC privilege escalation: bind, escalate, impersonate
• Kubernetes incident response with audit logs