RBAC cannot see inside your YAML, and that is the whole problem

Every cluster I have picked up had RBAC configured and nothing after it. Service accounts scoped, roles reviewed, bindings tidy. And a privileged root container running in production, created entirely within the rules.

That is not an RBAC failure. RBAC answered the question it was asked: may this identity create a Pod in this namespace. Yes. What was in the Pod was never part of the question.

The gap between “who may act” and “what may exist” is where admission control lives, and it is the part most teams skip.

What Kyverno is

A policy engine that runs as a Kubernetes admission webhook. The API server calls it while a request is in flight, and Kyverno can validate the object, mutate it, or generate companion resources from it.

The thing that made it stick for me was that policies are plain Kubernetes YAML. No separate language to learn, which matters when the person maintaining the policy is the same person on call at 3am. That was the standing argument for Kyverno over OPA Gatekeeper, where you also have to learn Rego.

Worth knowing before you invest: that advantage is being walked back. As of 1.19 the ClusterPolicy API used below is deprecated in favour of CEL-based policies. More on that at the end, because you will see the warning the moment you apply anything.

What each layer can see

A request does not go straight into the cluster. It walks a chain, and each stage can only see certain things:

Authentication asks who you are. RBAC asks whether that identity may perform this verb on this resource, in this namespace. Neither of them opens the object. That part is not a Kyverno detail, it is how the API server is built.

After authorization the request enters admission, and admission is not one thing. It has two layers with very different powers:

The mutating phase runs first: built-in mutators and mutating webhooks may rewrite the object. You already depend on this without thinking about it. ServiceAccount injects the account and its token into every Pod that never asked for one, and LimitRanger fills in the default requests and limits you left out. By the time anything validates your Pod, it is not the Pod you submitted. Then the API server checks the result against the schema. Then the validating phase decides whether to accept it.

Both kinds of checker live in that validating phase side by side, and the difference between them is what they know:

Built-in plugins ship with Kubernetes and are already running. The one that matters here is PodSecurity, on by default since 1.25. It reads the whole spec, but it knows exactly one ruleset: the Pod Security Standards. You pick a level per namespace and that is the entire vocabulary.

Admission webhooks are external services the API server calls out to. They see the same complete object and answer with rules you wrote yourself. Kyverno registers in both phases: mutating webhooks that rewrite, validating webhooks that accept or reject.

So it is not a ladder of three separate stages. It is one authorization step that cannot see the body, followed by one admission stage where a fixed built-in standard and your own rules both get a vote.

So “we have RBAC” is not an answer to “how do you stop privileged containers”. But neither is “install Kyverno”, until you know what the built-in layer already covers for free. The walkthrough below goes through all three in order.

Inside the mutating phase, order matters

The diagram above draws mutate as one dot. It is not. It is a queue, and the queue is where the confusing behaviour comes from.

Three consequences worth knowing before you ship a mutating policy.

A webhook is not alone. Mutating webhooks are called one at a time, so whichever runs last sees an object the earlier ones have already rewritten. If your policy asserts something about a field another controller injects, the answer depends on who went first.

The order is by configuration name, and that is not a contract. Configurations are processed in name order, and webhooks inside one run in the order they are listed. Relying on that is relying on an implementation detail, so write rules that hold whatever the order turns out to be.

Reinvocation is why a webhook can run twice. The default reinvocationPolicy: Never calls each webhook once. With IfNeeded, a webhook is called at least one more time if another plugin modified the object after its first call. Read that bound carefully: the API server does not promise the extra invocation happens exactly once, and it is free to reorder webhooks to keep the number down. So the rule is not “handle a second pass”, it is “be idempotent”, which is what the field documentation demands in as many words. If you need to judge the object after every mutation is settled, that is a validating webhook, not a mutating one.

One thing deliberately not drawn here is failurePolicy, because it is not a position in the chain but what happens when the webhook does not answer at all. It has its own note further down.

Prove it yourself

Everything below runs on a laptop. No cloud account, no cost. If you have never used Kubernetes, this is a reasonable first hour with it.

If you only want to know what admission would say about a manifest you already have, the audit tool runs the same two profiles in the browser, no cluster involved. The walkthrough below is the version that actually proves it.

You need Docker running, plus kind and kubectl. On macOS:

brew install kind kubernetes-cli

1. A throwaway cluster

kind create cluster --name policy-demo
kubectl cluster-info --context kind-policy-demo

kind runs a full Kubernetes cluster inside Docker containers. It exists until you delete it, and it cannot touch anything real.

2. Confirm the gap

Before installing anything, create the container that should not exist:

kubectl run rooty --image=nginx:latest \
  --overrides='{"spec":{"containers":[{"name":"rooty","image":"nginx:latest","securityContext":{"privileged":true,"runAsUser":0}}]}}'

pod/rooty created

Created. Privileged, running as UID 0, on the latest tag. You are cluster-admin here, so RBAC had no reason to object, and nothing else looked.

Clean up:

kubectl delete pod rooty

3. Close the easy half without installing anything

Kubernetes already ships an admission plugin for exactly this. It is off in the sense that no namespace opts into it, not in the sense that it is missing. One label turns it on:

kubectl create namespace demo
kubectl label namespace demo pod-security.kubernetes.io/enforce=restricted

Now try the same pod in that namespace:

kubectl run rooty -n demo --image=nginx:latest \
  --overrides='{"spec":{"containers":[{"name":"rooty","image":"nginx:latest","securityContext":{"privileged":true,"runAsUser":0}}]}}'

Error from server (Forbidden): pods "rooty" is forbidden:
violates PodSecurity "restricted:latest": privileged (container "rooty" must not set
securityContext.privileged=true), allowPrivilegeEscalation != false, unrestricted
capabilities, runAsNonRoot != true, runAsUser=0, seccompProfile

Blocked, with a list of every standard it broke. No installation, no webhook, no extra component to keep alive. If your requirement is “follow the Pod Security Standards”, you are done here and you do not need a policy engine.

That is worth saying plainly, because a lot of Kyverno tutorials use this exact example and never mention that Kubernetes solves it on its own.

4. Find the edge of the built-in layer

PodSecurity knows one vocabulary. Anything outside the Pod Security Standards is invisible to it. Image tags, for instance.

Build a pod that satisfies restricted completely, and pin it to latest:

kubectl run webapp -n demo --image=nginx:latest \
  --overrides='{"spec":{"containers":[{"name":"webapp","image":"nginx:latest","securityContext":{"runAsNonRoot":true,"runAsUser":1000,"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"seccompProfile":{"type":"RuntimeDefault"}}}]}}'

pod/webapp created

Accepted. The security context is spotless, and the image is a moving target that will silently become a different build on the next pull. PodSecurity has no opinion about that, because tags are not part of the standard it enforces.

This is the actual case for a policy engine: not “RBAC is not enough”, but “the built-in standard does not cover my rule”.

kubectl delete pod webapp -n demo

5. Install Kyverno

kubectl create -f https://github.com/kyverno/kyverno/releases/download/v1.19.0/install.yaml
kubectl wait --for=condition=Ready pod -l app.kubernetes.io/component=admission-controller -n kyverno --timeout=180s

Check that the webhook actually registered. This is the part that does the work:

kubectl get validatingwebhookconfiguration | grep kyverno

kyverno-cel-exception-validating-webhook-cfg
kyverno-cleanup-validating-webhook-cfg
kyverno-exception-validating-webhook-cfg
kyverno-global-context-validating-webhook-cfg
kyverno-policy-validating-webhook-cfg
kyverno-resource-validating-webhook-cfg
kyverno-ttl-validating-webhook-cfg

Seven configurations, and the one that does the work here is kyverno-resource-validating-webhook-cfg. If that line is missing, Kyverno is installed but not wired in, and every policy you write will be silently ignored.

The admission controller takes a while to report ready, because it generates its certificates and registers those webhooks on first start. On my run it needed about five minutes on a fresh kind cluster, and the first kubectl wait timed out before it settled. That is normal, run the wait again.

6. A policy, in audit mode first

The rule PodSecurity could not express: no latest tags.

cat <<'EOF' | kubectl apply -f -
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disallow-latest-tag
spec:
  validationFailureAction: Audit
  background: true
  rules:
    - name: require-explicit-tag
      match:
        any:
          - resources:
              kinds: [Pod]
      exclude:
        any:
          - resources:
              namespaces: [kube-system, kyverno, local-path-storage]
      validate:
        message: "Images must use an explicit tag, not latest."
        pattern:
          spec:
            containers:
              - image: "!*:latest"
EOF

Applying it prints a warning before the confirmation:

Warning: ClusterPolicy (kyverno.io) is deprecated and will be removed in a future
release; migrate to ValidatingPolicy, MutatingPolicy, GeneratingPolicy or
ImageValidatingPolicy (policies.kyverno.io)
clusterpolicy.kyverno.io/disallow-latest-tag created

Ignore it for now, the policy works. It is covered at the end.

Two details in the manifest matter more than the rule itself.

validationFailureAction: Audit reports violations without blocking anything. Enforcing a brand new policy on a live cluster is how you find out which system component you just broke, during an incident, at the worst possible moment.

The exclude block keeps system namespaces out of scope. Without it a policy can block the components that keep the cluster alive, including the ones you need to remove the policy. local-path-storage is in the list because that is kind’s storage provisioner, and it runs an image the rule would flag.

Create the same pod as before and read the report instead of an error:

kubectl run webapp -n demo --image=nginx:latest \
  --overrides='{"spec":{"containers":[{"name":"webapp","image":"nginx:latest","securityContext":{"runAsNonRoot":true,"runAsUser":1000,"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"seccompProfile":{"type":"RuntimeDefault"}}}]}}'

kubectl get policyreports -n demo

NAME                                   KIND   NAME     PASS   FAIL   WARN   ERROR   SKIP   AGE
3c01d18a-99a2-4435-acdb-b493be964f16   Pod    webapp   0      1      0      0       0      0s

One failure, nothing blocked. Reports are per namespace, so narrow the query down rather than reading every system namespace at once. This is the state a policy should live in for a few days on a real cluster.

7. Switch to enforce

kubectl patch clusterpolicy disallow-latest-tag \
  --type merge -p '{"spec":{"validationFailureAction":"Enforce"}}'

kubectl delete pod webapp -n demo
kubectl run webapp -n demo --image=nginx:latest \
  --overrides='{"spec":{"containers":[{"name":"webapp","image":"nginx:latest","securityContext":{"runAsNonRoot":true,"runAsUser":1000,"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"seccompProfile":{"type":"RuntimeDefault"}}}]}}'

Error from server: admission webhook "validate.kyverno.svc-fail" denied the request:

resource Pod/demo/webapp was blocked due to the following policies

disallow-latest-tag:
  require-explicit-tag: 'validation error: Images must use an explicit tag, not latest.
    rule require-explicit-tag failed at path /spec/containers/0/image/'

The request never reached etcd. Same namespace, same security context that PodSecurity was perfectly happy with. The only thing that changed is that a rule you wrote finally got to look.

8. Tear down

kind delete cluster --name policy-demo

What this buys you

A rule that lived in a wiki page now lives in the cluster. The difference shows up in the failure mode: a documented rule fails silently months later when somebody who never read the wiki ships a manifest, and an enforced rule fails immediately, in the pull request, with a message naming the fix.

It also changes what you can say during an audit. “We ask teams to pin image tags” and “the cluster rejects unpinned images, here is the policy and here is the report history” are different claims.

The layering matters as much as the tooling. PodSecurity costs one label and no moving parts, so it should carry everything it can. A policy engine is a second component to run, upgrade and page someone about, and it earns that only for the rules the built-in layer cannot express.

What to watch out for

Enforce is a production change. Audit first, read the reports, then switch. Treat the switch itself as a deploy, not as a config tweak.

Exclude system namespaces from day one. A policy that blocks kube-system can lock you out of fixing it. The exclusion above is the minimum.

Understand failurePolicy. With the default Fail, an unavailable Kyverno means the API server rejects everything the policy matches. Run at least two replicas, and use Ignore for rules where a policy outage is worse than a missed check.

Mutation fights GitOps. If Kyverno adds a field and Argo CD compares against git, the app oscillates between synced and out of sync. Declare those paths in ignoreDifferences before you ship a mutating policy.

Do not spend a policy engine on what a label already does. If the rule is inside the Pod Security Standards, PodSecurity enforces it with nothing to install and nothing to keep alive. Reach for Kyverno when the rule falls outside that vocabulary, as the tag rule above does.

A registered webhook is not a working policy. The webhook can be present while a policy matches nothing, because of a namespace selector, a typo in kinds, or a pattern that is valid YAML and wrong. The only proof is the negative test: try to create the thing that should be rejected.

The API under your feet is moving

The ClusterPolicy above works, and it is what almost every tutorial and Stack Overflow answer still shows. It is also on the way out. Kyverno 1.19 ships a second policy API in the policies.kyverno.io group with ValidatingPolicy, MutatingPolicy, GeneratingPolicy and ImageValidatingPolicy, and rules there are written in CEL rather than YAML patterns.

The same rule in the new API:

apiVersion: policies.kyverno.io/v1
kind: ValidatingPolicy
metadata:
  name: disallow-privileged
spec:
  validationActions: [Deny]
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE", "UPDATE"]
        resources: ["pods"]
  matchConditions:
    - name: skip-system-namespaces
      expression: "!(request.namespace in ['kube-system', 'kyverno'])"
  validations:
    - expression: >-
        !object.spec.containers.exists(c,
          has(c.securityContext) && has(c.securityContext.privileged) && c.securityContext.privileged)
      message: "Privileged containers are not allowed."

It blocks the same pod:

Error from server: admission webhook
"vpol.validate.kyverno.svc-fail-finegrained-disallow-privileged" denied the request:
Policy disallow-privileged failed: Privileged containers are not allowed.

Two things to notice. System namespaces are no longer excluded with an exclude block but with a CEL matchCondition, and I lost twenty minutes to that: my first attempt kept the old excludeResourceRules shape, excluded exactly what it matched, and produced a policy that silently checked nothing. kubectl get vpol <name> -o jsonpath='{.status}' reporting generated: false is what gave it away.

The second thing is the bigger one. !object.spec.containers.exists(c, has(c.securityContext) && ...) is an expression language. The argument that Kyverno spares you from learning one is now on a deprecation timer.

If you are picking a policy engine today, pick it on the CEL API, not on the YAML patterns, because that is what you will be maintaining.

Takeaway

RBAC and admission control answer different questions, and only one of them opens the manifest. If your threat model includes what runs, not just who deploys, RBAC alone leaves the interesting half unguarded.

Whichever engine you choose, the rule that matters is the boring one: run it in audit until the reports are quiet, and only then let it say no.

Verified end to end on Kyverno 1.19.0, Kubernetes 1.37.0 via kind 0.33.0, September 2026. Every command and every output above was run, not reconstructed.