Mirage

Mirage makes a Kubernetes controller believe it has cluster-wide access when it only has access to one namespace.

Plenty of operators are written and packaged on the assumption that they own the cluster: their charts create ClusterRoles, and their code lists and watches resources across all namespaces with no option to narrow the scope. If you only have permissions in a single namespace — a shared, multi-tenant cluster, say — those operators simply will not start.

Mirage runs as a sidecar next to such a controller and confines its API requests to a single namespace. A cluster-wide LIST of Builds becomes a LIST of Builds in one namespace. The controller cannot tell the difference, and nothing about the cluster has to change.

Technical decisions are recorded in docs/adr/ and the domain language in CONTEXT.md.

How it works

The controller — the Client — is pointed at Mirage with a mounted kubeconfig instead of the real API server. Mirage inspects each request and does one of three things:

Decision	Behaviour
Confine	The path is a collection path for a resource listed under confined and names no namespace, so Mirage inserts the Target Namespace and forwards it. The Client's "every namespace" becomes "this namespace".
Mask	The resource is listed under masked. Mirage answers itself, always reporting it as existing and empty, and never contacts the API server.
Pass Through	Everything else is forwarded byte-for-byte, unchanged.

Confining is a URL-only edit, and only ever an insertion. A request that already names a namespace is left exactly as it arrived — even one naming somebody else's namespace — so the API server stays the only thing deciding whether it is allowed. Requests for a single object or a subresource are left alone too: they cannot be namespace-less in the first place.

Two properties are worth knowing before you rely on it:

Mirage never adds authority. It forwards the Client's own Authorization header untouched and holds no credentials of its own. The API server remains the sole enforcement point, so the worst a broken Mirage can do is cause the Client to receive 403s — it can never grant access the Client's ServiceAccount did not already have.

Mirage only confines what you list. Anything not named in the config is passed through unchanged. A forgotten entry therefore fails exactly as it would have without Mirage — a 403 from the API server — rather than silently exposing another tenant's namespace.

Configuration

# A cluster-wide request for one of these has the Target Namespace inserted into its path.
confined:
  - {group: "",            plural: pods}          # "" is the core group
  - {group: tekton.dev,    plural: taskruns}
  - {group: shipwright.io, plural: builds}
  - {group: shipwright.io, plural: buildruns}
  - {group: shipwright.io, plural: buildstrategies}

# Mirage answers these itself, always as existing but empty. Never forwarded.
masked:
  - {group: shipwright.io, plural: clusterbuildstrategies, kind: ClusterBuildStrategy}

A resource may appear under one key or the other, never both — Mirage refuses to start otherwise, rather than let its behaviour depend on an ordering you cannot see. Listing a cluster-scoped resource under confined is refused for the same reason: there is no namespaced path to confine it into, so namespaces in particular is rejected with a message pointing at masked instead.

plural is the lowercase plural name as it appears in the URL and in a CRD's spec.names.plural — builds, not Build. That is what a request path carries, so it is what Mirage matches on.

API versions are deliberately not part of the config — an entry applies to every version of that resource, so a controller reading both v1alpha1 and v1beta1 needs one entry, not two. kind is required for masked entries only, so Mirage can name the empty list it synthesises (ClusterBuildStrategyList).

The Target Namespace is always the namespace Mirage's own Pod runs in, read from the projected ServiceAccount volume. The Upstream API server is derived from Mirage's own KUBERNETES_SERVICE_HOST / KUBERNETES_SERVICE_PORT. Neither is configurable.

Installing

Mirage ships as a container image plus the patch below. There is no Helm chart and no operator.

Requirements

• Kubernetes 1.29+ / OpenShift 4.16+, for native sidecar support.
• The Client must honour KUBECONFIG — true of anything using controller-runtime's GetConfig(). A Client calling rest.InClusterConfig() directly will ignore it.
• The Client must not be installed by OLM, which reverts sidecar patches. See ADR 0005.

1. Create the ConfigMap holding Mirage's config and the kubeconfig the Client will use:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mirage-config
data:
  config.yaml: |
    confined:
      - {group: shipwright.io, plural: builds}
      # ... as above
  kubeconfig: |
    apiVersion: v1
    kind: Config
    current-context: mirage
    clusters:
      - name: mirage
        cluster:
          # https, not http — clientcmd reads a kubeconfig's credentials only for
          # an https server, and drops them silently otherwise. Mirage generates a
          # self-signed certificate at startup and nothing verifies it, so
          # verification is skipped. See ADR 0002.
          server: https://127.0.0.1:8001
          insecure-skip-tls-verify: true
    users:
      - name: mirage
        user:
          tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
    contexts:
      - name: mirage
        context:
          cluster: mirage
          user: mirage

2. Patch the Client's Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: shipwright-build-controller
spec:
  template:
    spec:
      volumes:
        - name: mirage-config
          configMap:
            name: mirage-config
      initContainers:
        - name: mirage
          image: ghcr.io/OWNER/mirage:TAG
          restartPolicy: Always          # makes this a native sidecar
          args: ["--config", "/etc/mirage/config.yaml"]
          volumeMounts:
            - name: mirage-config
              mountPath: /etc/mirage
          startupProbe:
            # host is required: without it the kubelet probes the Pod IP, and
            # Mirage binds loopback only. scheme is required for the same kind of
            # reason — the listener is HTTPS.
            httpGet: {path: /healthz, port: 8001, host: 127.0.0.1, scheme: HTTPS}
          securityContext:
            runAsNonRoot: true
            allowPrivilegeEscalation: false
            capabilities: {drop: [ALL]}
            seccompProfile: {type: RuntimeDefault}
      containers:
        - name: shipwright-build-controller
          env:
            - name: KUBECONFIG
              value: /etc/mirage/kubeconfig
          volumeMounts:
            - name: mirage-config
              mountPath: /etc/mirage

restartPolicy: Always on an init container is what makes it a native sidecar: the kubelet starts Mirage and waits for its startupProbe before the Client starts, and shuts it down only after the Client has exited. Both matter — without the first the Client crash-loops on startup, and without the second the Client's final API calls fail, including releasing its leader-election Lease.

Do not remove or remap the default ServiceAccount mount. It is load-bearing in three independent ways: the kubeconfig reads the token from it, Mirage reads ca.crt from it to reach the API server, and controller-runtime's leader election finds its namespace by reading the namespace file directly. Nothing warns you if it disappears.

Limitations

• One namespace only. Mirage cannot present a set of namespaces as the cluster; doing so would require merging list and watch streams, which is a different program. See ADR 0003.
• Masking works for custom resources only. Mirage answers masked requests in JSON and returns 406 for protobuf, mirroring how the API server treats CRDs. Built-in types, where client-go negotiates protobuf and will not fall back, would need protobuf support first.
• Masking requires the CRD to exist in the cluster. Discovery is Passed Through, so a Masked Resource whose CRD is not installed does not appear in discovery either — a typed client's RESTMapper cannot resolve the type and errors before it ever issues a request Mirage could answer. Masking works when the resource exists cluster-wide and the Deployer merely cannot read it, which is the Shipwright case. It is not a way to invent a resource the cluster has never heard of.
• Cluster-scoped resources you actually need cannot be faked. Masking tells the Client the resource is empty. For Shipwright specifically this means every Build in the namespace must use kind: BuildStrategy; a Build referencing a ClusterBuildStrategy will never resolve its strategy. Most Shipwright examples use ClusterBuildStrategy, so this is the first mistake people make.

Debugging

Mirage logs its full resolved configuration at startup — Target Namespace, Upstream, and every confined and masked entry — so the first ten lines of its logs tell you what it actually loaded.

Run with debug logging for one structured line per request showing the decision, the inbound path and the outbound path.

If the Client is getting unexplained 403s, look for Mirage's warnings about cluster-wide requests that were passed through: that is the signature of a resource missing from confined.

Check "authorization": false on those same lines first, though. It means the request reached Mirage with no Authorization header and will arrive at the API server as system:anonymous, so every request 403s rather than just the unconfigured ones — usually because the Client is not reading the kubeconfig, or is reading one whose server is not https://. Mirage forwards that header untouched and never logs its value.

A 5xx is Mirage's own, since Upstream's statuses are forwarded unchanged; those are logged at error level with the cause, whatever the log level. Note that a Client using client-go reports any failed discovery request as the single word unknown regardless of what actually happened, so Mirage's logs, not the Client's, are where the answer is.

Development

just lists everything. Two test tiers:

just test              # the unit suites — fast, no external anything
just test-integration  # Mirage against a real etcd and kube-apiserver

The integration suite is behind the integration build tag, so just test neither compiles nor runs it — via the tag plus a --skip-package, since ginkgo -r discovers suites by filename before build constraints apply. just test-integration downloads the control-plane binaries with setup-envtest (~150 MB, cached under ~/.local/share/kubebuilder-envtest) and puts them on KUBEBUILDER_ASSETS for you. Linux and macOS, no Docker, no root. The Kubernetes version is pinned in the justfile; see ADR 0007.

It deliberately does not skip itself when the binaries are missing — that would be a green tick over the one test standing between an Echo upgrade and a silent production failure.

Verbosity

Extra flags reach Ginkgo, so just test-integration -vv --focus=RBAC works. Verbosity comes in layers, loudest last:

(default)	Spec names only. Mirage's own logs are captured and printed for failing specs only.
-v	Streams Mirage's logs as they happen — one line per request with the decision, the inbound path and the outbound path. Usually the layer you want.
-vv	Adds Ginkgo's own node-by-node tracing.
KUBEBUILDER_ATTACH_CONTROL_PLANE_OUTPUT=true	etcd and kube-apiserver's logs. Thousands of lines, and on stdout rather than nested under the spec, so it interleaves. Reach for it when the question is why the API server refused something Mirage forwarded faithfully.

MIRAGE_TEST_LOG_LEVEL (debug by default, as above) turns Mirage's own logging down when its volume is what is in the way.

License

Apache License 2.0 — see LICENSE.

Copyright 2026 Swiss Data Science Center, EPFL and ETH Zurich.