Proxy-sidecar: cooperative egress mode opt-out for platforms without iptables (ROSA HCP)

[akram]

Problem

On ROSA HCP (and likely other managed OpenShift platforms), agent pods in proxy-sidecar mode are stuck in Init:CrashLoopBackOff because the proxy-init container cannot initialize iptables:

• iptables-legacy: iptable_nat module not loaded; SELinux blocks module load (RH KB 5990311)
• iptables (nft): Could not fetch rule set generation id: Permission denied
• Auto-detection in init-iptables.sh picks legacy when nft is required; probes pass individually but full nat pipeline fails in setup_enforce_redirect() (see Bug: init-iptables.sh picks wrong backend on nft-only nodes (readability probe ≠ liveness) #502)

SCC (kagenti-authbridge, NET_ADMIN + NET_RAW, runAsUser 0) is not the blocker — the pod is admitted. The limitation is structural: node kernel modules and SELinux policy are not under tenant control on managed OpenShift.

Impact: proxy-sidecar is the rhoai-3.5 default (kagenti#1428, kagenti-operator#361). Broken proxy-init blocks agent deployments on ROSA HCP.

Agreed direction

Per Slack discussion with @haihuang (thread): add explicit cooperative mode opt-out for platforms where iptables is unavailable. Document limitations clearly. Keep enforce-redirect as the default where iptables works.

Proposed solution

Add egressEnforcement configuration:

Value	Behavior
enforce-redirect (default)	Current behavior — inject proxy-init, iptables REDIRECT
none	Skip proxy-init; set HTTP_PROXY / HTTPS_PROXY only (cooperative)

Runtime behavior when none

1. Do not inject proxy-init init container
2. Authbridge sidecar still runs with forward/reverse proxy listeners
3. HTTP_PROXY env vars point agent traffic at authbridge
4. Cooperative agents → authenticated egress via token exchange
5. Non-cooperative agents → direct egress without token → destination inbound AuthBridge rejects (401)

Security model (must be documented)

What cooperative mode provides:

• HTTP_PROXY path for agents that respect proxy settings
• Inbound AuthBridge JWT validation on destination services
• Egress NetworkPolicy can restrict L4 destinations

Known limitations:

• NetworkPolicy is L4 — cannot distinguish authbridge vs agent traffic within the same pod network namespace
• Non-cooperative agents that target allowed destinations can exfiltrate without token exchange
• Admin must explicitly opt in and acknowledge the trade-off

Scope (this repo)

• Add egressEnforcement to auth config / mode presets (or equivalent)
• Skip proxy-init injection path when none
• Document security limitations in README / mode docs
• Default remains enforce-redirect

Out of scope (tracked separately)

• seLinuxOptions in securityContext to enable iptables nft on OpenShift
• Separate-pod authbridge-proxy for pod-level NetworkPolicy enforcement
• kagenti-operator CRD/webhook changes (tracked in RHAIENG-5702)

Acceptance criteria

1. egressEnforcement: none → no proxy-init injected; pods can start on ROSA HCP
2. Default unchanged (enforce-redirect)
3. Security limitations documented at opt-in point

References

• Red Hat Jira: RHAIENG-5702
• Related: Bug: init-iptables.sh picks wrong backend on nft-only nodes (readability probe ≠ liveness) #502 (iptables backend detection), feat(cmd): Add authbridge-proxy and authbridge-envoy lite binaries #407 (enforce-redirect), feat: Add transparent listener + enforce-redirect for proxy-sidecar egress #485 (redirect-not-drop)
• Slack: enforce-redirect thread

[huang195]

closing it for now as this is not our priority at the moment