Skip to content

feat: unify GPU permissions across deployments - #183

Merged
kerwtsai-amd merged 65 commits into
developfrom
fix/unify-gpu-permissions
Jul 31, 2026
Merged

feat: unify GPU permissions across deployments#183
kerwtsai-amd merged 65 commits into
developfrom
fix/unify-gpu-permissions

Conversation

@MioYuuIH

@MioYuuIH MioYuuIH commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • remove renderGid and GPU-specific supplementalGroups from Helm, Hub, Spawner, installer overlays, and generated deployment artifacts
  • replace AUPLC-maintained GPU udev rules with AMD official amdgpu-insecure-instinct-udev-rules package 30.30.4.0-2341068.24.04
  • use the package policy unchanged: /dev/kfd and DRM renderD* receive mode 0666; card* remains under the normal system policy
  • keep singleuser.fsGid: 100 strictly for notebook storage ownership
  • simplify SSH/PXE discovery to boolean GPU enablement and remove render-GID state, unanimity, and PXE finalizer flows
  • keep GPU images free of udev, chmod, and video/render group compatibility logic

Package deployment

  • single-node installs a checksummed pinned deb and verifies package status, version, ownership, and exact conffile content
  • offline bundles include the pinned deb
  • Ansible live hosts and PXE rootfs install and verify the same package
  • only exact shipped legacy AUPLC rules are removed, after package installation and verification
  • package-owned non-exact versions converge to the pinned version; unknown unowned rules fail closed

Infrastructure contract

The AMD device plugin and labeller remain separate cluster prerequisites. Device-plugin allocation controls which containers receive GPU devices; AUPLC only requests amd.com/gpu and does not inject Linux groups.

runtime verification

Tested directly on both Ready GPU nodes with a non-root UID 12345 process carrying only GID 100:

  • /dev/kfd and renderD* at mode 0666 were accessible
  • card* remained root:video 0660 and was intentionally inaccessible
  • rocminfo succeeded on aup-395-23 (gfx1151) and aup-sha-nv20 (gfx1200)
  • CPU Pods without amd.com/gpu received no GPU device nodes
  • temporary probe resources were removed and no host permissions were changed

Verification

  • 303 repository tests passed
  • 16 focused Hub configuration/spawner tests passed
  • Ruff lint and format passed
  • yamllint passed
  • Helm strict lint and template passed
  • Ansible syntax checks passed for ROCm, udev, and PXE controller playbooks
  • Oracle package-adoption review passed with no remaining HIGH/MEDIUM findings

Known baseline

The complete Hub test sweep still has three pre-existing collection failures when local test stubs do not provide jupyterhub.scopes; the changed Hub configuration and GPU Spawner tests pass.

@MioYuuIH
MioYuuIH marked this pull request as draft July 28, 2026 02:31

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MioYuuIH We provide deb package for setup kdf and dri permission, do we have a specific reason why we don't use this ?

sudo apt update
wget https://repo.radeon.com/amdgpu/30.30.4/ubuntu/pool/main/a/amdgpu-insecure-instinct-udev-rules/amdgpu-insecure-instinct-udev-rules_30.30.4.0-2341068.24.04_all.deb
sudo apt install ./amdgpu-insecure-instinct-udev-rules_30.30.4.0-2341068.24.04_all.deb

Reference

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Replaced custom udev rules with AMD's official package and verified it on both GPU nodes.

@kerwtsai-amd
kerwtsai-amd marked this pull request as ready for review July 29, 2026 03:27
@MioYuuIH
MioYuuIH requested a review from kerwtsai-amd July 29, 2026 04:09
@kerwtsai-amd
kerwtsai-amd merged commit 8f4d711 into develop Jul 31, 2026
88 of 89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants