The *-torch2 base images (torch2, ffmpeg-torch2, opencv4-torch2, and everything downstream including all *-hf5) ship PyTorch but no C toolchain.
As the torch team is progressively replacing hand-written per-architecture CUDA kernels with Triton-generated ones (more portable across GPUs, easier to maintain), recent torch (2.10, Jan 2026) routes some eager ops (those run op-by-op in PyTorch's default mode, i.e. not traced/compiled under torch.compile) through Triton kernels (via the torch._native op registry), and Triton compiles a small host-side CUDA driver shim at runtime (this step needs a C compiler + libc headers).
On the slim python:3.10-slim base those are absent, so GPU inference dies at the first model.generate():
RuntimeError: Failed to find C compiler ...
# and after adding gcc alone:
.../triton/backends/nvidia/include/cuda.h: fatal error: stdlib.h: No such file or directory
This can hit any torch-based app, e.g. the SmolVLM2 captioner.
The recent release of the smol2 app had to add these system packages to its own Containerfile; this issue proposes the same mitigation at the base-image level so every torch-based app gets it for free.
Mitigation (proposed)
Add the minimal toolchain to the three *-torch2 containerfiles, where torch is installed:
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc libc6-dev \
&& rm -rf /var/lib/apt/lists/*
gcc + libc6-dev is sufficient (verified); build-essential installs the full C toolchain but is not needed.
Future direction
This is a workaround, not a cure: we ship a runtime build toolchain in a slim inference image only because Triton compiles kernels on-device.
The PyTorch 2.13 notes, released last week, point the same way:
- PyTorch is adding more runtime codegen backends (CuTeDSL alongside Triton), so the "kernels compile on the user's machine" direction is deepening, not reversing -- but that work is Inductor /
torch.compile only, so it does not touch our eager path.
- No change yet to Triton's runtime-compiled driver shim (its own
# TODO: make static), so the compiler requirement persists.
- CUDA 13.0 is now the default build -- a separate driver-floor consideration for whatever CUDA version future base images track.
Another mitigation is to pin torch (<=2.9.*) so Triton is not required at all.
That is not desirable, though: pinning down would freeze the base images on an old torch and forgo the bug fixes, performance work, and transformers/CUDA compatibility the torch team ships in newer releases.
Ideally, upstream would ship a pre-compiled driver shim and pip install would resolve every dependency -- but until that happens, keeping a minimal C toolchain in the base is the best mitigation, IMHO.
The
*-torch2base images (torch2,ffmpeg-torch2,opencv4-torch2, and everything downstream including all*-hf5) ship PyTorch but no C toolchain.As the torch team is progressively replacing hand-written per-architecture CUDA kernels with Triton-generated ones (more portable across GPUs, easier to maintain), recent torch (2.10, Jan 2026) routes some eager ops (those run op-by-op in PyTorch's default mode, i.e. not traced/compiled under
torch.compile) through Triton kernels (via thetorch._nativeop registry), and Triton compiles a small host-side CUDA driver shim at runtime (this step needs a C compiler + libc headers).On the slim
python:3.10-slimbase those are absent, so GPU inference dies at the firstmodel.generate():This can hit any torch-based app, e.g. the SmolVLM2 captioner.
The recent release of the smol2 app had to add these system packages to its own Containerfile; this issue proposes the same mitigation at the base-image level so every torch-based app gets it for free.
Mitigation (proposed)
Add the minimal toolchain to the three
*-torch2containerfiles, where torch is installed:RUN apt-get update \ && apt-get install -y --no-install-recommends gcc libc6-dev \ && rm -rf /var/lib/apt/lists/*gcc+libc6-devis sufficient (verified);build-essentialinstalls the full C toolchain but is not needed.Future direction
This is a workaround, not a cure: we ship a runtime build toolchain in a slim inference image only because Triton compiles kernels on-device.
The PyTorch 2.13 notes, released last week, point the same way:
torch.compileonly, so it does not touch our eager path.# TODO: make static), so the compiler requirement persists.Another mitigation is to pin torch (
<=2.9.*) so Triton is not required at all.That is not desirable, though: pinning down would freeze the base images on an old torch and forgo the bug fixes, performance work, and
transformers/CUDA compatibility the torch team ships in newer releases.Ideally, upstream would ship a pre-compiled driver shim and
pip installwould resolve every dependency -- but until that happens, keeping a minimal C toolchain in the base is the best mitigation, IMHO.