unix: support building Linux x86-64 from macOS aarch64#1056
Conversation
| host_platforms: | ||
| - linux_x86_64 | ||
| # Rosetta doesn't support AVX-512. So we cannot run x86-64-v4 binaries | ||
| # under Rosetta. But they can build correctly. |
There was a problem hiding this comment.
Should be moved to x86_64_v4-unknown-linux-musl
|
I tried this out locally and after building for For some reason armhf deb packages are being installed in the container and I think this is caused by docker-py not fully respecting the platform argument, see docker/docker-py#2990. There may be a workaround that involves removing cached images. |
|
That's a fun quirk. I'll take a look and try to concoct a workaround. I know a bit about Docker/Buildkit internals and may be able to devise something non-obvious. |
|
Yeah, I bet the code in |
|
Oh, I think it is sillier than that. We're adding |
Docker on aarch64 macOS will automagically virtualize x86-64 containers if containers are spawned with `platform=linux/amd64`. Performance of spawned containers is a bit slower than native, but not horrible. This functionality means it is viable to develop Linux x86-64 from modern Apple hardware. This commit teaches the build system to support cross-compiling Linux x86-64 from macOS aarch64. Implementing this wasn't too difficult: we need to pass `platform` into Docker's APIs for building and creating containers. We need to teach code to resolve the effective host platform when this scenario is detected. And we need to advertise support for cross-compiling in the `targets.yml` file. In case you are wondering, yes, a similar solution could be employed for Linux too by using emulation. But this requires Docker be configured to support emulation, which isn't common. Rosetta on macOS "just works" and is therefore the lowest hanging fruit to implement.
d1fddea to
affe720
Compare
|
I'm still getting armhf Debian packages installed into the |
Right - our build system is supposed to load the local image tags into Docker and then execute those images directly by specifying their SHA-256 digest. The only time we would be loading incorrect images is if the Try removing |
|
I tracked down why the wrong platform was being used on my Mac. client.api.build() uses Docker’s legacy builder rather than BuildKit. As described in docker/docker-py#2990, the legacy builder can incorrectly reuse a cached base-image variant despite an explicit platform argument. This can cause an image for the wrong architecture to be produced. A small reproducer is: #!/usr/bin/env -S uv run --no-dev
import io
import docker
IMAGE = "debian@sha256:32ad5050caffb2c7e969dac873bce2c370015c2256ff984b70c1c08b3a2816a0"
client = docker.from_env(timeout=600)
print("Seeding the local cache with linux/arm/v7...")
for _ in client.api.pull(IMAGE, platform="linux/arm/v7", stream=True, decode=True): pass
print("Requesting linux/amd64 through docker-py...")
fh = io.BytesIO(f"FROM {IMAGE}\nRUN uname -m && dpkg --print-architecture\n".encode())
for event in client.api.build(fileobj=fh, platform="linux/amd64", decode=True):
if "stream" in event:
print(event["stream"], end="")On my machine, this prints armv7l and armhf despite requesting linux/amd64. A workaround is to invoke docker buildx build in a subprocess so the image is built with BuildKit, which correctly resolves the requested platform. I’ve applied that workaround on top of this branch and submitted it as #1179. |
Update to #1056 to use `docker buildx` for reliable handeling of the `platform` argument. --------- Co-authored-by: Gregory Szorc <gregory.szorc@gmail.com>
2 participants
Docker on aarch64 macOS will automagically virtualize x86-64 containers if containers are spawned with
platform=linux/amd64. Performance of spawned containers is a bit slower than native, but not horrible. This functionality means it is viable to develop Linux x86-64 from modern Apple hardware.This commit teaches the build system to support cross-compiling Linux x86-64 from macOS aarch64.
Implementing this wasn't too difficult: we need to pass
platforminto Docker's APIs for building and creating containers. We need to teach code to resolve the effective host platform when this scenario is detected. And we need to advertise support for cross-compiling in thetargets.ymlfile.In case you are wondering, yes, a similar solution could be employed for Linux too by using emulation. But this requires Docker be configured to support emulation, which isn't as common. Rosetta on macOS "just works" and is therefore the lowest hanging fruit to implement.