diff --git a/content/manuals/build-cloud/ci.md b/content/manuals/build-cloud/ci.md index 110853ecd513..b19d4b83ba83 100644 --- a/content/manuals/build-cloud/ci.md +++ b/content/manuals/build-cloud/ci.md @@ -23,9 +23,10 @@ registry directly, rather than loading the image and then pushing it. Pushing directly speeds up your builds and avoids unnecessary file transfers. If you just want to build and discard the output, export the results to the -build cache or build without tagging the image. When you use Docker Build Cloud, -Buildx automatically loads the build result if you build a tagged image. -See [Loading build results](./usage/#loading-build-results) for details. +build cache or build without tagging the image. With no explicit output, Buildx +leaves an untagged result in the cloud build cache and automatically loads +eligible tagged images. See +[Loading build results](./usage/#loading-build-results) for details. > [!NOTE] > @@ -67,6 +68,17 @@ If you are not an organization administrator: ## CI platform examples +The following examples require Docker CLI with Buildx version 0.37.0 or later, +which includes the `cloud` driver. Check the Buildx version available on your CI +runner: + +```console +$ docker buildx version +``` + +If the runner doesn't include a compatible version, install Buildx as a +[Docker CLI plugin](https://github.com/docker/buildx#manual-download). + > [!NOTE] > > In your CI/CD configuration, set the following variables/secrets: @@ -78,6 +90,8 @@ If you are not an organization administrator: ### GitHub Actions + + ```yaml name: ci @@ -140,19 +154,12 @@ For more information about the `BUILDX_BUILDER` environment variable, see ```yaml default: - image: docker:24-dind + image: docker:cli services: - - docker:24-dind + - docker:dind before_script: - docker info - echo "$DOCKER_ACCESS_TOKEN" | docker login --username "$DOCKER_ACCOUNT" --password-stdin - - | - apk add curl jq - ARCH=${CI_RUNNER_EXECUTABLE_ARCH#*/} - BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))") - mkdir -vp ~/.docker/cli-plugins/ - curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL - chmod a+x ~/.docker/cli-plugins/docker-buildx - docker buildx create --use --driver cloud ${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME} variables: @@ -195,13 +202,6 @@ jobs: steps: - checkout - - run: | - mkdir -vp ~/.docker/cli-plugins/ - ARCH=amd64 - BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))") - curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL - chmod a+x ~/.docker/cli-plugins/docker-buildx - - run: echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin - run: docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}" @@ -218,13 +218,6 @@ jobs: steps: - checkout - - run: | - mkdir -vp ~/.docker/cli-plugins/ - ARCH=amd64 - BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))") - curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL - chmod a+x ~/.docker/cli-plugins/docker-buildx - - run: echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin - run: docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}" @@ -281,28 +274,6 @@ steps: Create the `build.sh` script: ```bash -DOCKER_DIR=/usr/libexec/docker - -# Get download link for latest buildx binary. -# Set $ARCH to the CPU architecture (e.g. amd64, arm64) -UNAME_ARCH=`uname -m` -case $UNAME_ARCH in - aarch64) - ARCH="arm64"; - ;; - amd64) - ARCH="amd64"; - ;; - *) - ARCH="amd64"; - ;; -esac -BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))") - -# Download docker buildx with Build Cloud support -curl --silent -L --output $DOCKER_DIR/cli-plugins/docker-buildx $BUILDX_URL -chmod a+x ~/.docker/cli-plugins/docker-buildx - # Connect to your builder and set it as the default builder docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}" @@ -328,7 +299,6 @@ pipeline { agent any environment { - ARCH = 'amd64' DOCKER_ACCESS_TOKEN = credentials('docker-access-token') DOCKER_ACCOUNT = credentials('docker-account') CLOUD_BUILDER_NAME = '' @@ -337,13 +307,7 @@ pipeline { stages { stage('Build') { - environment { - BUILDX_URL = sh (returnStdout: true, script: 'curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\\"linux-$ARCH\\"))"').trim() - } steps { - sh 'mkdir -vp ~/.docker/cli-plugins/' - sh 'curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL' - sh 'chmod a+x ~/.docker/cli-plugins/docker-buildx' sh 'echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin' sh 'docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}"' // Cache-only build @@ -372,12 +336,7 @@ env: before_install: | echo "$DOCKER_ACCESS_TOKEN" | docker login --username "$DOCKER_ACCOUNT" --password-stdin -install: | - set -e - BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$TRAVIS_CPU_ARCH\"))") - mkdir -vp ~/.docker/cli-plugins/ - curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL - chmod a+x ~/.docker/cli-plugins/docker-buildx +before_script: | docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}" script: | @@ -393,18 +352,13 @@ script: | # Prerequisites: $DOCKER_ACCOUNT, $CLOUD_BUILDER_NAME, $DOCKER_ACCESS_TOKEN setup as deployment variables # This pipeline assumes $BITBUCKET_REPO_SLUG as the image name -image: atlassian/default-image:3 +image: docker:cli pipelines: default: - step: name: Build multi-platform image script: - - mkdir -vp ~/.docker/cli-plugins/ - - ARCH=amd64 - - BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))") - - curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL - - chmod a+x ~/.docker/cli-plugins/docker-buildx - echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin - docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}" - IMAGE_NAME=$BITBUCKET_REPO_SLUG @@ -421,15 +375,6 @@ pipelines: ```bash #!/bin/bash -# Get download link for latest buildx binary. Set $ARCH to the CPU architecture (e.g. amd64, arm64) -ARCH=amd64 -BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))") - -# Download docker buildx with Build Cloud support -mkdir -vp ~/.docker/cli-plugins/ -curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL -chmod a+x ~/.docker/cli-plugins/docker-buildx - # Login to Docker Hub with an access token. See https://docs.docker.com/build-cloud/ci/#creating-access-tokens echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin @@ -452,35 +397,19 @@ docker buildx build \ ### Docker Compose -Use this implementation if you want to use `docker compose build` with -Docker Build Cloud in CI. +This example requires Buildx version 0.37.0 or later and a Docker Compose +version that supports Docker Build Cloud. Set the cloud builder as the default +builder before running `docker compose build` in CI: ```bash #!/bin/bash -# Get download link for latest buildx binary. Set $ARCH to the CPU architecture (e.g. amd64, arm64) -ARCH=amd64 -BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))") -COMPOSE_URL=$(curl -sL \ - -H "Accept: application/vnd.github+json" \ - -H "Authorization: Bearer " \ - -H "X-GitHub-Api-Version: 2022-11-28" \ - https://api.github.com/repos/docker/compose-desktop/releases \ - | jq "[ .[] | select(.prerelease==false and .draft==false) ] | .[0].assets.[] | select(.name | endswith(\"linux-${ARCH}\")) | .browser_download_url") - -# Download docker buildx with Build Cloud support -mkdir -vp ~/.docker/cli-plugins/ -curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL -curl --silent -L --output ~/.docker/cli-plugins/docker-compose $COMPOSE_URL -chmod a+x ~/.docker/cli-plugins/docker-buildx -chmod a+x ~/.docker/cli-plugins/docker-compose - # Login to Docker Hub with an access token. See https://docs.docker.com/build-cloud/ci/#creating-access-tokens echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin # Connect to your builder and set it as the default builder docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}" -# Build the image build +# Build the images docker compose build ``` diff --git a/content/manuals/build-cloud/setup.md b/content/manuals/build-cloud/setup.md index 88aca9d035b8..2e39063cb7d5 100644 --- a/content/manuals/build-cloud/setup.md +++ b/content/manuals/build-cloud/setup.md @@ -15,21 +15,28 @@ environment. To get started with Docker Build Cloud, you need to: -- Download and install Docker Desktop version 4.26.0 or later. +- Install Docker Desktop version 4.26.0 or later, or Buildx version 0.37.0 or + later. - Create a cloud builder on the [Docker Build Cloud Dashboard](https://app.docker.com/build/). - When you create the builder, choose a name for it (for example, `default`). You will use this name as `BUILDER_NAME` in the CLI steps below. ### Use Docker Build Cloud without Docker Desktop -To use Docker Build Cloud without Docker Desktop, you must download and install -a version of Buildx with support for Docker Build Cloud (the `cloud` driver). -You can find compatible Buildx binaries on the releases page of -[this repository](https://github.com/docker/buildx-desktop). +Buildx version 0.37.0 or later includes the +[`cloud` driver](/manuals/build/builders/drivers/cloud.md). To use Docker Build +Cloud without Docker Desktop, check your installed version: -If you plan on building with Docker Build Cloud using the `docker compose -build` command, you also need a version of Docker Compose that supports Docker -Build Cloud. You can find compatible Docker Compose binaries on the releases -page of [this repository](https://github.com/docker/compose-desktop). +```console +$ docker buildx version +``` + +If your Docker CLI installation doesn't include a compatible Buildx version, +[install Buildx](https://github.com/docker/buildx#manual-download) as a Docker +CLI plugin. + + +To use Docker Build Cloud with Docker Compose, you need Buildx version 0.37.0 or +later and a Docker Compose version that supports Docker Build Cloud. ## Steps diff --git a/content/manuals/build-cloud/usage.md b/content/manuals/build-cloud/usage.md index a6970aeceb16..b46b67360675 100644 --- a/content/manuals/build-cloud/usage.md +++ b/content/manuals/build-cloud/usage.md @@ -46,8 +46,8 @@ builder for the `docker buildx build` command. The `docker build` command still uses the `default` builder, unless you specify the `--builder` flag explicitly. If you use build scripts, such as `make`, that use the `docker build` command, -we recommend updating your build commands to `docker buildx build`. Alternatively, -you can set the [`BUILDX_BUILDER` environment +update your build commands to `docker buildx build`. Alternatively, you can set +the [`BUILDX_BUILDER` environment variable](/manuals/build/building/variables.md#buildx_builder) to specify which builder `docker build` should use. @@ -72,13 +72,16 @@ variable](/manuals/build/building/variables.md#buildx_builder) to select the clo ## Loading build results -Building with `--tag` loads the build result to the local image store -automatically when the build finishes. To build without a tag and load the -result, you must pass the `--load` flag. +When you don't specify an output, Buildx leaves an untagged build result in the +cloud build cache and doesn't load it into Docker Engine. If you use `--tag`, +Buildx automatically loads the image when the build targets a single platform +and runs on one cloud node. Use `--load` to request loading explicitly. -Loading the build result for multi-platform images is not supported. Use the -`docker buildx build --push` flag when building multi-platform images to push -the output to a registry. +The target Docker context determines which Docker Engine image store receives +the result. For details about loading behavior and other output configurations, +see [Load results from Docker Build Cloud](/manuals/build/exporters/_index.md#load-results-from-docker-build-cloud). + +To push a multi-platform image to a registry instead of loading it: ```console $ docker buildx build --builder cloud-- \ @@ -87,8 +90,8 @@ $ docker buildx build --builder cloud-- \ --push . ``` -If you want to build with a tag, but you don't want to load the results to your -local image store, you can export the build results to the build cache only: +To keep a tagged result in the build cache instead of loading it into Docker +Engine, use the `cacheonly` exporter: ```console $ docker buildx build --builder cloud-- \ diff --git a/content/manuals/build/_index.md b/content/manuals/build/_index.md index c703c678dfb0..664b52b2bba7 100644 --- a/content/manuals/build/_index.md +++ b/content/manuals/build/_index.md @@ -29,6 +29,10 @@ grid: description: Configure where and how you run your builds. icon: wrench-screwdriver link: /build/builders/drivers/ +- title: Docker Build Cloud + description: Run builds on managed cloud builders with a shared cache. + icon: /icons/logo-build-cloud.svg + link: /build-cloud/ - title: Exporters description: Export any artifact you like, not just Docker images. icon: arrow-up-on-square diff --git a/content/manuals/build/builders/drivers/_index.md b/content/manuals/build/builders/drivers/_index.md index a90261ca451a..f54bfbbf9468 100644 --- a/content/manuals/build/builders/drivers/_index.md +++ b/content/manuals/build/builders/drivers/_index.md @@ -1,7 +1,7 @@ --- title: Build drivers description: Build drivers are configurations for how and where the BuildKit backend runs. -keywords: build, buildx, driver, builder, docker-container, kubernetes, remote +keywords: build, buildx, driver, builder, cloud, docker-container, kubernetes, remote aliases: - /build/buildx/drivers/ - /build/building/drivers/ @@ -15,6 +15,7 @@ Buildx supports the following drivers: - `docker`: uses the BuildKit library bundled into the Docker daemon. - `docker-container`: creates a dedicated BuildKit container using Docker. +- `cloud`: connects to a managed builder in Docker Build Cloud. - `kubernetes`: creates BuildKit pods in a Kubernetes cluster. - `remote`: connects directly to a manually managed BuildKit daemon. @@ -25,25 +26,28 @@ provide more flexibility and are better at handling advanced scenarios. The following table outlines some differences between drivers. -| Feature | `docker` | `docker-container` | `kubernetes` | `remote` | -| :--------------------------- | :---------: | :----------------: | :----------: | :----------------: | -| **Automatically load image** | ✅ | | | | -| **Cache export** | ✅\* | ✅ | ✅ | ✅ | -| **Tarball output** | | ✅ | ✅ | ✅ | -| **Multi-arch images** | | ✅ | ✅ | ✅ | -| **BuildKit configuration** | | ✅ | ✅ | Managed externally | +| Feature | `docker` | `docker-container` | `cloud` | `kubernetes` | `remote` | +| :--------------------------- | :---------: | :----------------: | :--------------: | :----------: | :----------------: | +| **Automatically load image** | ✅ | | Conditional | | | +| **Cache export** | ✅\* | ✅ | ✅ | ✅ | ✅ | +| **Tarball output** | | ✅ | ✅ | ✅ | ✅ | +| **Multi-arch images** | | ✅ | ✅ | ✅ | ✅ | +| **BuildKit configuration** | | ✅ | Managed by Docker | ✅ | Managed externally | \* _The `docker` driver doesn't support all cache export options. See [Cache storage backends](/manuals/build/cache/backends/_index.md) for more information._ ## Loading to local image store -Unlike when using the default `docker` driver, images built using other drivers -aren't automatically loaded into the local image store. If you don't specify an -output, the build result is exported to the build cache only. +The `docker` driver automatically loads images into the local image store. +With Docker Build Cloud, an untagged result remains in the cloud build cache +when you don't specify an output. Using `--tag` instead automatically loads the +image when the build targets a single platform and runs on one cloud node. With +other drivers, the build result remains in the build cache if you don't specify +an output. -To build an image using a non-default driver and load it to the image store, - use the `--load` flag with the build command: +To build an image using a driver that doesn't load results automatically, use +the `--load` flag with the build command: ```console $ docker buildx build --load -t --builder=container . @@ -85,7 +89,8 @@ flag. Read about each driver: - - [Docker driver](./docker.md) - - [Docker container driver](./docker-container.md) - - [Kubernetes driver](./kubernetes.md) +- [Docker driver](./docker.md) +- [Docker container driver](./docker-container.md) +- [Cloud driver](./cloud.md) +- [Kubernetes driver](./kubernetes.md) - [Remote driver](./remote.md) diff --git a/content/manuals/build/builders/drivers/cloud.md b/content/manuals/build/builders/drivers/cloud.md new file mode 100644 index 000000000000..7d38fa300cd7 --- /dev/null +++ b/content/manuals/build/builders/drivers/cloud.md @@ -0,0 +1,30 @@ +--- +title: Cloud driver +description: The cloud driver connects Buildx to managed builders in Docker Build Cloud. +keywords: build, buildx, driver, builder, cloud, Docker Build Cloud +--- + +The Buildx cloud driver connects to managed BuildKit instances in Docker Build +Cloud. Docker provisions and maintains the builder infrastructure, shared build +cache, and native `linux/amd64` and `linux/arm64` nodes. + +The cloud driver requires Buildx version 0.37.0 or later and access to Docker +Build Cloud. + +## Synopsis + +Before connecting Buildx, create a cloud builder on the +[Docker Build Cloud Dashboard](https://app.docker.com/build/). Then sign in and +connect to the builder: + +```console +$ docker login +$ docker buildx create --driver cloud / +``` + +The command adds the cloud builder to your local Buildx configuration. It does +not create a builder in Docker Build Cloud. + +For service prerequisites and authentication, see +[Docker Build Cloud setup](/manuals/build-cloud/setup.md). For build examples, +see [Building with Docker Build Cloud](/manuals/build-cloud/usage.md). diff --git a/content/manuals/build/exporters/_index.md b/content/manuals/build/exporters/_index.md index 57f465191adc..a2c3f8a517ad 100644 --- a/content/manuals/build/exporters/_index.md +++ b/content/manuals/build/exporters/_index.md @@ -80,6 +80,34 @@ Images loaded to the image store are available to `docker run` immediately after the build finishes, and you'll see them in the list of images when you run the `docker images` command. +#### Load results from Docker Build Cloud + +Builds that use the [cloud driver](/manuals/build/builders/drivers/cloud.md) run +on remote Docker Build Cloud workers. You can still load an image into Docker +Engine so that it's available to `docker run` and `docker images` without first +pushing it to a registry. + +The [target Docker context](/manuals/engine/manage-resources/contexts.md) +determines which Docker Engine image store receives the image. A local context +loads it into a local image store. A remote context loads it into the remote +Docker daemon's image store. + +When you don't specify an output, Buildx leaves an untagged build result in the +cloud build cache and doesn't load it into Docker Engine. If you use `--tag`, +Buildx automatically loads the image when the build targets a single platform +and runs on one cloud node. Use `--load` to request loading explicitly. The +`--load` flag is shorthand for `--output type=docker`. + +To leave a tagged result in the build cache, use `--output type=cacheonly`. +Setting `default-load=false` doesn't turn off automatic loading for a tagged +cloud build with no explicit output. + +Buildx uses the standard exporter path when direct loading isn't available. +This includes multi-node or multi-platform builds, requests for more than one +distinct image output, outputs targeting different Docker contexts, and +outputs written to a file, standard output, or a directory. The standard path +also applies if the target context doesn't support direct loading. + ### Push to registry To push a built image to a container registry, you can use the `registry` or @@ -195,7 +223,7 @@ different exporters: - The `registry` exporter to push the image to a registry - The `local` exporter to extract the build results to the local filesystem -- The `--load` flag (a shorthand for the `image` exporter) to load the results to the local image store. +- The `--load` flag (a shorthand for the `docker` exporter) to load the results to the local image store. ```console $ docker buildx build \