Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 25 additions & 96 deletions content/manuals/build-cloud/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@ registry directly, rather than loading the image and then pushing it. Pushing
directly speeds up your builds and avoids unnecessary file transfers.

If you just want to build and discard the output, export the results to the
build cache or build without tagging the image. When you use Docker Build Cloud,
Buildx automatically loads the build result if you build a tagged image.
See [Loading build results](./usage/#loading-build-results) for details.
build cache or build without tagging the image. With no explicit output, Buildx
leaves an untagged result in the cloud build cache and automatically loads
eligible tagged images. See
[Loading build results](./usage/#loading-build-results) for details.

> [!NOTE]
>
Expand Down Expand Up @@ -67,6 +68,17 @@ If you are not an organization administrator:

## CI platform examples

The following examples require Docker CLI with Buildx version 0.37.0 or later,
which includes the `cloud` driver. Check the Buildx version available on your CI
runner:

```console
$ docker buildx version
```

If the runner doesn't include a compatible version, install Buildx as a
[Docker CLI plugin](https://github.com/docker/buildx#manual-download).

> [!NOTE]
>
> In your CI/CD configuration, set the following variables/secrets:
Expand All @@ -78,6 +90,8 @@ If you are not an organization administrator:

### GitHub Actions

<!-- TODO: Confirm whether standard Buildx requires a minimum setup-buildx-action version. -->

```yaml
name: ci

Expand Down Expand Up @@ -140,19 +154,12 @@ For more information about the `BUILDX_BUILDER` environment variable, see

```yaml
default:
image: docker:24-dind
image: docker:cli
services:
- docker:24-dind
- docker:dind
before_script:
- docker info
- echo "$DOCKER_ACCESS_TOKEN" | docker login --username "$DOCKER_ACCOUNT" --password-stdin
- |
apk add curl jq
ARCH=${CI_RUNNER_EXECUTABLE_ARCH#*/}
BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))")
mkdir -vp ~/.docker/cli-plugins/
curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL
chmod a+x ~/.docker/cli-plugins/docker-buildx
- docker buildx create --use --driver cloud ${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}

variables:
Expand Down Expand Up @@ -195,13 +202,6 @@ jobs:
steps:
- checkout

- run: |
mkdir -vp ~/.docker/cli-plugins/
ARCH=amd64
BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))")
curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL
chmod a+x ~/.docker/cli-plugins/docker-buildx

- run: echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin
- run: docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}"

Expand All @@ -218,13 +218,6 @@ jobs:
steps:
- checkout

- run: |
mkdir -vp ~/.docker/cli-plugins/
ARCH=amd64
BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))")
curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL
chmod a+x ~/.docker/cli-plugins/docker-buildx

- run: echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin
- run: docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}"

Expand Down Expand Up @@ -281,28 +274,6 @@ steps:
Create the `build.sh` script:

```bash
DOCKER_DIR=/usr/libexec/docker

# Get download link for latest buildx binary.
# Set $ARCH to the CPU architecture (e.g. amd64, arm64)
UNAME_ARCH=`uname -m`
case $UNAME_ARCH in
aarch64)
ARCH="arm64";
;;
amd64)
ARCH="amd64";
;;
*)
ARCH="amd64";
;;
esac
BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))")

# Download docker buildx with Build Cloud support
curl --silent -L --output $DOCKER_DIR/cli-plugins/docker-buildx $BUILDX_URL
chmod a+x ~/.docker/cli-plugins/docker-buildx

# Connect to your builder and set it as the default builder
docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}"

Expand All @@ -328,7 +299,6 @@ pipeline {
agent any

environment {
ARCH = 'amd64'
DOCKER_ACCESS_TOKEN = credentials('docker-access-token')
DOCKER_ACCOUNT = credentials('docker-account')
CLOUD_BUILDER_NAME = '<CLOUD_BUILDER_NAME>'
Expand All @@ -337,13 +307,7 @@ pipeline {

stages {
stage('Build') {
environment {
BUILDX_URL = sh (returnStdout: true, script: 'curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\\"linux-$ARCH\\"))"').trim()
}
steps {
sh 'mkdir -vp ~/.docker/cli-plugins/'
sh 'curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL'
sh 'chmod a+x ~/.docker/cli-plugins/docker-buildx'
sh 'echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin'
sh 'docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}"'
// Cache-only build
Expand Down Expand Up @@ -372,12 +336,7 @@ env:
before_install: |
echo "$DOCKER_ACCESS_TOKEN" | docker login --username "$DOCKER_ACCOUNT" --password-stdin

install: |
set -e
BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$TRAVIS_CPU_ARCH\"))")
mkdir -vp ~/.docker/cli-plugins/
curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL
chmod a+x ~/.docker/cli-plugins/docker-buildx
before_script: |
docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}"

script: |
Expand All @@ -393,18 +352,13 @@ script: |
# Prerequisites: $DOCKER_ACCOUNT, $CLOUD_BUILDER_NAME, $DOCKER_ACCESS_TOKEN setup as deployment variables
# This pipeline assumes $BITBUCKET_REPO_SLUG as the image name

image: atlassian/default-image:3
image: docker:cli

pipelines:
default:
- step:
name: Build multi-platform image
script:
- mkdir -vp ~/.docker/cli-plugins/
- ARCH=amd64
- BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))")
- curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL
- chmod a+x ~/.docker/cli-plugins/docker-buildx
- echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin
- docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}"
- IMAGE_NAME=$BITBUCKET_REPO_SLUG
Expand All @@ -421,15 +375,6 @@ pipelines:
```bash
#!/bin/bash

# Get download link for latest buildx binary. Set $ARCH to the CPU architecture (e.g. amd64, arm64)
ARCH=amd64
BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))")

# Download docker buildx with Build Cloud support
mkdir -vp ~/.docker/cli-plugins/
curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL
chmod a+x ~/.docker/cli-plugins/docker-buildx

# Login to Docker Hub with an access token. See https://docs.docker.com/build-cloud/ci/#creating-access-tokens
echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin

Expand All @@ -452,35 +397,19 @@ docker buildx build \

### Docker Compose

Use this implementation if you want to use `docker compose build` with
Docker Build Cloud in CI.
This example requires Buildx version 0.37.0 or later and a Docker Compose
version that supports Docker Build Cloud. Set the cloud builder as the default
builder before running `docker compose build` in CI:

```bash
#!/bin/bash

# Get download link for latest buildx binary. Set $ARCH to the CPU architecture (e.g. amd64, arm64)
ARCH=amd64
BUILDX_URL=$(curl -s https://raw.githubusercontent.com/docker/actions-toolkit/main/.github/buildx-lab-releases.json | jq -r ".latest.assets[] | select(endswith(\"linux-$ARCH\"))")
COMPOSE_URL=$(curl -sL \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer <GITHUB_TOKEN>" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/docker/compose-desktop/releases \
| jq "[ .[] | select(.prerelease==false and .draft==false) ] | .[0].assets.[] | select(.name | endswith(\"linux-${ARCH}\")) | .browser_download_url")

# Download docker buildx with Build Cloud support
mkdir -vp ~/.docker/cli-plugins/
curl --silent -L --output ~/.docker/cli-plugins/docker-buildx $BUILDX_URL
curl --silent -L --output ~/.docker/cli-plugins/docker-compose $COMPOSE_URL
chmod a+x ~/.docker/cli-plugins/docker-buildx
chmod a+x ~/.docker/cli-plugins/docker-compose

# Login to Docker Hub with an access token. See https://docs.docker.com/build-cloud/ci/#creating-access-tokens
echo "$DOCKER_ACCESS_TOKEN" | docker login --username $DOCKER_ACCOUNT --password-stdin

# Connect to your builder and set it as the default builder
docker buildx create --use --driver cloud "${DOCKER_ACCOUNT}/${CLOUD_BUILDER_NAME}"

# Build the image build
# Build the images
docker compose build
```
25 changes: 16 additions & 9 deletions content/manuals/build-cloud/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,21 +15,28 @@ environment.

To get started with Docker Build Cloud, you need to:

- Download and install Docker Desktop version 4.26.0 or later.
- Install Docker Desktop version 4.26.0 or later, or Buildx version 0.37.0 or
later.
- Create a cloud builder on the [Docker Build Cloud Dashboard](https://app.docker.com/build/).
- When you create the builder, choose a name for it (for example, `default`). You will use this name as `BUILDER_NAME` in the CLI steps below.

### Use Docker Build Cloud without Docker Desktop

To use Docker Build Cloud without Docker Desktop, you must download and install
a version of Buildx with support for Docker Build Cloud (the `cloud` driver).
You can find compatible Buildx binaries on the releases page of
[this repository](https://github.com/docker/buildx-desktop).
Buildx version 0.37.0 or later includes the
[`cloud` driver](/manuals/build/builders/drivers/cloud.md). To use Docker Build
Cloud without Docker Desktop, check your installed version:

If you plan on building with Docker Build Cloud using the `docker compose
build` command, you also need a version of Docker Compose that supports Docker
Build Cloud. You can find compatible Docker Compose binaries on the releases
page of [this repository](https://github.com/docker/compose-desktop).
```console
$ docker buildx version
```

If your Docker CLI installation doesn't include a compatible Buildx version,
[install Buildx](https://github.com/docker/buildx#manual-download) as a Docker
CLI plugin.

<!-- TODO: Specify the minimum Compose version after the coordinated Compose release. -->
To use Docker Build Cloud with Docker Compose, you need Buildx version 0.37.0 or
later and a Docker Compose version that supports Docker Build Cloud.

## Steps

Expand Down
23 changes: 13 additions & 10 deletions content/manuals/build-cloud/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ builder for the `docker buildx build` command. The `docker build` command still
uses the `default` builder, unless you specify the `--builder` flag explicitly.

If you use build scripts, such as `make`, that use the `docker build` command,
we recommend updating your build commands to `docker buildx build`. Alternatively,
you can set the [`BUILDX_BUILDER` environment
update your build commands to `docker buildx build`. Alternatively, you can set
the [`BUILDX_BUILDER` environment
variable](/manuals/build/building/variables.md#buildx_builder) to specify which
builder `docker build` should use.

Expand All @@ -72,13 +72,16 @@ variable](/manuals/build/building/variables.md#buildx_builder) to select the clo

## Loading build results

Building with `--tag` loads the build result to the local image store
automatically when the build finishes. To build without a tag and load the
result, you must pass the `--load` flag.
When you don't specify an output, Buildx leaves an untagged build result in the
cloud build cache and doesn't load it into Docker Engine. If you use `--tag`,
Buildx automatically loads the image when the build targets a single platform
and runs on one cloud node. Use `--load` to request loading explicitly.

Loading the build result for multi-platform images is not supported. Use the
`docker buildx build --push` flag when building multi-platform images to push
the output to a registry.
The target Docker context determines which Docker Engine image store receives
the result. For details about loading behavior and other output configurations,
see [Load results from Docker Build Cloud](/manuals/build/exporters/_index.md#load-results-from-docker-build-cloud).

To push a multi-platform image to a registry instead of loading it:

```console
$ docker buildx build --builder cloud-<ORG>-<BUILDER_NAME> \
Expand All @@ -87,8 +90,8 @@ $ docker buildx build --builder cloud-<ORG>-<BUILDER_NAME> \
--push .
```

If you want to build with a tag, but you don't want to load the results to your
local image store, you can export the build results to the build cache only:
To keep a tagged result in the build cache instead of loading it into Docker
Engine, use the `cacheonly` exporter:

```console
$ docker buildx build --builder cloud-<ORG>-<BUILDER_NAME> \
Expand Down
4 changes: 4 additions & 0 deletions content/manuals/build/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ grid:
description: Configure where and how you run your builds.
icon: wrench-screwdriver
link: /build/builders/drivers/
- title: Docker Build Cloud
description: Run builds on managed cloud builders with a shared cache.
icon: /icons/logo-build-cloud.svg
link: /build-cloud/
- title: Exporters
description: Export any artifact you like, not just Docker images.
icon: arrow-up-on-square
Expand Down
37 changes: 21 additions & 16 deletions content/manuals/build/builders/drivers/_index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Build drivers
description: Build drivers are configurations for how and where the BuildKit backend runs.
keywords: build, buildx, driver, builder, docker-container, kubernetes, remote
keywords: build, buildx, driver, builder, cloud, docker-container, kubernetes, remote
aliases:
- /build/buildx/drivers/
- /build/building/drivers/
Expand All @@ -15,6 +15,7 @@ Buildx supports the following drivers:

- `docker`: uses the BuildKit library bundled into the Docker daemon.
- `docker-container`: creates a dedicated BuildKit container using Docker.
- `cloud`: connects to a managed builder in Docker Build Cloud.
- `kubernetes`: creates BuildKit pods in a Kubernetes cluster.
- `remote`: connects directly to a manually managed BuildKit daemon.

Expand All @@ -25,25 +26,28 @@ provide more flexibility and are better at handling advanced scenarios.

The following table outlines some differences between drivers.

| Feature | `docker` | `docker-container` | `kubernetes` | `remote` |
| :--------------------------- | :---------: | :----------------: | :----------: | :----------------: |
| **Automatically load image** | ✅ | | | |
| **Cache export** | ✅\* | ✅ | ✅ | ✅ |
| **Tarball output** | | ✅ | ✅ | ✅ |
| **Multi-arch images** | | ✅ | ✅ | ✅ |
| **BuildKit configuration** | | ✅ | ✅ | Managed externally |
| Feature | `docker` | `docker-container` | `cloud` | `kubernetes` | `remote` |
| :--------------------------- | :---------: | :----------------: | :--------------: | :----------: | :----------------: |
| **Automatically load image** | ✅ | | Conditional | | |
| **Cache export** | ✅\* | ✅ | ✅ | ✅ | ✅ |
| **Tarball output** | | ✅ | ✅ | ✅ | ✅ |
| **Multi-arch images** | | ✅ | ✅ | ✅ | ✅ |
| **BuildKit configuration** | | ✅ | Managed by Docker | ✅ | Managed externally |

\* _The `docker` driver doesn't support all cache export options.
See [Cache storage backends](/manuals/build/cache/backends/_index.md) for more information._

## Loading to local image store

Unlike when using the default `docker` driver, images built using other drivers
aren't automatically loaded into the local image store. If you don't specify an
output, the build result is exported to the build cache only.
The `docker` driver automatically loads images into the local image store.
With Docker Build Cloud, an untagged result remains in the cloud build cache
when you don't specify an output. Using `--tag` instead automatically loads the
image when the build targets a single platform and runs on one cloud node. With
other drivers, the build result remains in the build cache if you don't specify
an output.

To build an image using a non-default driver and load it to the image store,
use the `--load` flag with the build command:
To build an image using a driver that doesn't load results automatically, use
the `--load` flag with the build command:

```console
$ docker buildx build --load -t <image> --builder=container .
Expand Down Expand Up @@ -85,7 +89,8 @@ flag.

Read about each driver:

- [Docker driver](./docker.md)
- [Docker container driver](./docker-container.md)
- [Kubernetes driver](./kubernetes.md)
- [Docker driver](./docker.md)
- [Docker container driver](./docker-container.md)
- [Cloud driver](./cloud.md)
- [Kubernetes driver](./kubernetes.md)
- [Remote driver](./remote.md)
Loading