How to Use Docker Buildx Bake to Create Complex Image Build Pipelines
docker buildx command group uses BuildKit to expose advanced image build capabilities. Baked builds are a high-level feature that can be used to define automated build pipelines. They lets you produce multiple images from a single build operation.
Baked workflows are helpful when you want to publish different variants of your images or build several linked projects in parallel. In this article we’ll cover the key features of
docker buildx bake and how you can use them to streamline complex builds.
docker buildx bake command executes multiple build “targets” that each produce a container image. Targets run in parallel where possible to maximize performance. Targets may also directly reference predecessors to create sequential pipelines.
Build targets can be defined using several different mechanisms including existing Docker Compose files. Buildx will automatically build all the images identified in the file.
More advanced features are exposed when you list build targets in JSON or HCL files. These support variables, functions, and value interpolation to customize your builds.
buildx bake command looks for the following files in order:
You can specify a different file with the
-f command flag.
Build targets encapsulate all the configuration related to your build. They include details such as
- the path to the Dockerfile to build
- build context paths, defining the content available within your Dockerfile
- tags and labels to attach to the output images
- the platforms to produce images for.
A complete list of supported config fields is available in the documentation. Previously you may have supplied these settings as command-line flags to
docker buildx build (or even plain
docker build), forcing you to remember the correct values each time. With
buildx bake you can reliably use the same values by defining them in your version-controlled baked file.
Here’s a simple example of a
docker-bake.hcl command that defines a single build target:
target "default" dockerfile = "app/Dockerfile" contexts = app = "app/src" shared = "shared-components/src" tags = ["my-app:latest", "docker.io/my-org/my-app:latest"]
docker buildx bake with this bake file will load the
app/Dockerfile Dockerfile from your working directory. It’ll have access to the
shared-components/src directories as build contexts. The image that’s produced will be assigned two tags.
default target is built automatically when you run
docker buildx bake. You can also define named targets that can be built on-demand:
target "app" // ...
$ docker buildx bake app
Using Multiple Targets
You can build another image simultaneously by defining it as a new target inside your bake file:
group "default" targets = ["app", "api"] target "app" dockerfile = "app/Dockerfile" contexts = app = "app/src" shared = "shared-components/src" tags = ["my-app:latest", "docker.io/my-org/my-app:latest"] target "api" dockerfile = "api/Dockerfile" contexts = src = "https://www.howtogeek.com/devops/how-to-use-docker-buildx-bake-to-create-complex-image-build-pipelines/api/src" tags = ["my-api:latest", "docker.io/my-org/my-api:latest"]
These images can be built simultaneously because they’re nested into a group. The
app images will be built in parallel each time you run the
docker buildx bake command as the
default group is automatically selected. You can use named groups similarly to the named targets example above.
Build Target Inheritance
Build targets can inherit from each other to reuse configuration. One scenario where this can be useful concerns images that need to be customized for different environments. You might want to add extra config files to image variants intended for development use. Here’s a
docker-bake.hcl that demonstrates this model:
group "default" targets = ["backend", "backend-dev"] target "backend" dockerfile = "backend/Dockerfile" contexts = src = "https://www.howtogeek.com/devops/how-to-use-docker-buildx-bake-to-create-complex-image-build-pipelines/api/src" config = "api/config" tags = ["backend:latest"] target "backend-dev" inherits = ["backend"] contexts = config = "api/config-dev" tags = ["backend:dev"]
backend-dev target inherits all the properties of the
backend target but overrides the
config context and applies a different tag.
You can preview the merged file structure by running the
bake command with the
$ docker buildx bake --print ... "backend-dev": "context": ".", "contexts": "config": "api/config-dev", "src": "https://www.howtogeek.com/devops/how-to-use-docker-buildx-bake-to-create-complex-image-build-pipelines/api/src" , "dockerfile": "backend/Dockerfile", "tags": [ "backend:dev" ] ...
Using a Previous Target as a Base Image
Sometimes you might want a build target to use the image created by a previous target as its own base. This is an alternative to multi-stage builds that can be used when your Dockerfiles depend on each other but can’t be merged together, perhaps because they exist in different projects.
group "default" targets = ["org-base-image", "api"] target "org-base-image" dockerfile = "docker-base/Dockerfile" tags = ["org-base-image:latest"] target "api" dockerfile = "api/Dockerfile" contexts = base = "target:org-base-image" tags = ["api:latest"]
The example first builds the
org-base-image target. This could contain some utilities that are common to your organization’s containerized workloads. The
api target is then built with the output from the
org-base-image target accessible as the
base build-context. The API Dockerfile can now reference content inside the base image:
COPY --from=base /utilities/example /usr/bin/example-utility
This is a powerful pattern that lets you create dependency links between images while maintaining separate Dockerfiles.
Overriding Properties of Targets at Build Time
docker buildx bake command lets you override properties of your targets when you run your build:
$ docker buildx bake --set api.dockerfile="api/Dockerfile-dev"
This example changes the Dockerfile of the
api target. The
* wildcard is supported when identifying the target to change.
* on its own selects every target while
api* will modify all the targets that begin with
HCL files can define variables that you can reference in your build targets. use a
variable block to set them up:
variable "TAG" default = "latest" group "default" targets = ["app"] target "app" dockerfile = "src/Dockerfile" tags = ["my-app:$TAG"]
docker buildx bake with this configuration will tag the
app target as
my-app:latest. You can change the value of the
TAG variable by setting an environment variable before you execute the command:
$ TAG=v1 docker buildx bake
You can use all the variable interpolation and comparison capabilities of the HCL language to make your build targets reusable. Functions are available too for parsing and transforming your values.
Baked Buildx builds let you encapsulate image build configuration as “targets” defined in a file. When you run
buildx bake, images for all the referenced targets are built in parallel.
Targets can inherit from and depend on each other. You can also use variables and functions to create highly complex and configurable build pipelines.
docker buildx bake command is a high-level operation that’s not necessary in every workflow. You don’t need to use it when you’re creating simple images with no cross-project dependencies. Using
docker compose build is a better alternative for most use cases that keeps build configuration in your
docker-compose.yml file. Switching to baked builds should be considered when you’re building many images simultaneously using different variables, platforms, build contexts, and config overrides.