Skip to content

hack: use bake to build buildkit images #6107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 67 additions & 37 deletions .github/workflows/buildkit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,6 @@ jobs:
prepare:
runs-on: ubuntu-24.04
outputs:
tag: ${{ steps.prep.outputs.tag }}
push: ${{ steps.prep.outputs.push }}
platforms: ${{ steps.prep.outputs.platforms }}
steps:
-
Expand All @@ -48,25 +46,6 @@ jobs:
name: Prepare
id: prep
run: |
TAG=pr
PUSH=false
if [ "${{ github.event_name }}" = "schedule" ]; then
TAG=nightly
PUSH=push
elif [[ $GITHUB_REF == refs/tags/v* ]]; then
TAG=${GITHUB_REF#refs/tags/}
PUSH=push
elif [[ $GITHUB_REF == refs/heads/* ]]; then
TAG=$(echo ${GITHUB_REF#refs/heads/} | sed -r 's#/+#-#g')
if [ $GITHUB_REF = "refs/heads/${{ github.event.repository.default_branch }}" ]; then
PUSH=push
fi
fi
if [ "$GITHUB_REPOSITORY" != "moby/buildkit" ]; then
PUSH=false
fi
echo "tag=${TAG}" >>${GITHUB_OUTPUT}
echo "push=${PUSH}" >>${GITHUB_OUTPUT}
platforms=$(docker buildx bake release --print | jq -cr '.target."release".platforms')
echo "platforms=$platforms" >>${GITHUB_OUTPUT}

Expand Down Expand Up @@ -183,21 +162,29 @@ jobs:
image:
runs-on: ubuntu-24.04
needs:
- prepare
- test
strategy:
fail-fast: false
matrix:
target-stage:
target:
- ''
- rootless
steps:
-
name: Checkout
uses: actions/checkout@v5
-
name: Expose GitHub Runtime
uses: crazy-max/ghaction-github-runtime@v3
name: Prepare
run: |
if [ -n "${{ matrix.target }}" ]; then
echo "TAG_SUFFIX=-${{ matrix.target }}" >> $GITHUB_ENV
fi
if [[ $GITHUB_REF == refs/tags/v* ]]; then
if [[ "${GITHUB_REF#refs/tags/}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
if [ -n "${{ matrix.target }}" ]; then
echo "TAG_LATEST=${{ matrix.target }}" >> $GITHUB_ENV
else
echo "TAG_LATEST=latest" >> $GITHUB_ENV
fi
fi
fi
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
Expand All @@ -208,23 +195,66 @@ jobs:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.IMAGE_NAME }}
# versioning strategy
## push semver tag v0.24.0
### moby/buildkit:v0.24.0
### moby/buildkit:latest
### moby/buildkit:v0.24.0-rootless
### moby/buildkit:rootless
## push semver prerelease tag v0.24.0-rc1
### moby/buildkit:v0.24.0-rc1
### moby/buildkit:v0.24.0-rc1-rootless
## push on master
### moby/buildkit:master
### moby/buildkit:master-rootless
## scheduled event on master
### moby/buildkit:nightly
### moby/buildkit:nightly-rootless
tags: |
type=schedule,pattern=nightly,suffix=${{ env.TAG_SUFFIX }}
type=ref,event=branch,suffix=${{ env.TAG_SUFFIX }}
type=ref,event=pr,suffix=${{ env.TAG_SUFFIX }}
type=semver,pattern={{raw}},suffix=${{ env.TAG_SUFFIX }}
type=raw,value=${{ env.TAG_LATEST }}
flavor: |
latest=false
annotations: |
org.opencontainers.image.title=BuildKit
org.opencontainers.image.vendor=Moby
bake-target: meta-helper
-
name: Login to DockerHub
if: needs.prepare.outputs.push == 'push'
if: ${{ github.repository == 'moby/buildkit' && (github.event_name == 'schedule' || github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/v')) }}
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build ${{ needs.prepare.outputs.tag }}
run: |
./hack/images "${{ needs.prepare.outputs.tag }}" "$IMAGE_NAME" "${{ needs.prepare.outputs.push }}"
name: Build
uses: docker/bake-action@v6
with:
source: ${{ github.server_url }}/${{ github.repository }}.git#${{ github.ref }}
Copy link
Member Author

@crazy-max crazy-max Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't use Git context from our actions-toolkit: https://github.com/docker/actions-toolkit/blob/633bcf1936ffff0e05a4bbe2b23c71abf6f4111e/src/context.ts#L45-L60

Because currently it fetches by sha and not git ref so tags are not pulled: https://github.com/crazy-max/buildkit/actions/runs/16967780276/job/48096258733#step:7:440. That would therefore set a commit sha as BuildKit version: https://github.com/crazy-max/buildkit/actions/runs/16967780276/job/48096258733#step:7:1654

docker/actions-toolkit#677 would fix it but it's probably better to wait for #5974.

So in the meantime set Git context manually with full git ref as we currently do in our hack script:

currentcontext="https://github.com/$GITHUB_REPOSITORY.git#$GITHUB_REF"

files: |
./docker-bake.hcl
cwd://${{ steps.meta.outputs.bake-file-tags }}
cwd://${{ steps.meta.outputs.bake-file-annotations }}
targets: image-cross
push: ${{ github.repository == 'moby/buildkit' && (github.event_name == 'schedule' || github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/v')) }}
provenance: mode=max
sbom: true
set: |
*.cache-from=type=gha,scope=image${{ matrix.target }}
*.cache-to=type=gha,scope=image${{ matrix.target }}
*.no-cache-filter=${{ (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/v')) && 'buildkit-export,gobuild-base,rootless' || '' }}
env:
RELEASE: ${{ github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/v') }}
TARGET: ${{ matrix.target-stage }}
CACHE_FROM: type=gha,scope=image${{ matrix.target-stage }}
CACHE_TO: type=gha,scope=image${{ matrix.target-stage }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
IMAGE_TARGET: ${{ matrix.target }}

scout:
runs-on: ubuntu-24.04
Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ cross:
.PHONY: images
images:
# moby/buildkit:local and moby/buildkit:local-rootless are created on Docker
hack/images local moby/buildkit
TARGET=rootless hack/images local moby/buildkit
$(BUILDX_CMD) bake image
IMAGE_TARGET=rootless $(BUILDX_CMD) bake image

.PHONY: install
install:
Expand Down
29 changes: 29 additions & 0 deletions docker-bake.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ variable "ARCHUTIL_MULTIPLATFORM" {
default = null
}

variable "IMAGE_TARGET" {
default = null
}

# Defines the output folder
variable "DESTDIR" {
default = ""
Expand Down Expand Up @@ -78,6 +82,11 @@ function "bindir" {
result = DESTDIR != "" ? DESTDIR : "./bin/${defaultdir}"
}

# Special target: https://github.com/docker/metadata-action#bake-definition
target "meta-helper" {
tags = [IMAGE_TARGET != null && IMAGE_TARGET != "" ? "moby/buildkit:local-${IMAGE_TARGET}" : "moby/buildkit:local"]
}

target "_common" {
args = {
ALPINE_VERSION = ALPINE_VERSION
Expand Down Expand Up @@ -133,6 +142,26 @@ target "release" {
output = [bindir("release")]
}

target "image" {
inherits = ["_common", "meta-helper"]
target = IMAGE_TARGET
cache-to = ["type=inline"]
output = ["type=docker"]
}

target "image-cross" {
inherits = ["image"]
output = ["type=image"]
platforms = [
"linux/amd64",
"linux/arm/v7",
"linux/arm64",
"linux/s390x",
"linux/ppc64le",
"linux/riscv64"
]
}

target "integration-tests-base" {
inherits = ["_common"]
target = "integration-tests-base"
Expand Down
115 changes: 0 additions & 115 deletions hack/images

This file was deleted.