How to enable a registry mirror for an ECR pull-through cache?

[bscaleb]

I have created an ECR pull-through cache rule in my AWS account that allows pull-through from Docker Hub. These images are pulled through under the docker-hub prefix.

I build images using the buildkit container. I have a buildkitd.toml file which looks like this:

debug = true
[registry."docker.io"]
  mirrors = ["123456789.dkr.ecr.us-east-1.amazonaws.com/docker-hub"]

I added it to the buildkit image by building and publishing my own Dockerfile:

FROM 123456789.dkr.ecr.us-east-1.amazonaws.com/docker-hub/moby/buildkit:v0.19.0-rootless

COPY buildkitd.toml /home/user/.config/buildkit/buildkitd.toml

The full command I run inside this container to build an image looks like this:

buildctl-daemonless.sh build '--frontend=dockerfile.v0' --local 'context=my-image' --local 'dockerfile=my-image' --opt 'filename=Dockerfile' --export-cache 'type=registry,ref=123456789.dkr.ecr.us-east-1.amazonaws.com/my-cache:example,mode=max,image-manifest=true' --import-cache 'type=registry,ref=123456789.dkr.ecr.us-east-1.amazonaws.com/my-cache:example' --output 'type=image,name=123456789.dkr.ecr.us-east-1.amazonaws.com/example:example,push=true' 

I tried building this Dockerfile using my updated buildkit image:

FROM busybox:latest

and it does not use the mirror registry. It still just pulls from docker.io. Has anyone tried to do something like this before? It's hard to tell what I'm doing wrong because nothing additional is logged.

[bscaleb]

What I described above does in fact work, assuming your ECR pull-through cache rule for Docker Hub uses the docker-hub prefix (this is configurable when you create the rule in AWS). The reason it wasn't working was a silly mistake. Some things to note that made this a bit more confusing for me to debug:

• Setting debug = true does not affect the log output of the buildctl-daemonless.sh script. It only affects the log output of the buildkit daemon, which in my setup (Gitlab CI pipeline with a script supplied to the image) is not capture anywhere.
• The log output of the build process also does not change when it is using the registry mirror. The only way I found to determine if this is doing what I want is to try building an image that is not yet present on the pull-through cache (in my case I used busybox:latest) and see if the ECR repo and image get created at 123456789.dkr.ecr.us-east-1.amazonaws.com/docker-hub/library/busybox:latest.