Extracting base image from llbdefinition

[jgarec]

Hello,

Trying to find the base image name and its digest, I try to evaluate if I can get this from the metadata file with provenance=max.

"buildConfig": {
      "llbDefinition": [
        {
          "id": "step0",
          "op": {
            "Op": {
              "source": {
                "identifier": "docker-image://docker.io/library/busybox:latest@sha256:9ae97d36d26566ff84e8893c64a6dc4fe8ca6d1144bf5b87b2b85a32def253c7",
                "attrs": {
                  "image.resolvemode": "local"
                }
              }
            },
            "platform": {
              "Architecture": "arm64",
              "OS": "linux"
            },
            "constraints": {}
          }
        }

With docker, I do this :

BUILDX_METADATA_PROVENANCE=max docker buildx build --metadata-file=metadata.json .
jq -r '.["buildx.build.provenance"].buildConfig.llbDefinition[0].op.Op.source.identifier' metadata.json


docker-image://docker.io/library/busybox:latest@sha256:9ae97d36d26566ff84e8893c64a6dc4fe8ca6d1144bf5b87b2b85a32def253c7

After some tests, it seems always true (no matter if this is a multi stage build or not) except if the base image is scratch.

Do you think this is reliable ?

Regards,

[tonistiigi]

The correct way to get the base image is to look at the materials section of the provenance attestation (does not need to be mode=max) and find a dependency that is the subarray of your image layers. You can combine it with llbDefinition if you need to extract more info about the steps layout/config. In mode=max you also have the layer arrays of all build steps (if they have layers) in the provenance that you can use for subarray comparison. Do not make assumptions about the order of steps in the llbDefinition, except that the last step is the pointer to the build result.

[tonistiigi]

The correlation between layers and steps is stable and you can use that. It takes some semantics though to understand what is the step that generated your base image in more complicated builds. That's why it is probably simpler to just look at the layers for the final step and find another step with a subarray of that. Then you can find a matching step definition and even dockerfile line if you look at the source locations for that step.

[Kevin-Molina]

I think I understand + Leveraging the metadata file to avoid the extra registry requests like you're suggesting is absolutely way more efficient / stable (saves a significant amount of traffic to the registry) / etc 👍

I intent to code and depend on this, so please let me know if my heuristic is flawed / absolutely flat out school me on the right methodology if I'm off on anything!

Observations:

1. Images used (tags omitted):

• Ubuntu
• Alpine
• Node (This is my "parent" image, and it is important to call out - This image [Node]'s parent is Alpine)

2. Images associated by step in llbDefinition:

• Node (step0)
• Ubuntu (step1)
• Alpine (step4)

Note - Won't be depending on ordering here as you mentioned.

3. We can assume the last item in 'buildx.build.provenance'.metadata.'https://mobyproject.org/buildkit@v1#metadata'.layers is the build result.
   That is actually a map, so we should get the last item in llbDefinition, grab its key, and that gives us the stepXXX that's the build result step name/number. Then we can grab the layers for it by going into 'buildx.build.provenance'.metadata.'https://mobyproject.org/buildkit@v1#metadata'.layers and grabbing the entry with key <stepXXX>:0.

4. We can correlate steps in llbDefinition to layers in 'buildx.build.provenance'.metadata.'https://mobyproject.org/buildkit@v1#metadata'.layers

Methodology

1. Grab our build result's layers via the last step in 'buildx.build.provenance'.metadata.'https://mobyproject.org/buildkit@v1#metadata'.layers.
2. Go through llbDefinition and get all the "steps" that denote an image used in the build.
   2.1. Find all steps with "identifier": "docker-image:
   2.2. Extract the step# (In this case, step0, step1, step4) + image. Example
3. For each step that maps to an image, append :0 and grab its layers in 'buildx.build.provenance'.metadata.'https://mobyproject.org/buildkit@v1#metadata'.layers.
4. Apply the logic described above (common contiguous layers from layer 0 / longest subarray to match layers)

Example:

1. We grab these layers: https://github.com/Kevin-Molina/provenance-metadata/blob/master/metadata.json#L774-L812
2. We grab steps:

• step0 = docker.io/library/node:24.0.0-alpine3.20@sha256:dc077fd8a65f08c6e3f29b07088622bc1ba707a91d9183f4507b9b4a8a874d91
• step1 = docker.io/library/ubuntu:24.04@sha256:6015f66923d7afbc53558d7ccffd325d43b4e249f41a6e93eef074c9505d2233
• step4 = docker.io/library/alpine:3.20@sha256:de4fe7064d8f98419ea6b49190df1abbf43450c1702eeb864fe9ced453c1cc5f

3. We append :0 and grab their layers:

• Step0
• Step1
• Step4

4. Do the above common subarray check.

• We see that both Node/Alpine (The Node image's parent is the Alpine image) share layer[0] with buildResult[0], so we narrow it down to those (2) candidates. Then the image with the longest common subarray is Alpine, so Alpine is our parent image (This is a decent example because it handles that edge case of multiple images that contributed to the build all sharing layer 0).

Questions

1. The steps in layers always have a :0 suffix (e.g. step0 in llbDefinition, and step0:0 in ...layers) - Is this stable and will always be :0 / can you speak to this (It feels brittle / like I'm being hacky to append a :0 as a consumer of the file to correlate)?
2. Are there any assumptions I'm making that aren't correct / the interfaces (object keys, JSON paths, the docker-image:// usage in llbDefinition, etc) aren't stable / things that aren't safe to depend on?
3. Does my understanding and this approach make sense - Basically, am I good to build functionality (Deriving the parent image from a Dockerfile via the provenance metadata / heuristic above) on top of this?

If there's any interest in my contributing this back / make any changes that would strengthen these relationships / etc, I'd be more than happy to do that. It would be great / feel much more supported/stable to leverage this / accomplish this goal with native moby/buildkit tooling instead of parsing the metadata file on my end so 100% would love to do that if there's any interest (Priority is really just stability of functionality/touch points, so doing it as a consumer is totally OK too, but wanting to triple check the metadata file structure/etc's stability).

Lastly, I cannot thank you enough for the advice / knowledge 🫡

[Kevin-Molina]

Sat down to code this out and noticed something

Do not make assumptions about the order of steps in the llbDefinition, except that the last step is the pointer to the build result.

The llbDefinition steps list has a step9 entry, however the layers metadata section does not. There, the build result's layers exist in step8:0.

Since the layers metadata section is a Map type, I also can't grab the last item there.
I suppose I could hack something together where I split the steps out into <step>|<number>|:0, do a numeric sort on , and splice it back together to get the build result step's layers, but that feels like the wrong approach - Is there not a mechanism to grab the build result layers without being hacky?

[tonistiigi]

The last step in LLB graph isn't an actual build step but a pointer to the inputs that need to be built and becomes build result. The difference is the index. A build result needs to be a specific output index of a completed step like RUN. You can also see that this last step doesn't have op type. Running a container can create multiple output mounts, but only one can be designated as a build result.

[tonistiigi]

The steps in layers always have a :0 suffix (e.g. step0 in llbDefinition, and step0:0 in ...layers) - Is this stable and will always be :0

No, you should not assume that. Build steps allow arbitrary amount of inputs and outputs and this is the index. step8:0 means first output (index 0) of step8.