dockerize the build of vector package

[europaul]

Description

This PR addresses some suggestions from the original PR that introduced vector #5008 (comment).

Remove the dependency on the minimal version of vector built externally. Now we have the full vector package built in the EVE build system, using lf-edge's eve-rust image as a base toolchain.

PR dependencies

None.

How to test and validate this PR

Only includes changes to the EVE build. No testing necessary.

Changelog notes

N/A

PR Backports

Nope.

Checklist

• I've provided a proper description
• I've added the proper documentation
• I've tested my PR on amd64 device
• I've tested my PR on arm64 device
• I've written the test verification instructions
• I've set the proper labels to this PR

[europaul]

I think the comments from yetus can be disregarded.

Check Docker Hashes Consistency seems to be making a mistake: we base this Dockerfile on eve-rust:1.85.1-2, not on eve-rust:1.85.1-1

[christoph-zededa]

Check Docker Hashes Consistency seems to be making a mistake: we base this Dockerfile on eve-rust:1.85.1-2, not on eve-rust:1.85.1-1

It is complaining that monitor is using a different image version than vector. Why would we need two different versions?

[europaul]

Check Docker Hashes Consistency seems to be making a mistake: we base this Dockerfile on eve-rust:1.85.1-2, not on eve-rust:1.85.1-1

It is complaining that monitor is using a different image version than vector. Why would we need two different versions?

Those are two different tags. Both use rust 1.85.1, but the first one uses gcc for cross-compilation and the second one uses clang. If clang does a good job in the future, I think we'll use only clang version starting from the next rust version update.

[christoph-zededa]

Check Docker Hashes Consistency seems to be making a mistake: we base this Dockerfile on eve-rust:1.85.1-2, not on eve-rust:1.85.1-1

It is complaining that monitor is using a different image version than vector. Why would we need two different versions?

Those are two different tags. Both use rust 1.85.1, but the first one uses gcc for cross-compilation and the second one uses clang. If clang does a good job in the future, I think we'll use only clang version starting from the next rust version update.

sounds like the image should be called eve-rust-gcc:... and eve-rust-clang:...

btw. the checker does not work on the diff, so if it gets merged, then every PR will be red

[OhmSpectator]

Check Docker Hashes Consistency seems to be making a mistake: we base this Dockerfile on eve-rust:1.85.1-2, not on eve-rust:1.85.1-1

It is complaining that monitor is using a different image version than vector. Why would we need two different versions?

Those are two different tags. Both use rust 1.85.1, but the first one uses gcc for cross-compilation and the second one uses clang. If clang does a good job in the future, I think we'll use only clang version starting from the next rust version update.

sounds like the image should be called eve-rust-gcc:... and eve-rust-clang:...

I want to here @rucoder's opinion here then =)

[OhmSpectator]

I see all the ARM builds were cancelled... Did anyone do it on purpose?

[rucoder]

@europaul @OhmSpectator we talked about this before with @rene and eve-rust image must be excluded from that check! this is perfectly fine and expected to have different rust versions for different packages!

[christoph-zededa]

@europaul @OhmSpectator we talked about this before with @rene and eve-rust image must be excluded from that check! this is perfectly fine and expected to have different rust versions for different packages!

Doesn't this duplicate work, f.e. checking CVEs?

[OhmSpectator]

@europaul @OhmSpectator we talked about this before with @rene and eve-rust image must be excluded from that check! this is perfectly fine and expected to have different rust versions for different packages!

Could you explain why different components require different versions of our own package? Do we have examples of other similar packages? @christoph-zededa, is there a mechanism to configure this in the checker?

[christoph-zededa]

@europaul @OhmSpectator we talked about this before with @rene and eve-rust image must be excluded from that check! this is perfectly fine and expected to have different rust versions for different packages!

Could you explain why different components require different versions of our own package? Do we have examples of other similar packages? @christoph-zededa, is there a mechanism to configure this in the checker?

We do, see commit by @rene: 6b8a09b

[christoph-zededa]

This adds a different version of busybox to the image and squashfs cannot deduplicate it then. Is it really needed?

[christoph-zededa]

I would suggest, to throw out the different version of busybox, etc. but therefore keep the debug symbols.

[rene]

could we use eve-alpine here or 3.21 somehow is required?

[europaul]

thanks @christoph-zededa! I'll change it to eve-alpine

[Copilot]

Pull Request Overview

This PR dockerizes the build of the vector package by replacing the external dependency on a minimal vector build with a full vector package built within the EVE build system. The change uses the lf-edge eve-rust image as the base toolchain and implements a multi-stage Docker build to compile vector from source with specific features enabled.

Key changes:

• Replaces external vector image dependency with in-house Docker build process
• Implements multi-stage build with cross-platform support for amd64, arm64, and riscv64
• Adds comprehensive vector feature configuration and Rust optimization flags

Comments suppressed due to low confidence (1)

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[europaul]

I see all the ARM builds were cancelled... Did anyone do it on purpose?

I also found it strange...

[europaul]

btw. the checker does not work on the diff, so if it gets merged, then every PR will be red

@christoph-zededa I think this problem will go away when we release the next version of eve-rust e.g. 1.86.0 (that will use clang) and use it to build both vector and monitor

btw, then we can also upgrade to vector v0.48

[christoph-zededa]

btw. the checker does not work on the diff, so if it gets merged, then every PR will be red

@christoph-zededa I think this problem will go away when we release the next version of eve-rust e.g. 1.86.0 (that will use clang) and use it to build both vector and monitor

btw, then we can also upgrade to vector v0.48

fyi: #4664 (comment)

[rene]

same, could we use eve-alpine + eve-alpine-deploy.sh script instead of RUN apk here?

[rucoder]

@europaul git perl protoc are part of eve-rust 1.85-2

[rucoder]

@europaul @OhmSpectator we talked about this before with @rene and eve-rust image must be excluded from that check! this is perfectly fine and expected to have different rust versions for different packages!

Could you explain why different components require different versions of our own package? Do we have examples of other similar packages? @christoph-zededa, is there a mechanism to configure this in the checker?

We do, see commit by @rene: 6b8a09b

@europaul just add vector to the list mentioned by @christoph-zededa
@OhmSpectator this is normal for rust apps to pin a rust version.

[europaul]

BTW, since we are updating the eve-alpine package here, the go version will be updated alongside to 1.25.0.

[OhmSpectator]

BTW, since we are updating the eve-alpine package here, the go version will be updated alongside to 1.25.0.

Wow-wow-wow... Can we do it as a dedicated PR?.. It's a major change, bigger than the vector itself.

[europaul]

BTW, since we are updating the eve-alpine package here, the go version will be updated alongside to 1.25.0.

Wow-wow-wow... Can we do it as a dedicated PR?.. It's a major change, bigger than the vector itself.

why is it a big change? we just recently bumped it in #5027, so I guess the difference is minimal

[rene]

BTW, since we are updating the eve-alpine package here, the go version will be updated alongside to 1.25.0.

Wow-wow-wow... Can we do it as a dedicated PR?.. It's a major change, bigger than the vector itself.

@europaul , let's not bump alpine packages here, please. You can just add the package that you need on the main file and make this change to Dockerfile:

diff --git a/pkg/alpine/Dockerfile b/pkg/alpine/Dockerfile
index 5be2a6ada..14a43d5bd 100644
--- a/pkg/alpine/Dockerfile
+++ b/pkg/alpine/Dockerfile
@@ -12,8 +12,7 @@ ARG ALPINE_VERSION=3.16
 # this is only needed once, when this package
 # is rebased on the new version of Alpine and
 # you have to have FROM alpine:x.y.z above:
-# Trigger update for go package
-RUN apk update && apk upgrade -a
+#RUN apk update && apk upgrade -a
 
 # Copy Dockerfile so we can include it in the hash
 COPY Dockerfile abuild.conf /etc/

without the apk update && apk upgrade -a the version will remain the same and it will install the inotify-tools package.

[rene]

No go version bump here. See my comment on how to proceed.

[rene]

FYI I just cancelled the workflows, it would take ages to run and changes were requested to @europaul anyways.

[europaul]

@rene could you explain why we shouldn't update eve-alpine normally? I don't really understand this...

[rene]

@rene could you explain why we shouldn't update eve-alpine normally? I don't really understand this...

Our approach is to use eve-alpine as an image with all packages that we need cached inside (i.e., those listed on pkg/alpine/mirrors). Also, apk keeps the list of all packages available on all the repos and their versions.

When you need a package that is not installed in the eve-alpine image, you should add it to the corresponding file under pkg/alpine/mirrors/ (according to which repo the package is located, i.e., main, community, etc). Notice that add a new package doesn't update any other package.

When we want to bump all packages and their versions, for instance, when we want to bump Go version, then we need to run the following commands inside pkg/alpine/Dockerfile:

apk update && apk upgrade -a 

This was the case when I updated Alpine to 3.16.9 (458c9eb) and @christoph-zededa bumped Go version (eabd917). However, in case of this PR, we just want to add inotify-tools to be cached inside eve-alpine image and that's why you can safely comment the commands I cited above.

The point is that we don't need to bump all packages again, let's wait a bit for more stability and here you just need to add the inotify-tools package.