Add Alpine Linux package management tools for EVE OS #5179

rucoder · 2025-08-14T17:59:49Z

Description

this PR addresses #5172

Add three new Python tools for managing Alpine Linux package lists in the EVE build system to handle Alpine version upgrades and package maintenance.

alpine_index.py: Core library providing shared utilities for APKINDEX parsing, dependency resolution, and package classification across branches and architectures.

alpine-migrate.py: Migrates complete package lists between Alpine versions (e.g. 3.16 → 3.21), handling package relocations between main/community branches, renamed packages, and changed dependencies.

alpine-pkg-add.py: Adds specific packages with full dependency resolution to existing Alpine package lists, useful for expanding the package cache with new tools or libraries.

Key features:

Automatic recursive dependency resolution with cycle detection
Support for x86_64, aarch64, riscv64 architectures
Handles virtual packages (so:, cmd:, pc: prefixes) and metapackages
Package relocation handling between main/community branches
Missing package reporting for manual review
Optional dependency chain analysis for debugging
Maintains alpine-base container package cache directory structure

These tools are essential for Alpine version upgrades where packages frequently move between branches, get renamed, or have dependency changes, ensuring complete and consistent package sets for reproducible EVE OS builds.

How to test and validate this PR

Basic functionality testing:

Test alpine-migrate.py migration:

cd eve
# Test migration from existing version to new version
python3 tools/alpine-tools/alpine-migrate.py 3.16 3.21 --path pkg/alpine/mirrors

# Verify output files are created in pkg/alpine/mirrors/3.21/
ls pkg/alpine/mirrors/3.21/
# Should see: main, main.x86_64, main.aarch64, community, community.x86_64, etc.

# Check for missing packages (expect some due to renames/removals)
ls pkg/alpine/mirrors/3.21/MISSING.*

Test alpine-pkg-add.py package addition:

# Add a simple package
python3 tools/alpine-tools/alpine-pkg-add.py 3.21 vim

# Add multiple development tools
python3 tools/alpine-tools/alpine-pkg-add.py 3.21 git curl wget build-base

# Test with dependency chains
python3 tools/alpine-tools/alpine-pkg-add.py 3.21 python3-pip --print-chains

Test help and validation:

  # Verify help text displays correctly
python3 tools/alpine-tools/alpine-migrate.py --help
python3 tools/alpine-tools/alpine-pkg-add.py --help

# Test version validation
python3 tools/alpine-tools/alpine-migrate.py 3.15 3.21  # Should fail (3.15 not supported)
python3 tools/alpine-tools/alpine-pkg-add.py invalid-version vim  # Should fail

Expected outcomes:**

Package lists should be properly organized by branch and architecture
Dependencies should be automatically resolved and included
Missing packages should be reported in MISSING.* files
New packages should be added to existing lists without removing current ones
Dependency chains should display resolution paths when requested

Architecture testing:

Tools should work identically on x86_64 and aarch64 systems
Cross-architecture package resolution should handle arch-specific packages correctly

Changelog notes

Add Alpine Linux package management tools for automated version migration and package list maintenance. These tools handle Alpine version upgrades by resolving package relocations, dependencies, and architecture-specific availability changes, ensuring complete package sets for reproducible container builds.

PR Backports

14.5-stable: No, new tooling feature not required for stable releases.
13.4-stable: No, new tooling feature not required for stable releases.

Checklist

I've provided a proper description
I've added the proper documentation
I've tested my PR on amd64 device
I've tested my PR on arm64 device
I've written the test verification instructions
I've set the proper labels to this PR

And the last but not least:

I've checked the boxes above, or I've provided a good reason why I didn't
check them.

rene · 2025-08-15T08:51:00Z

@rucoder , thanks a lot for such great scripts, these will help a lot on upgrading Alpine and to fix all missed dependencies! Good job!

deitch · 2025-08-15T08:58:51Z

Um, holy cow? Wow? This is great!

tools/alpine_migrate.py

deitch · 2025-08-15T09:01:06Z

BTW, at the request of @christoph-zededa , linuxkit recently got the ability to reference other packages automatically used build-args. So we actually could replace all of the explicit FROM lfedge/eve-alpine:12345aabbccdd in each pkg/*/Dockerfile with a build arg that automatically gets populated.

I know, not a big thing compared to this, but another piece to help ease building. I was going to add a PR for it, but I prefer to let a real user (hint hint @christoph-zededa ) do it, so we can see how well it works for someone who did not build it.

Add three new tools for managing Alpine package lists in the EVE build system: - alpine_index.py: Core library for APKINDEX parsing and dependency resolution - alpine-migrate.py: Migrate package lists between Alpine versions (e.g. 3.16 → 3.21) - alpine-pkg-add.py: Add specific packages with dependencies to existing lists Features: - Automatic dependency resolution with cycle detection - Support for x86_64, aarch64, riscv64 architectures - Handle package relocations between main/community branches - Virtual package support (so:, cmd:, pc: prefixes) - Missing package reporting and dependency chain analysis - Maintains alpine-base container package cache structure These tools handle Alpine version upgrades where packages move between branches, get renamed, or have changed dependencies, ensuring complete package sets for reproducible EVE OS builds. Signed-off-by: Mikhail Malyshev <[email protected]>

Copilot

Pull Request Overview

This PR adds three new Python tools for managing Alpine Linux package lists in the EVE build system to handle Alpine version upgrades and package maintenance. The tools provide automated migration between Alpine versions, dependency resolution, and package addition capabilities with support for multiple architectures and repository branches.

Core library (alpine_index.py) providing APKINDEX parsing, dependency resolution, and package classification utilities
Migration tool (alpine_migrate.py) for complete package list migrations between Alpine versions with handling of package relocations and dependency changes
Package addition tool (alpine_pkg_add.py) for adding specific packages with full dependency resolution to existing package lists

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
tools/alpine-tools/alpine_index.py	Core library providing APKINDEX parsing, dependency resolution, and package classification utilities
tools/alpine-tools/alpine_migrate.py	Migration tool for migrating package lists between Alpine versions with dependency resolution
tools/alpine-tools/alpine_pkg_add.py	Tool for adding specific packages to existing Alpine package lists with dependency resolution
docs/ALPINE.md	Documentation updates describing the new Alpine package management tools and their usage

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-08-15T12:21:40Z

tools/alpine-tools/alpine_index.py

+            for provide in provides:
+                # Split on '=' to ignore version part
+                # e.g., 'pc:znc=1.9.1' becomes 'pc:znc'
+                # Note: Consider keeping versions for more precise dependency resolution


This TODO comment suggests an incomplete implementation. Version constraints in provides should either be preserved for precise dependency resolution or this comment should be removed if the current approach is intentional.

Suggested change

# Note: Consider keeping versions for more precise dependency resolution

Copilot · 2025-08-15T12:21:40Z

tools/alpine-tools/alpine_migrate.py

+        print(f"🚀 Starting migration from Alpine {args.old_version} "
+              f"to {args.new_version}")
+    print(f"📁 Package directory: {args.path}")
+


This check for same-version migration is duplicated (line 262 and 274). The logic should be consolidated to avoid redundant code and improve maintainability.

Suggested change

Copilot · 2025-08-15T12:21:41Z

tools/alpine-tools/alpine_pkg_add.py

+        # pylint: disable=unbalanced-tuple-unpacking
+        classified, missing_by_branch_arch = result
+        chains_by_branch_arch = None
+


The pylint disable comment for 'unbalanced-tuple-unpacking' suggests a potential issue with tuple unpacking. The function resolve_and_classify_packages() returns different tuple lengths based on collect_chains parameter, which could lead to runtime errors if not handled correctly.

Suggested change

result = resolve_and_classify_packages(

packages_to_add, apk_indexes, provides_indexes, available_archs, collect_chains=print_chains

)

# Always unpack three values; chains_by_branch_arch may be None if not collected

if print_chains:

classified, missing_by_branch_arch, chains_by_branch_arch = result

else:

classified, missing_by_branch_arch, chains_by_branch_arch = result

rucoder · 2025-08-18T13:01:22Z

@rene @OhmSpectator are we merging it? eden is not required here at all

christoph-zededa · 2025-08-18T13:05:28Z

@rene @OhmSpectator are we merging it? eden is not required here at all

Famous last words before EVE crash ;-)

famleebob · 2025-08-18T13:45:40Z

Looks good! I look forward to using the scripts

famleebob

Looks good to me, nice clean scripts.

rene · 2025-08-18T17:01:53Z

These are just script tools, it has nothing with the execution of pillar, I think it's fine to merge...

christoph-zededa · 2025-08-20T15:46:09Z

BTW, at the request of @christoph-zededa , linuxkit recently got the ability to reference other packages automatically used build-args. So we actually could replace all of the explicit FROM lfedge/eve-alpine:12345aabbccdd in each pkg/*/Dockerfile with a build arg that automatically gets populated.

I know, not a big thing compared to this, but another piece to help ease building. I was going to add a PR for it, but I prefer to let a real user (hint hint @christoph-zededa ) do it, so we can see how well it works for someone who did not build it.

for reference: #5190

rucoder requested a review from deitch as a code owner August 14, 2025 17:59

github-actions bot requested review from eriknordmark and jsfakian August 14, 2025 18:00

rucoder force-pushed the rucoder/alpine-tools branch from 81b29f3 to d296fd9 Compare August 14, 2025 18:01

rucoder requested review from rene and famleebob August 14, 2025 18:03

rucoder force-pushed the rucoder/alpine-tools branch from d296fd9 to c91d17d Compare August 14, 2025 18:29

rucoder mentioned this pull request Aug 14, 2025

Commit scripts to update Alpine package lists to eve repo #5172

Open

rucoder force-pushed the rucoder/alpine-tools branch from c91d17d to fc2e860 Compare August 15, 2025 07:42

deitch approved these changes Aug 15, 2025

View reviewed changes

rene reviewed Aug 15, 2025

View reviewed changes

tools/alpine_migrate.py Outdated Show resolved Hide resolved

rucoder force-pushed the rucoder/alpine-tools branch from fc2e860 to 4f06969 Compare August 15, 2025 12:10

github-actions bot requested a review from deitch August 15, 2025 12:10

OhmSpectator requested a review from Copilot August 15, 2025 12:20

Copilot AI reviewed Aug 15, 2025

View reviewed changes

europaul mentioned this pull request Aug 18, 2025

Get go from go.dev instead of alpine's edge/community mirror #5184

Merged

6 tasks

famleebob approved these changes Aug 18, 2025

View reviewed changes

rene merged commit 099e0f1 into lf-edge:master Aug 18, 2025
39 of 45 checks passed

rucoder deleted the rucoder/alpine-tools branch August 19, 2025 12:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add Alpine Linux package management tools for EVE OS #5179

Add Alpine Linux package management tools for EVE OS #5179

Uh oh!

rucoder commented Aug 14, 2025 •

edited

Loading

Uh oh!

rene commented Aug 15, 2025

Uh oh!

deitch commented Aug 15, 2025

Uh oh!

Uh oh!

deitch commented Aug 15, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Aug 15, 2025

Uh oh!

Copilot AI Aug 15, 2025

Uh oh!

Copilot AI Aug 15, 2025

Uh oh!

rucoder commented Aug 18, 2025

Uh oh!

christoph-zededa commented Aug 18, 2025

Uh oh!

famleebob commented Aug 18, 2025

Uh oh!

famleebob left a comment

Uh oh!

rene commented Aug 18, 2025

Uh oh!

Uh oh!

christoph-zededa commented Aug 20, 2025

Uh oh!

Uh oh!

+    result = resolve_and_classify_packages(
+        packages_to_add, apk_indexes, provides_indexes, available_archs, collect_chains=print_chains
+    )
+    # Always unpack three values; chains_by_branch_arch may be None if not collected
+    if print_chains:
+        classified, missing_by_branch_arch, chains_by_branch_arch = result
+    else:
+        classified, missing_by_branch_arch, chains_by_branch_arch = result

Add Alpine Linux package management tools for EVE OS #5179

Add Alpine Linux package management tools for EVE OS #5179

Uh oh!

Conversation

rucoder commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How to test and validate this PR

Changelog notes

PR Backports

Checklist

Uh oh!

rene commented Aug 15, 2025

Uh oh!

deitch commented Aug 15, 2025

Uh oh!

Uh oh!

deitch commented Aug 15, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 15, 2025

Choose a reason for hiding this comment

Uh oh!

rucoder commented Aug 18, 2025

Uh oh!

christoph-zededa commented Aug 18, 2025

Uh oh!

famleebob commented Aug 18, 2025

Uh oh!

famleebob left a comment

Choose a reason for hiding this comment

Uh oh!

rene commented Aug 18, 2025

Uh oh!

Uh oh!

christoph-zededa commented Aug 20, 2025

Uh oh!

Uh oh!

rucoder commented Aug 14, 2025 •

edited

Loading