Using new build system from v2025.5.0.rc1, receive errors when building amuse/ amuse-framework

[bbanne]

When attempting to build the amuse packages in a new conda env (called amuse), I am receiving errors during the outlined process. I will include files with terminal and logs, but the summary is that I created and activated a new conda env, installed the dependencies as outlined in the build instructions and the output of the setup script, but when I go to run ./setup install all, I receive the following error:

--------------------------------------------------------------------------
The Open MPI wrapper compiler was unable to find the specified compiler
x86_64-conda_cos6-linux-gnu-cc in your PATH.

Note that this compiler was either specified at configure time or in
one of several possible environment variables.
--------------------------------------------------------------------------
make[1]: *** [support/shared/lib-targets.mk:103: amuse_mpi.mo] Error 1
make[1]: Leaving directory '/opt/amuse/amuse-2025.5.0.rc1/lib/amuse_mpi'
make: *** [Makefile:17: amuse_mpi] Error 2
make: Leaving directory '/opt/amuse/amuse-2025.5.0.rc1/lib'
support/setup/installing.sh: line 98: ../support/logs/install-amuse-framework.exit_code: No such file or directory
cat: support/logs/install-amuse-framework.exit_code: No such file or directory

amuse-framework failed to build and install correctly.

This is not supposed to happen!

Please contact the developers on Slack or make an issue on GitHub,
so that we can fix this together.

I definitely have the prerequisites installed, and the env active, so if there was something else I needed to change as far en environment variables before beginning the installation, it wasn't included in either the docs or the setup script process.

Our environment is rhel 8.10 with a conda version of 24.7.1. I've included output from the setup and install process, install logs, and the listing of all packages installed in our conda environment. Any additional guidance is appreciated. I am sure it is probably something I have missed.

TerminalOutputAmuse.txt
CondaEnvAmuse.txt

[LourensVeen]

Hi! Thanks for the report!

I think the issue may be that you've got the dependencies installed from different channels. You have compilers from conda-forge, but openmpi from somewhere else (probably the main Anaconda channel), and I think that they don't understand each other and that that's why you're getting that error.

To check whether that's the case, could you activate the environment and then type

conda info

For me, that shows

           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch

and I suspect that you may have the same channels but with pkgs above conda-forge, and that you installed some of the dependencies without specifying -c conda-forge, so that they ended up coming from pkgs instead.

If that's really what happened, and you don't need this particular environment, then it's easiest to remove the environment (conda deactivate and then conda env remove -n amuse) and make a new one that has conda-forge as the only channel:

conda create -c conda-forge --override-channels -n Amuse-env

and then try again. That should avoid mixups.

Please let me know if that solves the problem, or not, if it does then I'm going to add that --override-channels to the command that ./setup prints to help other people avoid this trap.

[bbanne]

Thanks for the guidance. That did help, and I made ti further along in the build this time. It would be valuable to add that to the documentation for other people in the future.

The issue I hit now is with a "shallow clone"

The end of the build log:

make: Leaving directory '/opt/amuse/amuse-2025.5.0.rc1/lib'
Processing /opt/amuse/amuse-2025.5.0.rc1/src
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [35 lines of output]
      Traceback (most recent call last):
        File "/opt/anaconda3/envs/amuse/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
          ~~~~^^
        File "/opt/anaconda3/envs/amuse/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
                                   ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/anaconda3/envs/amuse/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 178, in prepare_metadata_for_build_whe
el
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
        File "/tmp/pip-build-env-hp5eaz7h/overlay/lib/python3.13/site-packages/hatchling/build.py", line 58, in build_wheel
          return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard'])))
                                  ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-hp5eaz7h/overlay/lib/python3.13/site-packages/hatchling/builders/plugin/interface.py", line 90, in build
          self.metadata.validate_fields()
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
        File "/tmp/pip-build-env-hp5eaz7h/overlay/lib/python3.13/site-packages/hatchling/metadata/core.py", line 265, in validate_fields
          _ = self.version
              ^^^^^^^^^^^^
        File "/tmp/pip-build-env-hp5eaz7h/overlay/lib/python3.13/site-packages/hatchling/metadata/core.py", line 149, in version
          self._version = self._get_version()
                          ~~~~~~~~~~~~~~~~~^^
        File "/tmp/pip-build-env-hp5eaz7h/overlay/lib/python3.13/site-packages/hatchling/metadata/core.py", line 248, in _get_version
          version = self.hatch.version.cached
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-hp5eaz7h/overlay/lib/python3.13/site-packages/hatchling/metadata/core.py", line 1456, in cached
          raise type(e)(message) from None
      RuntimeError: Error getting the version from source `versioningit`: NotSdistError: /opt/amuse/amuse-2025.5.0.rc1/src does not contain a PKG-INFO file
      
      versioningit could not find a version for the project in /opt/amuse/amuse-2025.5.0.rc1/src!
      
      You may be installing from a shallow clone, in which case you need to unshallow it first.
      
      Alternatively, you may be installing from a Git archive, which is not supported by default.  Install from a git+https://... URL instead.
      
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

When I cloned the repo a week or so ago, I used this command:
curl -L -O "https://github.com/amusecode/amuse/archive/refs/tags/v2025.5.0.rc1.tar.gz"

Was this not the correct way?

[LourensVeen]

Ah, apologies, yes it is but we had an issue building from tar archives.

What you've got there is actually not a clone, which is a copy of a git repository including git metadata, but an archive, which just contains the files stored in that repository without any git stuff. The AMUSE build system determines which version we're building from the git metadata, and an archive doesn't contain that, so it fails.

I've added a fix for this in v2025.5.0.rc2 a couple weeks ago, so downloading that instead should fix it. It's a bit of a work in progress, fixing #1157 may change how it works again internally but that won't make a difference to users, only to developers.

Sorry for the mess, I think we're almost there with this new build system but it still has some teething issues, as you've unfortunately discovered 😦.

[risojevicv]

Unfortunately, the shallow clone problem persists in v2025.5.0.rc2. However, when I clone the repo, git clone https://github.com/amusecode/amuse.git, I can install amuse-framework and majority of packages.

I ran into problems with amuse-seba. In src/amuse_seba/toolbox.patch I had to change
-CXXFLAGS += -I./include -I./include/star -DHAVE_CONFIG_H -DTOOLBOX
to
-CXXFLAGS += -I./include -I./include/star -DHAVE_CONFIG_H -DTOOLBOX -Wconversion.
After that, the build process started, but failed with
/home/vlado/anaconda3/envs/amuse/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find src/node/libnode.a: No such file or directory collect2: error: ld returned 1 exit status make: *** [Makefile:53: seba_worker] Error 1
Apparently, src/node/libnode.a library is missing.

[LourensVeen]

Interesting, I just tried and I can build successfully from the v2025.5.0.rc2.tar.gz file, except that the fix seems to have not made it to simplex, because that still fails. I'll fix that.

@risojevicv, Could you paste the contents of the VERSION file in the top amuse-2025.5.0.rc2 directory? It should read

# This file is used to record the version of AMUSE in a .tar.gz archive made using git
# archive. The build system will pick it up from here if it is not building from a git
# repository but from that tar.gz file.

v2025.5.0.rc2

As for the SeBa issue, I can't reproduce it. I made a separate issue for that at #1159, maybe you can paste the whole build log as explained there? It's clear that libnode.a is missing, but we need to know why and that's in that log somewhere. Thanks!

[risojevicv]

This is the contents of the VERSION file:

# This file is used to record the version of AMUSE in a .tar.gz archive made using git
# archive. The build system will pick it up from here if it is not building from a git
# repository but from that tar.gz file.

v2025.5.0.rc2

I downloaded it with curl -L -O "https://github.com/amusecode/amuse/archive/refs/tags/v2025.5.0.rc2.tar.gz".

I also attached the build log for amuse-framework:
install-amuse-framework.log

[LourensVeen]

Ha! That took a bit of figuring out, but I think I've found the problem: the code that looks for the VERSION file actually looks in the wrong location. I somehow had a VERSION file in that wrong location when I tested things, so it worked for me, but for the wrong reasons. So rc2 is still broken. I'll go and apply a fix and we'll do an rc3. And maybe I should add a CI run that tests installing from a tarball, while I'm working on that...

[LourensVeen]

Hi again, sorry the the long delay, I'm juggling way too many balls at the moment. But I've made some progress!

The SeBa issue should have been taken care of by #1160. I've made a new PR (#1167) that fixes the VERSION file problem and adds --channel-override to the conda command to hopefully help avoid mixed package sources if you're using a pre-existing environment. It also hopefully fixes the version problem properly this time.

I'll make an rc3 as soon as that is merged, and then we can try again.

I think you've got a working install now though using the git clone? In that case feel free to call it good enough, but please let me know so I can close this issue when #1167 is merged.

[risojevicv]

Yes, the installation from the git clone was successful. When you make rc3 I could test it again.