Re-add actions to build xmake with cosmocc

[A2va]

Is your feature request related to a problem? Please describe.

Previously, a workflow was created to build xmake with cosmocc
https://github.com/xmake-io/xmake/blob/b657911e2b0c34674570368c73c5025eff9d7649/.github/workflows/cosmocc.yml
However, this was not successful as it didn't run, but I noticed that this build was made using macos, so I tried to build it on linux.

First, I just tested that a normal xmake build (without embed) would work

a2va:~/xmake/core$ xmake f -p linux --cosmocc=y

The executable runs fine, then I enabled embed

a2va:~/xmake/core$ xmake f -p linux --cosmocc=y

Again, no problem with the executable, then I copied the file to Windows (the build was done in WSL Ubuntu). And it starts, but with this error:

PS D:\\xm-tt> .\\xmake-tt.exe --version
Error: @programdir/core/base/task.lua:398: @programdir/core/base/winos.lua:238: Field '_registry_query' is not accessible (a null value)
Stack traceback:
    [@programdir/core/base/winos.lua:238]: in function 'registry_query'.
    [@programdir/core/base/cpu.lua:222]: in function '_info'.
    [@programdir/core/base/cpu.lua:286]: in function 'Vendor'
    [@programdir/core/base/cpu.lua:352]:
    [@programdir/core/base/os.lua:1443]: in function 'default_njob'.
    [@programdir/plugins/pack/xmake.lua:31]: in main chunk

I suspect this is because Cosmopolitan doesn't implement the Windows API for editing the registry, but those can added if needed (eg: jart/cosmopolitan#1318), or the other solution would be to completely disable this functionality.

Describe the solution you'd like

With this feature the xmake bundle executable which is for now windows only could be extended to other platforms.

Describe alternatives you've considered

No response

Additional context

No response

[waruqi]

I tried it before, but it still has many problems, such as it will stuck on macos ci and some windows api problems.

But I don't have time to solve them recently, so I can only delete it temporarily. If you can solve these problems, you can submit a PR.

[A2va]

I managed to built xmake in the CI but one of the test fails, I don't know if this is due to cmake or not.

> /usr/bin/gcc -c -m64 -DAL_LIBTYPE_STATIC -isystem /home/runner/.xmake/packages/o/openal-soft/1.23.1/ac313711f22a45f2a1a2f9b24f64d1d1/include -isystem /home/runner/.xmake/packages/l/libsndio/1.9.0/aa973ae5d04a4f53a02cf28d02c69457/include -isystem /home/runner/.xmake/packages/a/alsa-lib/1.2.10/60f829908b344b8193f7d458e8a93b5a/include -o /tmp/.xmake1001/241227/_B5599DC13D654B00859AC66A1E9E1B10.o /tmp/.xmake1001/241227/_76360DFFA3454E3488F03C9FE8D87B65.c
> /usr/bin/g++ -o /tmp/.xmake1001/241227/_B5599DC13D654B00859AC66A1E9E1B10.b /tmp/.xmake1001/241227/_B5599DC13D654B00859AC66A1E9E1B10.o -m64 -L/home/runner/.xmake/packages/o/openal-soft/1.23.1/ac313711f22a45f2a1a2f9b24f64d1d1/lib -L/home/runner/.xmake/packages/l/libsndio/1.9.0/aa973ae5d04a4f53a02cf28d02c69457/lib -L/home/runner/.xmake/packages/a/alsa-lib/1.2.10/60f829908b344b8193f7d458e8a93b5a/lib -lopenal -lsndio -latopology -lasound -lpthread -ldl
> checking for c includes(AL/al.h)
> checking for c funcs(alGetProcAddress)
> checking for c links(openal, sndio, atopology, asound, pthread, dl)
> checking for c snippet(has_cfuncs)
  => install openal-soft 1.23.1 .. ok
  => download https://www.sfml-dev.org/files/SFML-2.5.1-sources.zip .. ok
error: @programdir/core/sandbox/modules/os.lua:273: -- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - donCMake Error at /usr/local/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find OpenGL (missing: OPENGL_opengl_LIBRARY OPENGL_glx_LIBRARY
  OPENGL_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/local/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake-3.31/Modules/FindOpenGL.cmake:579 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  cmake/Macros.cmake:279 (find_package)
  src/SFML/Window/CMakeLists.txt:257 (sfml_find_package)


r remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Configuring incomplete, errors occurred!

stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [@programdir/core/sandbox/modules/os.lua:273]:
    [@programdir/core/sandbox/modules/os.lua:291]: in function 'vrunv'
    [@programdir/modules/package/tools/cmake.lua:1317]: in function 'configure'
    [@programdir/modules/package/tools/cmake.lua:1359]: in function 'install'
    [./xmake.lua:192]: in function 'script'
    [...dir/modules/private/action/require/impl/utils/filter.lua:114]: in function 'call'
    [.../modules/private/action/require/impl/actions/install.lua:452]:

  => install sfml 2.5.1 .. failed
error: @programdir/core/main.lua:329: @programdir/modules/async/runjobs.lua:325: .../modules/private/action/require/impl/actions/install.lua:561: install failed!
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [.../modules/private/action/require/impl/actions/install.lua:561]: in function 'catch'
    [@programdir/core/sandbox/modules/try.lua:123]: in function 'try'
    [.../modules/private/action/require/impl/actions/install.lua:419]:
    [...modules/private/action/require/impl/install_packages.lua:510]: in function 'jobfunc'
    [@programdir/modules/async/runjobs.lua:241]:

stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:1075: in function 'os.raiselevel'
	(...tail calls...)
	@programdir/core/main.lua:329: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:406: in function <@programdir/core/base/scheduler.lua:399>
>>     test failed: @programdir/core/sandbox/modules/os.lua:337: exec(xmake f -c -D -y) failed(255)
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [@programdir/core/sandbox/modules/os.lua:337]: in function 'exec'
    [./tests/test_utils/test_build.lua:4]: in function 'build'
    [./tests/projects/package/components/test.lua:10]:

>>       function test_main tests/projects/package/components/test.lua:1
error: @programdir/core/main.lua:329: ./tests/test_utils/print_error.lua:46: aborting because of unhandled error ...
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [./tests/test_utils/print_error.lua:46]: in function 'print_error'
    [./tests/runner.lua:58]: in function 'catch'
    [@programdir/core/sandbox/modules/try.lua:123]: in function 'try'
    [./tests/runner.lua:45]:
    [./tests/run.lua:7]: in function '_run_test'
    [./tests/run.lua:42]:
    [@programdir/plugins/lua/main.lua:124]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:244]:
    [@programdir/core/base/task.lua:491]: in function 'run'
    [@programdir/core/main.lua:327]: in function 'cotask'
    [@programdir/core/base/scheduler.lua:406]:

stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:1075: in function 'os.raiselevel'
	(...tail calls...)
	@programdir/core/main.lua:329: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:406: in function <@programdir/core/base/scheduler.lua:399>
Error: Process completed with exit code 255.

https://github.com/A2va/xmake/blob/cosmocc-ci/.github/workflows/cosmocc.yml
https://github.com/A2va/xmake/actions/runs/12518925059/job/34922147269#step:7:2163

[waruqi]

I don't know why. I haven't had time to look at it, but if you can get it to pass on all ci tests, you can open a PR to rebuild xmake with cosmocc binary

[A2va]

I just updated my fork to the latest commit of the master branch and ran to both linux and cosmocc ci. Both fail on the same test, which seems to indicate that xmake cosmocc is acting the same as the regular one.

What is weird is that the same test on the xmake main repo with the linux ci passes.

error: @programdir/core/main.lua:329: @programdir/actions/build/main.lua:148: @programdir/modules/async/runjobs.lua:325: @programdir/actions/build/kinds/binary.lua:53: @programdir/core/sandbox/modules/os.lua:378: execv(/usr/bin/clang++ -o build/linux/x86_64/release/dependence2 build/.objs/dependence2/linux/x86_64/release/src/main.cpp.o build/.objs/dependence2/linux/x86_64/release/src/foo.mpp.o build/.objs/dependence2/linux/x86_64/release/src/bar.mpp.o build/.objs/dependence2/linux/x86_64/release/src/zoo.mpp.o build/.objs/dependence2/linux/x86_64/release/src/cat.mpp.o -m64 -s -stdlib=libc++) failed(1)
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:1075]:
    [@programdir/core/sandbox/modules/os.lua:378]: in function 'execv'
    [@programdir/modules/core/tools/gcc.lua:599]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:244]:
    [@programdir/core/tool/linker.lua:232]: in function 'link'
    [@programdir/actions/build/kinds/binary.lua:53]: in function 'callback'
    [@programdir/modules/core/project/depend.lua:217]: in function 'on_changed'
    [@programdir/actions/build/kinds/binary.lua:41]: in function '_do_link_target'
    [@programdir/actions/build/kinds/binary.lua:83]:
    [@programdir/actions/build/kinds/binary.lua:110]: in function '_link_target'
    [@programdir/actions/build/kinds/binary.lua:138]: in function 'jobfunc'
    [@programdir/modules/async/runjobs.lua:241]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:244]: in function 'trycall'
    [@programdir/core/sandbox/modules/try.lua:117]: in function 'try'
    [@programdir/modules/async/runjobs.lua:224]: in function 'cotask'
    [@programdir/core/base/scheduler.lua:406]:

stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:1075: in function 'os.raiselevel'
	(...tail calls...)
	@programdir/core/main.lua:329: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:406: in function <@programdir/core/base/scheduler.lua:399>
warning: std and std.compat modules not found! maybe try to add --sdk=<PATH/TO/LLVM> or install libc++
warning: No clang-scan-deps found ! using fallback scanner

Linux: https://github.com/A2va/xmake/actions/runs/12675795887/job/35327578205#step:8:944
Cosmocc: https://github.com/A2va/xmake/actions/runs/12675715771/job/35327336647#step:7:958

[waruqi]

I don't know why. you can diff ci yml.

[A2va]

The PR #6065 seems to have the test problem I was getting on my fork, so I made a PR to add a cosmocc action.

There is still this windows crash that I need to resolve. At first I tought that it was a missing stub in the win32 api of cosmocc, but that does not seems the case.
What I suspect is that the winos functions is not compiled because TB_CONFIG_OS_WINDOWS is not defined.

xmake/core/src/xmake/engine.c

Lines 383 to 398 in 58c00fc

	#ifdef TB_CONFIG_OS_WINDOWS
	static luaL_Reg const g_winos_functions[] =
	{
	{ "cp_info", xm_winos_cp_info }
	, { "console_cp", xm_winos_console_cp }
	, { "console_output_cp", xm_winos_console_output_cp }
	, { "oem_cp", xm_winos_oem_cp }
	, { "ansi_cp", xm_winos_ansi_cp }
	, { "logical_drives", xm_winos_logical_drives }
	, { "registry_query", xm_winos_registry_query }
	, { "registry_keys", xm_winos_registry_keys }
	, { "registry_values", xm_winos_registry_values }
	, { "short_path", xm_winos_short_path }
	, { tb_null, tb_null }
	};
	#endif

And so the winos function doesn't get registered with ``xm_lua_register'', and it returns nil.

xmake/core/src/xmake/engine.c

Line 1356 in 58c00fc

#ifdef TB_CONFIG_OS_WINDOWS

I tried to build in debug, to see if the stacktrace tells a bit more

[xmake]: init: ..
[xmake]: version: xmake_v2_9_7_20250114_x86_64_sse2_elf by gnu c/c++ 14.1
[tbox]: init: ..
[tbox]: [backtrace]: [warning]: noimpl at tb_backtrace_frames(): 136, src/tbox/tbox/src/tbox/platform/libc/backtrace.c
[tbox]: version: tbox_v1_7_6_20250114_x86_64_sse2_elf by gnu c/c++ 14.1
[tbox]: init: ok
[xmake]: init: ok
[xmake]: [engine]: project: /D/Telechargements/xmake-cosmocc-x86_64
[xmake]: [engine]: programfile: /D/Telechargements/xmake-cosmocc-x86_64/xmakec.exe
[xmake]: [engine]: programdir: /tmp/.xmake13793/2.9.7+test-cosmocc.34a7088
[xmake]: [engine]: main: /tmp/.xmake13793/2.9.7+test-cosmocc.34a7088/core/_xmake_main.lua
error: @programdir/core/base/task.lua:400: @programdir/core/base/winos.lua:242: attempt to call a nil value (field '_registry_query')
stack traceback:
    [@programdir/core/base/winos.lua:242]: in function 'registry_query'
    [@programdir/core/base/cpu.lua:222]: in function '_info'
    [@programdir/core/base/cpu.lua:286]: in function 'vendor'
    [@programdir/core/base/cpu.lua:352]:
    [@programdir/core/base/os.lua:1514]: in function 'default_njob'
    [@programdir/plugins/pack/xmake.lua:31]: in main chunk

[tbox]: [lock_profiler]: 
[tbox]: [lock_profiler]: lock: 0x6ffffffb07a4, name: native_large_allocator, occupied: 0
[tbox]: [lock_profiler]: lock: 0x6ffffffb214c, name: allocator, occupied: 0
[tbox]: [small_allocator]: 
[tbox]: [static_fixed_pool]: [16]: peak_size: 48, wast_rate: 4074/10000, pred_failed: 0, item_maxn: 245, free_count: 5, malloc_count: 5
[tbox]: [static_fixed_pool]: [32]: peak_size: 32, wast_rate: 2558/10000, pred_failed: 0, item_maxn: 246, free_count: 2, malloc_count: 2
[tbox]: [static_fixed_pool]: [64]: peak_size: 64, wast_rate: 1466/10000, pred_failed: 0, item_maxn: 247, free_count: 19, malloc_count: 19
[tbox]: [static_fixed_pool]: [128]: peak_size: 384, wast_rate: 791/10000, pred_failed: 0, item_maxn: 249, free_count: 206, malloc_count: 206
[tbox]: [static_fixed_pool]: [192]: peak_size: 384, wast_rate: 541/10000, pred_failed: 0, item_maxn: 250, free_count: 166, malloc_count: 166
[tbox]: [static_fixed_pool]: [256]: peak_size: 256, wast_rate: 411/10000, pred_failed: 0, item_maxn: 251, free_count: 1, malloc_count: 1
[tbox]: [static_fixed_pool]: [384]: peak_size: 1152, wast_rate: 278/10000, pred_failed: 0, item_maxn: 252, free_count: 3, malloc_count: 3
[tbox]: [static_fixed_pool]: [512]: peak_size: 512, wast_rate: 210/10000, pred_failed: 0, item_maxn: 253, free_count: 1, malloc_count: 1
[tbox]: [static_fixed_pool]: [1024]: peak_size: 3072, wast_rate: 106/10000, pred_failed: 0, item_maxn: 254, free_count: 233, malloc_count: 233
[tbox]: [static_fixed_pool]: [2048]: peak_size: 2048, wast_rate: 53/10000, pred_failed: 0, item_maxn: 255, free_count: 33, malloc_count: 33
[tbox]: [static_fixed_pool]: [3072]: peak_size: 3072, wast_rate: 35/10000, pred_failed: 0, item_maxn: 255, free_count: 9, malloc_count: 9
[tbox]: [native_large_allocator]: 
[tbox]: [native_large_allocator]: peak_size: 2467336
[tbox]: [native_large_allocator]: wast_rate: 12/10000
[tbox]: [native_large_allocator]: free_count: 285
[tbox]: [native_large_allocator]: malloc_count: 285
[tbox]: [native_large_allocator]: ralloc_count: 96
[tbox]: exit: ok

How can I fix this ?

[waruqi]

I don't know how cosmocc calls the windows api。

[A2va]

They have a windows.h as well as some stub functions for win32
https://github.com/jart/cosmopolitan/blob/master/libc/nt/windows.h
https://github.com/jart/cosmopolitan/blob/master/libc/nt/advapi32/RegCloseKey.S

I will try to compile the winos module with cosmocc, and adding some check at runtime in the lua script to verify that is runs on windows.

[A2va]

I created a PR on the cosmopolitan project to add some win32 apis that xmake relies on but were not present in the compiler. jart/cosmopolitan#1358

[waruqi]

ok, thanks