Libsass/Too many files error on big sites when running server on MacOS

[FuadEfendi]

Error: error building site: TOCSS: failed to transform

What version of Hugo are you using (hugo version)?

0.128.1
0.128.2

Does this issue reproduce with the latest release?

Yes

Please see discussion at https://discourse.gohugo.io/t/hugo-embedded-web-server-limitations-error-tocss/50543 and https://github.com/orgs/hbstack/discussions/68

This is test site with randomly generated 40,000 files:
https://github.com/tokenizer-inc-canada/large-site-test

hugo server command fails with Error: error building site: TOCSS: failed to transform ... error message, and hugo command fails with different error message.

Another user, @razonyang tried on Windows, and it doesn't fail.

I tried on MacBook M1 Pro with 64G, and on MacBook Air M1, and it fails on both, tried with 0.127.0, 0.128.1, 0.128.2.

If I randomly delete any 10,000 files, build doesn't fail.

With configured "segments" feature, when I try to build small subset, build still fails, and fails in faster time.

[bep]

I have briefly looked at your site, and generating 40K content files with lots of random taxonomy values is blowing up the number of files, and is not what the taxonomies were designed for:

If you can create a more realistic breaking example, I can have a look at it, but I'm not going to fill my disk/CPU debugging this crazy stuff.

Please reopen if you have something more real.

[FuadEfendi]

I have few public websites (with average 35k -50k pages) having this issue on Mac (and not having with Netlify build). Finally I suspected that it cannot be malformed file, tried to reproduce, and reproduced.

[FuadEfendi]

I started encountering this error a few weeks ago and did a lot of research and spent many hours on it:

Error: error building site: TOCSS: failed to transform

Now, we documented it which is a good thing ;)

No need to waste time on it, I agree, it may unpredictably disappear in some new versions.

However, unfortunately, building by 20x smaller segments also fails with same error message... in Linux, I'd assume that ulimit nofile should be increased; not sure about Mac; could be internal bug in Mac.

[bep]

OK, I can reproduce this.

The odd thing is that even

hugo mod vendor
hugo server

Fails with ...

File to import not found or unreadable: /Users/bep/dump/large-site-test/_vendor/github.com/twbs/bootstrap/scss/_functions.scss.

This doesn't make sense, but it's certainly easier to reason about when having the deps vendored.

[FuadEfendi]

I can imagine weird path being generated by broken platform-specific tools, such as ../abc/../abc/../abc/.. of very large size, and we have file not found error; some internal tools are platform specific, in case of SCSS for example, SAAS compiler; maybe need to run npm audit fix etc.; something is platform-specific.