perf(web): use hash-wasm SHA-1 with pure-JS fallback for upload hashing#30107
Open
mosh-dev wants to merge 4 commits into
Open
perf(web): use hash-wasm SHA-1 with pure-JS fallback for upload hashing#30107mosh-dev wants to merge 4 commits into
mosh-dev wants to merge 4 commits into
Conversation
Adds a WebAssembly-accelerated SHA-1 path (hash-wasm) for hashing files during upload, tried first and falling back to the existing @noble/hashes pure-JS implementation if the WASM path throws for any reason. Benchmarked ~1.68x faster than the pure-JS path on large files, with identical hash output between the two.
|
Label error. Requires exactly 1 of: changelog:.*. Found: 🖥️web. A maintainer will add the required label. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
File upload hashing in the web client currently runs SHA-1 entirely in pure JS (
@noble/hashes) inside a Web Worker. This is a CPU-bound bottleneck for large file uploads, since JS-implemented SHA-1 is slower than a compiled/WASM implementation.This PR adds a WebAssembly-accelerated SHA-1 path using
hash-wasm, which is tried first. If WebAssembly is unsupported in the runtime, or the WASM path throws for any reason, hashing falls back to the existing pure-JS@noble/hashesimplementation, so correctness and browser compatibility are preserved.Benchmarked ~1.68x faster (on Intel 13th Gen 13650HX hardware) than the pure-JS path on large files, with identical hash output between the two implementations.
I'm a personal Immich user. A recent update's slower hashing made me curious whether I could improve it, so I looked into this and thought it might be a good speed boost. Please feel free to reject this pull request if there are any major concerns, and I'm happy to discuss further if any clarification is needed.
Fixes # (issue)
How Has This Been Tested?
Added
web/src/lib/workers/hash-file.spec.tscovering:hashFileJsproduces the correct known SHA-1 hex digest for a fixed inputhashFileWasmproduces the correct known SHA-1 hex digest for the same inputhashFileWasmandhashFileJsproduce identical output across a range of file sizes (0, 1, 1024, 10000 bytes)hashFilefalls back to the JS implementation when the WASM path throws (mockedhash-wasmrejection)hashFileuses the WASM path when available and matches the JS outputChecklist:
src/services/uses repositories implementations for database calls, filesystem operations, etc.src/repositories/is pretty basic/simple and does not have any immich specific logic (that belongs insrc/services/)Please describe to which degree, if any, an LLM was used in creating this pull request.
This utility function was previously implemented, tested in production, and used inside a web worker for another project that's still in production. I asked the LLM (Claude Code) to reuse that function for this change. Claude Code wrote the spec file and ran the tests; the WebAssembly-support detection/fallback logic is hand-written by me and tested across different browsers. I reviewed the diff, understand the change, and take responsibility for it.