|
surma
|
|
Aryan Pingle
Does JXL support pure single threading when building with emscripten? I can't seem to build without passing the `-pthread` flag even for the JXLDecoder API (version 0.7)
|
|
2023-08-30 11:07:07
|
|
|
2023-08-30 11:07:08
|
<@794205442175402004> <@179701849576833024> Can you help us out with this one? We are trying to update JXL for squoosh.app, but we don’t seem to be able to create a single-threaded build anymore, which seems like it might break certain devices from running our JXL encoder/decoder
|
|
|
Jake Archibald
|
2023-08-30 11:08:23
|
Not all browsers support wasm threads in all cases, so we want a single threaded build as a fallback
|
|
|
surma
|
2023-08-30 11:11:53
|
Specifically, it seems like this commit change the behavior that every emscripten build uses `-pthread`, regardless of whether Emscripten was invoked with `-pthread` or not. We tried monkey patching this out, but then the build fails, IIRC. So are single threaded builds now unsupported?
https://github.com/libjxl/libjxl/commit/ef97fe07656e385bbd1af3801635b66bdbd413f2#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20aR237-R242
|
|
|
|
veluca
|
2023-08-30 11:14:21
|
I have to say I haven't looked at wasm builds in a while
|
|
2023-08-30 11:14:30
|
probably since that commit 😛
|
|
|
surma
|
|
|
veluca
|
2023-08-30 11:14:43
|
but at least in principle doing a single thread build should be possible
|
|
2023-08-30 11:15:15
|
and I don't even think it would require significant changes
|
|
|
surma
|
2023-08-30 11:15:24
|
Okay, so I guess maybe the changes to the CMake file were just a bit overreaching? Not every Emscripten build _must_ be multithreaded, right? (Cause that’s what that commit is kinda introducing)
|
|
2023-08-30 11:15:49
|
Then we’d only need your help figuring out why the single threaded build is failing
|
|
2023-08-30 11:16:22
|
(to be clear: I can’t quite remember if the _build_ is failing or if the single-threaded module didn’t load/error’d...)
|
|
|
|
veluca
|
2023-08-30 11:16:52
|
I suspect the build is failing
|
|
2023-08-30 11:19:44
|
anyway, I might be able to give this a look at some point, but it'd be nice if I had some more precise repro instructions 😄
|
|
|
Jake Archibald
|
2023-08-30 11:20:08
|
We're doing a build now, to give you the error message we're seeing
|
|
|
surma
|
2023-08-30 11:20:09
|
Yeah I am working on that 😅
|
|
2023-08-30 11:35:19
|
I am waiting for Docker to do it’s thing lol. In the meantime: The current cmake is problematic for the following reasons: We are compiling libjxl.a without pthread, but the CMake injects phtread. So the libjxl.a ends up beig multithreaded anyway. However, our CPP code that links against libjxl.a is not single threaded, and the JS glue code generated by Emscripten is also not expecting multithreaded code, and so embind fails to transfer values between wasm and JS (returning a typed array from CPP to JS just throws)
|
|
2023-08-30 11:53:40
|
<@179701849576833024> Okay I was wrong. Removing that section form the CMake file is the solution. It builds fine and is single threaded (and multithreaded builds still work if you pass -pthread).
If it’s okay wiht you, <@606193820052488192> will open a PR so you can run your tests
|
|
|
|
veluca
|
2023-08-30 11:54:18
|
sure
|
|
2023-08-30 11:54:33
|
less work is always good 😛
|
|
|
Aryan Pingle
|
2023-09-04 12:37:51
|
Hey <@179701849576833024> thanks for your patience. I made 2 PR's in the Squoosh repo to explain the problem:
1. https://github.com/GoogleChromeLabs/squoosh/pull/1379 - Building and using libjxl v0.7.0 as is (throws an error in the JS glue code after the WASM finishes decoding successfully)
2. https://github.com/GoogleChromeLabs/squoosh/pull/1378 - libjxl after removing the CMakeLists.txt lines that <@228116142185512960> linked above (works perfectly)
You can view the sites live at https://deploy-preview-1379--squoosh.netlify.app and https://deploy-preview-1378--squoosh.netlify.app respectively
|
|
|
|
veluca
|
2023-09-04 12:42:56
|
you had convinced me of the problem already 🙂 feel free to open a PR to libjxl with the cmake change?
|
|
|
Aryan Pingle
|
2023-09-04 12:43:22
|
sure thing!
|
|
2023-09-04 01:02:26
|
PR: https://github.com/libjxl/libjxl/pull/2771
I'll add a description and context in some time
|
|
|
|
veluca
|
2023-09-04 02:05:19
|
mhhh looks like that causes issues with the internal build: https://github.com/libjxl/libjxl/actions/runs/6073883619/job/16477794072?pr=2771
|
|
|
Aryan Pingle
|
2023-09-04 02:08:32
|
Hmm I ran into something *kinda* similar on my local build
|
|
2023-09-04 02:09:08
|
After changing the CMakeLists.txt file the build would fail, but on running the exact same build command it built correctly
|
|
2023-09-04 02:24:28
|
<@179701849576833024> It's correctly building https://github.com/libjxl/libjxl/commit/f95da131cf7c7ccd4da256356fde2fec1fa23bb5 v0.7.0 without that block of code. Do you think I should I try to find the commit/s where it's failing?
|
|
|
|
veluca
|
2023-09-04 02:25:20
|
if you can that'd be ideal
|
|
|
Aryan Pingle
|
2023-09-04 02:28:13
|
Yep I'll try, lot of commits to go through though RIP
|
|
2023-09-05 01:43:32
|
<@179701849576833024> I checked out my PR and it built and installed libjxl without errors. Here's the bash code I used to build it:
```bash
cd libjxl
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF ..
cmake --build . -- -j$(nproc)
```
And to install it:
```bash
sudo cmake --install .
```
|
|
2023-09-05 01:43:47
|
(copied exactly from the README)
|
|
|
|
veluca
|
2023-09-05 01:44:08
|
was that a WASM build?
|
|
|
Aryan Pingle
|
2023-09-05 01:44:31
|
Nope I'm just building libjxl
|
|
|
|
veluca
|
2023-09-05 01:44:45
|
the CI failure I linked above is for building libjxl for the WASM target
|
|
|
Aryan Pingle
|
2023-09-05 01:45:00
|
Ohh I see
|
|
2023-09-05 01:45:11
|
Cool I'll try building the WASM
|
|
|
|
veluca
|
2023-09-05 01:46:03
|
see this workflow file: https://github.com/libjxl/libjxl/blob/main/.github/workflows/build_test.yml#L361
|
|
|
Aryan Pingle
|
2023-09-05 01:46:17
|
Yep was just about to ask that, thanks
|
|
2024-01-11 10:44:24
|
Update: https://github.com/libjxl/libjxl/pull/2771 Looks like we'll get an option that allows us to use single-threaded JXL
|
|