JPEG XL

Info

rules 57
github 35276
reddit 647

JPEG XL

tools 4225
website 1655
adoption 20712
image-compression-forum 0

General chat

welcome 3810
introduce-yourself 291
color 1414
photography 3435
other-codecs 23765
on-topic 24923
off-topic 22701

Voice Channels

General 2147

Archived

bot-spam 4380

Does JXL support pure single threading

surma
Aryan Pingle Does JXL support pure single threading when building with emscripten? I can't seem to build without passing the `-pthread` flag even for the JXLDecoder API (version 0.7)
2023-08-30 11:07:07
2023-08-30 11:07:08
<@794205442175402004> <@179701849576833024> Can you help us out with this one? We are trying to update JXL for squoosh.app, but we don’t seem to be able to create a single-threaded build anymore, which seems like it might break certain devices from running our JXL encoder/decoder
Jake Archibald
2023-08-30 11:08:23
Not all browsers support wasm threads in all cases, so we want a single threaded build as a fallback
surma
2023-08-30 11:11:53
Specifically, it seems like this commit change the behavior that every emscripten build uses `-pthread`, regardless of whether Emscripten was invoked with `-pthread` or not. We tried monkey patching this out, but then the build fails, IIRC. So are single threaded builds now unsupported? https://github.com/libjxl/libjxl/commit/ef97fe07656e385bbd1af3801635b66bdbd413f2#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20aR237-R242
veluca
2023-08-30 11:14:21
I have to say I haven't looked at wasm builds in a while
2023-08-30 11:14:30
probably since that commit 😛
surma
2023-08-30 11:14:36
lol
veluca
2023-08-30 11:14:43
but at least in principle doing a single thread build should be possible
2023-08-30 11:15:15
and I don't even think it would require significant changes
surma
2023-08-30 11:15:24
Okay, so I guess maybe the changes to the CMake file were just a bit overreaching? Not every Emscripten build _must_ be multithreaded, right? (Cause that’s what that commit is kinda introducing)
2023-08-30 11:15:49
Then we’d only need your help figuring out why the single threaded build is failing
2023-08-30 11:16:22
(to be clear: I can’t quite remember if the _build_ is failing or if the single-threaded module didn’t load/error’d...)
veluca
2023-08-30 11:16:52
I suspect the build is failing
2023-08-30 11:19:44
anyway, I might be able to give this a look at some point, but it'd be nice if I had some more precise repro instructions 😄
Jake Archibald
2023-08-30 11:20:08
We're doing a build now, to give you the error message we're seeing
surma
2023-08-30 11:20:09
Yeah I am working on that 😅
2023-08-30 11:35:19
I am waiting for Docker to do it’s thing lol. In the meantime: The current cmake is problematic for the following reasons: We are compiling libjxl.a without pthread, but the CMake injects phtread. So the libjxl.a ends up beig multithreaded anyway. However, our CPP code that links against libjxl.a is not single threaded, and the JS glue code generated by Emscripten is also not expecting multithreaded code, and so embind fails to transfer values between wasm and JS (returning a typed array from CPP to JS just throws)
2023-08-30 11:53:40
<@179701849576833024> Okay I was wrong. Removing that section form the CMake file is the solution. It builds fine and is single threaded (and multithreaded builds still work if you pass -pthread). If it’s okay wiht you, <@606193820052488192> will open a PR so you can run your tests
veluca
2023-08-30 11:54:18
sure
2023-08-30 11:54:33
less work is always good 😛
Aryan Pingle
2023-09-04 12:37:51
Hey <@179701849576833024> thanks for your patience. I made 2 PR's in the Squoosh repo to explain the problem: 1. https://github.com/GoogleChromeLabs/squoosh/pull/1379 - Building and using libjxl v0.7.0 as is (throws an error in the JS glue code after the WASM finishes decoding successfully) 2. https://github.com/GoogleChromeLabs/squoosh/pull/1378 - libjxl after removing the CMakeLists.txt lines that <@228116142185512960> linked above (works perfectly) You can view the sites live at https://deploy-preview-1379--squoosh.netlify.app and https://deploy-preview-1378--squoosh.netlify.app respectively
veluca
2023-09-04 12:42:56
you had convinced me of the problem already 🙂 feel free to open a PR to libjxl with the cmake change?
Aryan Pingle
2023-09-04 12:43:22
sure thing!
2023-09-04 01:02:26
PR: https://github.com/libjxl/libjxl/pull/2771 I'll add a description and context in some time
veluca
2023-09-04 02:05:19
mhhh looks like that causes issues with the internal build: https://github.com/libjxl/libjxl/actions/runs/6073883619/job/16477794072?pr=2771
Aryan Pingle
2023-09-04 02:08:32
Hmm I ran into something *kinda* similar on my local build
2023-09-04 02:09:08
After changing the CMakeLists.txt file the build would fail, but on running the exact same build command it built correctly
2023-09-04 02:24:28
<@179701849576833024> It's correctly building https://github.com/libjxl/libjxl/commit/f95da131cf7c7ccd4da256356fde2fec1fa23bb5 v0.7.0 without that block of code. Do you think I should I try to find the commit/s where it's failing?
veluca
2023-09-04 02:25:20
if you can that'd be ideal
Aryan Pingle
2023-09-04 02:28:13
Yep I'll try, lot of commits to go through though RIP
2023-09-05 01:43:32
<@179701849576833024> I checked out my PR and it built and installed libjxl without errors. Here's the bash code I used to build it: ```bash cd libjxl mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF .. cmake --build . -- -j$(nproc) ``` And to install it: ```bash sudo cmake --install . ```
2023-09-05 01:43:47
(copied exactly from the README)
veluca
2023-09-05 01:44:08
was that a WASM build?
Aryan Pingle
2023-09-05 01:44:31
Nope I'm just building libjxl
veluca
2023-09-05 01:44:45
the CI failure I linked above is for building libjxl for the WASM target
Aryan Pingle
2023-09-05 01:45:00
Ohh I see
2023-09-05 01:45:11
Cool I'll try building the WASM
veluca
2023-09-05 01:46:03
see this workflow file: https://github.com/libjxl/libjxl/blob/main/.github/workflows/build_test.yml#L361
Aryan Pingle
2023-09-05 01:46:17
Yep was just about to ask that, thanks
2024-01-11 10:44:24
Update: https://github.com/libjxl/libjxl/pull/2771 Looks like we'll get an option that allows us to use single-threaded JXL