JPEG XL

JXL lossless encoder

Pierre-Anthony Lemieux

2024-03-02 02:05:21
2024-03-02 02:05:22	Thanks! Will try now.
2024-03-02 05:20:50	It does not look like it is that simple since `enc_fast_lossless.h` redefines `JxlChunkedFrameInputSource`, and so code including `enc_fast_lossless.h` cannot also include decoding headers.

veluca

2024-03-02 07:15:18	ah, that is extremely annoying
2024-03-02 07:16:11	it might work to split the code into two files, but that's probably something we should look into fixing
2024-03-02 07:16:51	(i.e. one file for encoding, and one for decoding)

Pierre-Anthony Lemieux

2024-03-02 06:36:39	How would I add an image header, which is expected by the decoder, if I am not in standalone mode?
2024-03-02 06:37:38	JXL lossless encoder

veluca

2024-03-02 11:41:30	in non-standalone mode you're not supposed to be using fjxl directly, so you kind of cannot
2024-03-02 11:43:07	but actually... why do you need to include `<jxl/encode.h>` at all?
2024-03-02 11:43:38	it should not be necessary with enc_fast_lossless.h standalone mode, and even more so for decoding
2024-03-03 12:05:32	ok, after further thought, I am not convinced you can use fjxl if you also link libjxl statically without running into ODR issues
2024-03-03 12:05:49	I might prepare a patch for `libench` to use the libjxl encoder api tomorrow
2024-03-03 12:06:33	but then you can't pass the effort to fjxl anymore, so perhaps not... i'll think more about it

Traneptora

2024-03-03 12:17:47

what is the primary benefit of doing things this way instead of just calling libjxl with effort=1?

veluca

2024-03-03 03:11:04

https://github.com/sandflow/libench/pull/16 <@553324745240608773> that should do the trick (I did not manage to test it locally b/c I don't have kakadu, nor I figured out how to build ffmpeg, though)

Pierre-Anthony Lemieux

2024-03-03 04:22:52	ok
2024-03-03 04:23:02	thanks. will check shortly.
2024-03-03 04:24:18	Two reasons: (a) that is used to be the way to do it and (b) I wanted to make sure to get the best single-threaded lossless encode performance 🙂
2024-03-03 07:06:37	<@179701849576833024> Thanks for the pull request. Did you try running `./libench jxl_e2 sample.png` on your side?

veluca

2024-03-03 07:10:03

nope, I didn't manage to compile libench at all (I don't have kakadu and I got some errors about libavcodec.a)

Pierre-Anthony Lemieux

2024-03-03 07:19:09	ah. the encoder code crashes 😦
2024-03-03 07:45:14	maybe we can try to comment out the ffmpeg and kdu stuff to allow you to compile

veluca

2024-03-03 07:45:43

let's see

Pierre-Anthony Lemieux

2024-03-03 07:50:27

seems to be stuck in an infinite loop

veluca

2024-03-03 07:52:18	yep, I clearly wrote some bad code
2024-03-03 07:52:20	fixing it...
2024-03-03 07:52:33	I guess I should not have expected to write it right first try 🙂

Pierre-Anthony Lemieux

2024-03-03 07:53:16	regardless of the outcome, hopefully it will make it easier for the future generation to use JXL in lossless mode 🙂
2024-03-03 07:53:36	the one_shot sample is good, but not for lossless
2024-03-03 07:54:04	.... and the API changed since last time I used it to write a lossless encoder

veluca

2024-03-03 07:54:07

I also clearly got stuff wrong in the cmake file

Pierre-Anthony Lemieux

2024-03-03 07:54:40

```set(JPEGXL_ENABLE_AVX512 TRUE CACHE INTERNAL "" FORCE) include_directories(ext/libjxl/lib/include "${PROJECT_BINARY_DIR}/ext/libjxl/lib/include") add_subdirectory(ext/libjxl EXCLUDE_FROM_ALL)```

veluca

2024-03-03 07:54:41

ah, fast-lossless was never meant to have a public API, now that it's integrated in the main API it should be much better

Pierre-Anthony Lemieux

2024-03-03 07:55:04

maybe one-shot can be modified based on the outcome of this thread

veluca

2024-03-03 07:55:46	I did start from that one, fwiw
2024-03-03 07:55:55	but yes, having an example for fast-lossless could be useful
2024-03-03 08:08:43	<@553324745240608773> should encoding be parallelized too?
2024-03-03 08:11:38	I see libjxl decoding is parallel

Pierre-Anthony Lemieux

2024-03-03 08:45:25	`export OMP_NUM_THREADS=1` is set when running the benchmarks
2024-03-03 08:45:44	... as I recall, this was the easiest way to get single threaded results

veluca

2024-03-03 08:47:36	jxl shouldn't use OMP at all
2024-03-03 08:48:13	but never calling `JxlEncoderSetParallelRunner` should also give single-threaded results
2024-03-03 08:49:44	I updated the PR with a commit that works locally

Pierre-Anthony Lemieux

2024-03-03 09:20:59	pulliing now
2024-03-03 09:22:57	`libench jxl_e1` works!
2024-03-03 09:25:11	how to ensure single threaded encode and decode?

veluca

2024-03-03 09:30:07	I cna modify the PR for that
2024-03-03 09:34:23	did that, and the jxl_e1 -> jxl renaming
2024-03-03 09:34:42	(btw, do you know about https://github.com/veluca93/fpnge/tree/main ?)

Pierre-Anthony Lemieux

2024-03-03 09:48:10	Died on this image 👀
2024-03-03 09:48:38	We should add it to the benchmark suite!

veluca

2024-03-03 09:49:37	I'm going to bet I got something wrong with alpha... one sec
2024-03-03 09:51:59	should be easy 🙂
2024-03-03 09:52:16	it has pretty much the same API as the old jxl fast-lossless

Pierre-Anthony Lemieux

2024-03-03 09:53:09

I wrote the benchmark suite specifically anticipating the day "we" have the discussion about moving to a different codec for PNG

veluca

2024-03-03 09:59:16

ok, fixed it - and hopefully all the images with alpha

Pierre-Anthony Lemieux

2024-03-03 10:15:20

did you try the image above?

veluca

2024-03-03 10:29:12

yup

Pierre-Anthony Lemieux

2024-03-03 10:30:34

weird

veluca

2024-03-03 10:30:59	does it not work still?
2024-03-03 10:31:04	perhaps I didn't push it right

Pierre-Anthony Lemieux

2024-03-03 10:31:26

```./build/libench jxl /mnt/c/Users/pal/scratch/libench/benchmarks/qoi_benchmark_suite/images/icon_512/actions-address-book-new.png terminate called after throwing an instance of 'std::runtime_error' what(): JxlEncoderAddImageFrame failed Aborted```

veluca

2024-03-03 10:31:52

indeed not... now I actually pushed the fix

Pierre-Anthony Lemieux

2024-03-03 10:33:53

all good

veluca

2024-03-03 10:35:11

perfect

Pierre-Anthony Lemieux

2024-03-03 11:40:00	```./build/libench jxl /mnt/c/Users/pal/scratch/libench/benchmarks/qoi_benchmark_suite/images/textures_pk/pkw_wall02bf.png terminate called after throwing an instance of 'std::runtime_error' what(): Image does not match Aborted```
2024-03-03 11:41:00

veluca

2024-03-03 11:54:38

huh, that seems to be an actual libjxl bug - would you mind filing an issue? I'll give it a look during the week

Pierre-Anthony Lemieux

2024-03-04 04:25:14	Done
2024-03-04 05:00:12	https://github.com/libjxl/libjxl/issues/3357

veluca

2024-03-04 06:43:35	thanks!
2024-03-04 10:08:28	does the libjxl v0.10 code work now? 🙂

Pierre-Anthony Lemieux

2024-03-04 10:17:21	yes
2024-03-04 10:17:39	decoding is slower than it used to be
2024-03-04 10:17:44	maybe single threaded now?
2024-03-04 10:17:51	encoding is faster
2024-03-04 10:18:31	slightly faster than J2K but also sightly bigger

veluca

2024-03-04 10:18:32	yup, I made it 1t
2024-03-04 10:18:33	https://github.com/sandflow/libench/pull/16/files#diff-ca33b64c2f159487e595d4fd31578094e4d331fb5ff518f2799f9a0d69f946a3L103

Pierre-Anthony Lemieux

2024-03-04 10:18:48

veluca

2024-03-04 10:18:50	do you have a number in MP/s?
2024-03-04 10:19:04	for sufficiently large images you should see something like 300MP/s

Pierre-Anthony Lemieux

2024-03-04 10:21:53

300 10^6 RGB 8-bit pixels per second encode?

veluca

2024-03-04 10:22:09	yep
2024-03-04 10:23:30	well, depends on the CPU

Pierre-Anthony Lemieux

2024-03-04 10:23:43

377 MP/s

veluca

2024-03-04 10:24:06

sounds about right then

Pierre-Anthony Lemieux

2024-03-04 10:24:24

decode is 64 MP/s

veluca

2024-03-04 10:24:33

that also sounds about right

Pierre-Anthony Lemieux

2024-03-04 10:25:09	ok. thanks.
2024-03-04 10:27:40	thanks for your help bumping up JXL

veluca

2024-03-04 10:32:39

you're welcome! thanks for all the benchmarks 😉

Pierre-Anthony Lemieux

2024-03-05 05:14:38

Do not hesitate to create a PR with your proposed PNG codec

veluca

2024-03-05 09:05:15	might do that at some point, when I have time 😛 would that be as a separate codec or replace the lodepng encoder?
2024-03-05 01:41:29	turns out there's still some bugs... will update the PR when we iron them out

Traneptora

2024-03-05 05:35:20	were you the one who was doing htj2k or was that thardin
2024-03-05 05:36:02	I don't quite remember

Pierre-Anthony Lemieux

2024-03-06 06:20:32

<@853026420792360980> me?

Traneptora

2024-03-06 07:47:09

ah okay

Pierre-Anthony Lemieux

2024-03-06 06:01:20

<@853026420792360980> I meant were you asking if I was involved in HTJ2K in FFMPEG? 🙂

Traneptora

2024-03-06 10:43:06

ye, I remember someone was working on that but I don't remember who it was

Pierre-Anthony Lemieux

2024-03-07 04:19:28

I organized the GSoC project. Etemesi, helped by Aous Naman and Osamu Watanabe, did the hard work.

Info

JPEG XL

General chat

Voice Channels

Archived

JXL lossless encoder