JPEG XL

Info

rules 57
github 35276
reddit 647

JPEG XL

tools 4225
website 1655
adoption 20712
nomacs 7
UI 11
chrome 34
image-compression-forum 0

General chat

welcome 3810
introduce-yourself 291
color 1414
photography 3435
other-codecs 23765
on-topic 24923
off-topic 22701

Voice Channels

General 2147

Archived

bot-spam 4380

streaming mode

Lilli
 veluca I barely remember my name these days 😉
2025-08-12 07:52:18
2025-08-12 07:52:19
Hey Veluca, I'm really struggling to implement the streaming in my implementation. Would you have pointers for me ?
2025-08-12 07:52:35
streaming mode
veluca
2025-08-12 07:53:27
Really not the best timing to ask me 🙂 try again tomorrow maybe?
Lilli
2025-08-12 08:21:34
That works ! Thank you
2025-08-14 07:05:58
Hey, yesterday was chaotic for me, would you have time today?
veluca
2025-08-14 07:15:35
maybe
Lilli
2025-08-14 07:32:49
I'll try to make it as concise as possible: I noticed that when using --streaming_input AND --streaming_output, the memory usage dropped significantly. When using only --streaming_output, the memory used remained very high. It seems that the input is being fully copied into memory, which means the program consumes at least twice the input file size, before anything JXL related is allocated. (I'm talking about the cjxl implementation right now) Now my issue is, I need to replicate in our codebase the behavior of --streaming_input and --streaming_output for low memory use. I tried using the chunking API but I couldn't really make it work with `JxlEncoderFlushInput` as it ballooned the memory usage to 3GiB. I currently sit at about 832MiB for an input of size 286MiB. When using cjxl directly, I get 300MiB (should add 286MiB since I'll keep an internal copy) in total, which is in line with what I need. How should I properly use the chunking api to obtain these results? My input is float32 [0,1] RGB image data (linearSRGB). I use JXL_ENC_FRAME_SETTING_BUFFERING=2 My images are of the order of 11000x4000 (50MP) I only need to write the RGB buffer directly to a jxl file
veluca
2025-08-14 08:09:51
I think you want JxlEncoderSetOutputProcessor and JxlEncoderAddChunkedFrame to replicate what streaming_input and streaming_output do
Lilli
2025-08-14 08:34:03
So, you mean the callbacks that are placed in the structures?
veluca
2025-08-14 08:44:52
those are API functions
2025-08-14 08:45:01
that take callbacks as arguments
Lilli
2025-08-14 08:59:56
Yes I've tried them, I didn't succeed, at the time, I did not know of the flags --streaming_input and --streaming_output so I just implemented what seemed logical to me.. It did work though, the output jxl is fine, it just takes a lot of memory to make it :). I'll try to find which callbacks connect to streaming_input