JPEG XL

What is missing in libjxl to make it work directly without external tiling? We do have chunked encode and pixel callback for decoding, so both can be done in a streaming / limited memory way...

I've been looking at the chunked mode, but I think it won't work for streaming write

there are no guarantees about what order data_at() will fetch pixels, if pixels will be fetched more than once, etc., so it needs to be backed by the whole source image in memory

it would need to (eg.) only ask for pixels tiop to bottom in small regular chunks, and (maybe?) never ask for the same pixel twice

It doesn't, but maybe we need to make the api contract tighter

<@179701849576833024> any thoughts?

hydrium currently can handle massive images without storing the whole image in memory

My experience has been that decoding hydrium encodes is very slow

and ROI decoding is nearly impossible

this is without `--one-frame`

we should talk about it -- a single jxl file is a much better solution for many reasons, and ideally we'd get the API in a state where it can be used for this

But the upside is it works well at conserving memory

my 2p would be for JXL to add a tiled mode, fwiw write the tiles as a set of frames with a little extra metadata to say how they should be ordered, plus an index at the end

a jxl file is already tiled

and already has an index

that's what hydrium does, but frame tiling can't have an index

ok, then guarantees about chunked write data_at() ordering should be possible, I'd think

<@310374889540550660> Are you looking for an API like the one at https://github.com/Traneptora/hydrium/blob/a278f0ae4b4259181a3c34469682aea733024e8d/src/include/libhydrium/libhydrium.h#L273

something like adding a flag that guarantees it only requests access in scanline order for 2k x 2k tiles?

If so, that seems reasonable to add to libjxl

as an api

actually, it seems to do that now ok to copy-paste a shortish log?

sure

I am also pretty sure it will happen in practice

but we might want not to *guarantee* it unless you ask explicitly

for example, if you do "middle out" mode for progression you probably want non-scanline

``` john@banana ~/pics $ vips copy wtc.jpg x.jxl[chunked] vips_foreign_save_jxl_pixel_format: vips_foreign_save_jxl_pixel_format: vips_foreign_save_jxl_data_at: left = 0, top = 0, width = 2056, height = 2056 vips_foreign_save_jxl_pixel_format: vips_foreign_save_jxl_data_at: left = 2040, top = 0, width = 2064, height = 2056 vips_foreign_save_jxl_pixel_format: vips_foreign_save_jxl_data_at: left = 4088, top = 0, width = 2064, height = 2056 vips_foreign_save_jxl_pixel_format: vips_foreign_save_jxl_data_at: left = 6136, top = 0, width = 2064, height = 2056 vips_foreign_save_jxl_pixel_format: vips_foreign_save_jxl_data_at: left = 8184, top = 0, width = 1188, height = 2056 vips_foreign_save_jxl_pixel_format: vips_foreign_save_jxl_data_at: left = 0, top = 2040, width = 2056, height = 2064 vips_foreign_save_jxl_pixel_format: vips_foreign_save_jxl_data_at: left = 2040, top = 2040, width = 2064, height = 2064 vips_foreign_save_jxl_pixel_format: ... ```

I added some prints so you can see the rects it's asking for, and they seem to be top to bottom, phew!

with libjxl v0.11

yes, maybe just add a guarantee about this to the docs?

yep, perhaps conditional on setting some flag

<@310374889540550660> can you open an issue?

sure, I'll do it now

I'll upload a sample image too! hehe

What happens if the image is permuted

permuted toc

this is on the encoder side no?

if you ask for a permuted toc, you don't get asked input in scanline order 😛

I made an issue https://github.com/libjxl/libjxl/issues/3853

will there ever be parallel calls to data_at()? I guess not, though the docs don't say (I think)

there might be

well, you need to enable threads for it

I'd imagine that pixel callbacks can happen in parallel if threads are enabled

yea, true ig

you just have to pass data in scanline order

could that break ordering? eg. a thread asks for tile 0x0 and stalls, other threads ask for later tiles and continue down the image, the tiler 0x0 thread finally finishes, then the next tile is out of order

I believe (but I'd need to check) that threads will ask in scanline order

well, but they may finish out of scanline order

what happens in that scenario?

libvips has this problem 😦

it has a complicated mechanism to keep requests in scanline order even if individual threads stall

you could always have libjxl consider this as a possibility and use permuted TOC to fix it

jpeg xl has the lucky feature where you can calculate the tile permutation

so if the tiles end up out of order you just change how you send the TOC

that's nice

ok, i'll ignore the possibility of stalls breaking tile ordering

what do you care about exactly?

it sounds like the issue is that the API doesn't guarantee that it requests tiles in scanline order

it may do this anyway, but if it's not a guarantee, tufty has extra buffering concerns to worry about

yes but with multiple threads what would you *want* to have?

if the API promises that it won't request tiles outside of scanline order, then buffering is much easier on the clientside

does the order of requesting or of releasing matter?

I believe with multiple threads libjxl still *asks* in scanline order (unless you give a weird parallel runner)

but the time at which they get released might be somewhat shuffled

but that's handled gracefully by libjxl, is it not?

i.e. even if the threads release in a weird order, the jxl bitstream won't be shuffled

I think it'll work something like this: - I start a background write thread, that thread calls JxlEncoderAddChunkedFrame() - it calls data_at() and asks for scanline 0, I stall it while I prepare the pixels - I generate the first 2100 scanlines (2048 plus a safety margin) - I let data_at() return for the first set of tiles and wait - as soon as data_at() asks for scanline 2040, I stall it again - I move my scanline window down, copy over any excess, and generate the next few thousand lines - I let data_at() continue again and wait

I don't see `release_buffer` in here

so I need to know that as soon as data_at() asks for scanline 2040, no more requests for anything in scanline 0 will come in

yes, I'd need to do something about release_buffer too

needing a couple of semaphore and some threads is a bit painful, it'd be simpler with a push API, of course

it turns out the practical limit is 1000

I see that there is some kind of 0.00001% ..... It's worth trying for 10 hours<:Stonks:806137886726553651> However, thanks for the graphics

above 100 doesn't actually get slower because this only affects part of the algorithm which normally samples at 10%

related to the observed plateau in density, there is not linearly more calculation happening from 1-1000

and while I 100 is probably rarely best, it's at least on the good side of tendency for photo-like content. I 1000 is even more dubious, but it is possible that values over 100 would perform better than any lower value.

I remember seeing extreme differences when doing multiples of 5 and 10 compared to incrementing by 1

``` // TODO(firsching): add a function JxlDecoderSetDefaultCms() for setting a // default in case libjxl is build with a CMS. ``` <@795684063032901642> this should probably be done before v1.0, it should be easy enough, right? Or maybe we don't even need that function but just have that as the default unless/until the application sets a different cms?

I think it would also make sense to have some sensible default output color space when an application does not call `JxlDecoderSetOutputColorProfile` but a (default) cms is available, to make it less likely for viewers/applications to mess up badly. For example: - if the image space is CMYK: set the default output space to Display P3 - if the image space is HDR (PQ/HLG): if uint8 buffers are used and no desired intensity target is set, set the default output space to Display P3 with some reasonable intensity target (255?), otherwise set the default output space to the signaled space - if the image space is anything wider gamut than sRGB (Adobe98, ProPhoto etc): set the default output space to Display P3 - otherwise set the default output space to sRGB. The above should make things look OK for "dumb" viewers that assume everything is 8-bit RGB.

WDYT <@604964375924834314> <@795684063032901642> ?

right, let me revive https://github.com/libjxl/libjxl/pull/3062

Sure, sounds good to me

Could you JIT compile the MA trees to improve modular decoding speed?

although that might end up with more possible security bugs

https://discord.com/channels/794206087879852103/1065165415598272582/1238399364561506315

Maybe weighted could be improved with a Lookup table?

What exactly is weighted again?

self-correcting predictor

What’s the formula?

... It's complicated

it's the most complex beast in the entire Modular decoding process imo

Alexander Rhatushnyak invented the self-correcting predictor. Luca and I then made it even more complicated to make it more expressive and faster.

Oh geez, no wonder why it’s not particularly performant

Only thing is for image 3, sometimes the integer division can be faster than a lookup table due to cache issues

It performs well in terms of compression for photographic images. But speed-wise, you will see a difference between decoding e2 (which uses only the simple ClampedGradient predictor) and e3 (which uses the self-correcting predictor) images.

I’m guessing it’s one of the first things cut for faster_decoding?

A lookup table with 64 entries should be faster than integer division on pretty much any platform. Integer division is one of the worst operations out there...

Huh, missed the fact it’s only 64 entries

I misread the numerator to be (1<<x) where x <= 24

Do libjxl simd use optimisation?

Doesn’t Modular effectively forbid usage of simd?

libjxl uses hwy, a google C++ simd library.

it vectorizes the perf-critical stuff.

modular does not prevent vectorization.

to my knowledge anyways...

Add this?

https://youtu.be/J0fF4thhhe4?si=Qw847xAGIp0bo73f

I've got libvips using the new chunked save API, phew with a 30k x 30k test image I see: ``` $ /usr/bin/time -f %M:%e vips copy st-francis.jpg x.jxl 5719092:586.74 $ /usr/bin/time -f %M:%e vips copy st-francis.jpg x.jxl[chunked] 854360:573.48 ``` so very slightly faster (libvips will overlap JPG load with JXL save), but almost 7x lower memory use woo!

libjxl v0.11 never seems to run `data_at()` in parallel, which is interesting, I was expecting to see several run at once though I don't think it would help speed, in this case anyway

I guess you can keep 64 threads busy with one 2048x2048 chunk, so not much need to work on more than one of those at the same time, unless you have lots of cores...

For those interested, the `st-francis.jpg` test image can be found here: <https://commons.wikimedia.org/wiki/File:Giovanni_Bellini_-_Saint_Francis_in_the_Desert_-_Google_Art_Project.jpg>.

I tried making data_at() very slow: ``` vips gaussblur st-francis.jpg x.jxl[chunked] 100 --vips-progress ``` ie. a sigma 100 gaussian blur now data_at() is taking many seconds, but libjxl v0.11 doesn't seem to run more than one data_at() at once so utilisation drops to 100% (single threaded) for large parts of the save

I assume libjxl expects data_at() to return pretty much instantly, but I guess in your example you want it to work on multiple chunks at the same time?

fair point

well, I'd expect to do internal parallelization for that since it's running on a 4mp chunk, no?

yes, libvips could start up a threadpool to generate these pixels, though there would be some overhead (of course)

from the API docs I wondered if libjxl would start to run data_at() in parallel if it saw it was idle a lot of the time

nah, not happening

will data_at() always be single threaded? I could remove a few locks if that's guaranteed

also not guaranteed

hehe ah well

I can imagine us wanting to run it in parallel if you ever run this on a 7990x or some other insane cpu like that

ok, I'll subdivide libjxl tiles into eg 128x128 tiles and generate them in parallel in data_at()

thanks for the advice!

I notice cjxl has really poor multicore CPU utilization.

Wall time performance does not improve linearly with extra threads sharing the work, even for a huge image at effort=3

Most of the cores are idle

`30000 x 26319, 19.929 MP/s [19.93, 19.93], , 1 reps, 1 threads.` `30000 x 26319, 26.455 MP/s [26.45, 26.45], , 1 reps, 2 threads.` `30000 x 26319, 31.931 MP/s [31.93, 31.93], , 1 reps, 4 threads.` `30000 x 26319, 33.858 MP/s [33.86, 33.86], , 1 reps, 8 threads.` `30000 x 26319, 36.542 MP/s [36.54, 36.54], , 1 reps, 16 threads.`

That's pretty abysmal scaling.

not even a 2x improvement going from 1 to 16 cores.

Also, there's a superfluous comma, and I am not sure what the numbers in the brackets are supposed to represent.

That's weird behavior, what platform is this?

What's the input format?

`JPEG XL encoder v0.11.0 4df1e9e [AVX2,SSE2]` Windows 64

The input format is JPEG, with `-j 0`

It's the Francis painting

But I have noticed bad multi-CPU usage in cjxl for a long long time. And very bad sublinear scaling too

This just seems like a really good demonstration

Can you open an issue for this? I get very different behavior (much better scaling) so there might be something wrong with the platform-specific stuff

it's very difficult to benchmark stuff like that

you need to be careful around eg turbo boost, core affinity, etc

so for example, is the 1 thread case running with a 30% clock boost because only one core is being used?

is the two core case really running across two completely separate cores, or perhaps two cores that share some resources?

effort=7 has better scaling. Maybe that's why you noticed better scaling? `30000 x 26319, 1.586 MP/s [1.59, 1.59], , 1 reps, 2 threads.` `30000 x 26319, 7.676 MP/s [7.68, 7.68], , 1 reps, 16 threads.`

are threads moving between cores during execution etc etc

Well, that almost always happens due to OS CPU scheduling

sure, but it can influence benchmark results disabling turbo-boost is the main thing that's bitten me on stuff like that

and disabling hyper threading of course

But... a program that fully utilizes the CPU will have 100% cpu usage and you can measure the CPU time and see if the measured CPU time increases with the number of threads. If the CPU time is not increasing then it's not using the extra CPUs

If the CPUs are not being used then you know there's a problem that isn't caused by the OS or hyper threading

Because the OS and the CPU firmware will always try to give the program as much CPU as it asks for

ah that's true

I mean, it could still be caused by the platform specific thread library

either a bug there or more likely a bug in the application

not a serious bug, just a lot of performance left on the table :)

what's the exact cjxl command you're running <@1028567873007927297> ?

`cjxl -j 0 -d 1 -e 7`

and `cjxl -j 0 -d 1 -e 3`

thanks! I'll experiment

sometimes with `--num_threads=2` etc

if you omit `--num_threads` it detects the number of available logic cores and uses that number.

vips only requires just over a gig of memory to resize that image by the way.

~~a lot less memory than cjxl even when using jxlsave :)~~ actually, never mind that, cjxl and vips both use just over 5gb to convert it without resizing.

I'm a bit late, but that's min and max MP/s for when --num_reps is used

Ah. And the empty comma field?

Maybe the output could be formatted a little better. Also maybe cjxl could use an option parsing library.

like if not getopt then something similar to getopt

I *think* it's meant to be the variablility between min and max? `value_tendency, unit, value_min, value_max, variability);`

<https://github.com/libjxl/libjxl/blob/7e178a51b0a59fcc8e58c78ce88058a9b97abf5b/tools/speed_stats.cc#L90C12-L90C69>

```cc std::string mbs_stats = SummaryStat(file_size_ * 1e-6, "MB", s); ```

It's supposed to be this

The empty comma is supposed to be mb/s but for some reason `mbs_stats.c_str()` is an empty string.

Yes, the legibility is awful, I might try and reformat it tomorrow, but here's some data for me across all efforts. All Lossless``` Seconds 1 Thread 2 Thread Effort Real Time 0.68 0.52 1 CPU Time 0.66 0.70 1.66 1.05 2 1.48 1.38 2.92 1.74 3 2.61 2.84 8.10 4.39 4 7.86 8.12 13.40 7.14 5 13.23 13.50 18.23 9.65 6 17.86 18.30 26.81 13.91 7 26.47 26.84 66.98 34.78 8 66.16 67.84 110.59 58.82 9 109.75 115.12 532.16 495.80 10 520.50 500.95 ```

I can add more threads later on too, probably

That can't be right, CPU time is less with 2 threads?

Did you divide the CPU time by the number of CPUs?

Hmm, no, I think I see...

I assume you mean effort 10

The total CPU time should be the same or more, but not less, with additional threads...

Why would more CPUs = less total work?

e10 is almost entirely singlethreaded, so *something* in there must be getting skipped when multithreaded

The amount of work should be unchanged regardless of thread count. Or there should be some additional work overhead with additional threads.

I didn't expect the total amount of work to decrease

Oh actually, it's effort 10 and effort 2... Strange

But that's only the case with e=10, and e=2

Yeah, kinda weird

ah well :)

That's good scaling

I guess e=1 is single threaded too

Now do an 8 thread test... 😂

Tried another image, e2 has the expected 2x CPU time increase going to 2 threads, e10 is still lower by 0.06 seconds while still being faster

Now e2 has the same CPU time as singlethreaded, but e10 is *slightly* more CPU time

image dimensions?

is it a big image?

The last few results were 910 x 461, the table above was 3495 x 2458

bigger is better :)

Did you average over many runs? Timings can fluctuate quite a bit...

libvips chunked mode support is in this draft PR https://github.com/libvips/libvips/pull/4174/

it brings peak memuse for copy jpg->jxl on that big image down from 5.5gb to 800mb (and it's sligfhtly faster too)

nice

soon the vips will rule the galaxy

I'm hoping to get a new libvips done before xmas with chunked mode libjxl support, and JXL HDR too

Does this only happen with v0.11.0 for converting JPEG to lossy JXL? > Must not set non-zero distance in combination with --lossless_jpeg=1, which is set by default.

I didn't mention --lossless_jpeg=1 however

It's required to add --lossless_jpeg=0 as a workaround now

I think it should be more difficult for people to accidentally re-encode a JPEG from pixels

It's an error popped up if I attempt to do it without --lossless_jpeg=0

That's good

That shouldn't happen when -d isn't 0

It should... or it should trim the file without decoding to pixels if possible. It's good if it's less likely someone accidentally re-encodes a JPEG from pixels since that's usually dumb and unwanted behavior

Forcing a -j 0 to confirm before doing something stupid, is a good thing

Maybe it's better to notify users to add -j 0 if they really want to convert to lossy JXL

A "you’re about to do something stupid" warning. "Are you sure?"

Ideally there should never be a reason to decode to pixels and libjxl should be able to trim the file without decoding it

Imagine being able to use lossy compression but never having to worry that the output will be larger file size than the jpeg source

We could silently ignore non zero distance and just have j 0 as an expert option

I don't know why converting a higher quality JPEG to a lower quality JXL is a "stupid" decision

silently not doing what the user asked is bad behavior. Better to tell the user "you're dumb"

Maybe that's not for your usages, it's quite necessary to handle JPEG that's unreasonably at quality 100 for sharing or storing

Because the way it work right now, libjxl inflates the file and then recodes it less efficiently rather than re-using and trimming the DCT data.

That's definitely the least optimal way of doing it

The resulted file sizes matter more

I can convert one JPEG to -d 0 and -d [I want] and then compare which is smaller

A lot of JPEGs on web pages are huge and inefficient, as they went directly from camera to webpage, or were exported with 100 quality setting.

My own procedure is converting a bunch of JPEG to both -d 0 and -d 0.5. Larger -d 0.5 ones are removed as their original JPEG should be at a significantly lower quality

As long as you're encoding from a high quality source it should be fine.

But there absolutely should be a stupid-alert when doing -j 0 since it's kinda a footgun

unless the source JPEG is q100 or close to it, then libjxl is not going to be very efficient

In Cloudinary, if you do q_auto on an existing JPEG (without any modifications like downscaling), it looks at the quantization tables, estimates JPEG quality from those, and does one of these three things: - JPEG was low quality already (e.g. q60): don't touch it, just do lossless jpegtran optimization - JPEG is high quality: requantize it a bit in the DCT domain, e.g. bring a q93 quant table down to q85 or so, only touching coeffs where the factors get higher (different encoders use differently shaped quant matrices) - JPEG is very high quality (q96+): just decode it to pixels and do the usual q_auto encoding

ideally, libjxl would be able to lossy requantize JPEG without ballooning it first.

In principle we could do something similar in libjxl — though currently the encode API is not really designed for that...

Is the lossless jpeg transcoding exposed in libjxl or is it only a cjxl thing?

cjxl cannot do anything that libjxl cannot do. There are two ways in libjxl to encode a frame: one is by providing pixel data, the other is by providing a jpeg bitstream and telling it to losslessly transcode it. I suppose we could add an option or an additional API for providing a jpeg bitstream that doesn't necessarily apply lossless transcoding, but compares the estimated quality of the jpeg to a target distance and then either does lossless transcoding or re-encodes from pixels (or potentially does a lossy transcoding with requantization in the DCT domain, though I'm not sure if we want to go there)

If the JPEG is sufficiently high quality does it make sense to go to pixels? If so what’s the approx cutoff?

Does Cloudinary not do JPEG to jxl ?

Also what estimation algorithm are you using? Is it like the one in Exiftool?

I guess the cutoff depends on the ratio between input JPEG quality and desired target distance. If you want d1, probably you need libjpeg-turbo quality 95+, while if you want d3, probably libjpeg-turbo q85+ suffices.

I was talking about what we do when both input and output format are JPEG.

Does quality = quantisation weights ?

what I'm using as a proxy is just some weighted sum of the quantization factors, but there are probably better ways to do it. In particular, you can have encoders like guetzli and jpegli that use low quantization factors with variable deadzone quantization, and then just looking at the quantization tables kind of overestimates the actual quality...

Wait since when did JPEG have adaptive quantisation ?

it doesn't, but an encoder can choose to zero out coefficients even if they wouldn't actually quantize to zero. Doing that in a smart way (selectively, only when it doesn't hurt visually) is basically the main 'trick' jpegli uses to achieve better compression

Does 0 entropy better?

yes, the goal of most coding tools (prediction, frequency transforms, RCTs etc) is basically to get more zeroes (or lower-amplitude numbers) since those will be cheap after entropy coding. In particular for DCT coefficients, both JPEG and JXL additionally have cheap ways to signal "all the other coefficients in this block are zeroes"

That would be awesome

After a certain point the quantization noise is so small it won't really matter when it goes through vardct. But only at really high settings where there is very little noise like q97+

Otherwise the encoder tries to preserve that noise

is there an example for how to decode a region of interest/cropped decoding?

It's on the roadmap for 1.0 but not currently implemented in libjxl. JXL-Oxide has experimental cropped decoding if you'd like to try that

thanks I'll look into it

hello all, libvips dev here, I'm trying to improve JXL HDR support in libvips it's mostly working (I think?) https://github.com/libvips/libvips/pull/4174#issuecomment-2395009764

1. I think I need a HDR JXL image with very high dynamic range for testing, something that will decode to luminances well outside SDR does anyone have a good test image like that?

2. I think I have some kind of D50/D65 mixup, my images are rather yellowish 😦

AdobeRGB png -> lossy jxl -> png produces png with profile named "RGB_D65_0.639997;0.329997;0.210005;0.710005;0.149998;0.060004_Per_g0.454707". Is it expected?

https://sneyers.info/hdrtest/

Yes. There's no enum or shortcut for adobe rgb (also known as opRGB) so it always spells out the gamma and primary coordinates

Thank you. Not a problem, just looks weird in ps

Yeah. It would be nice if there was a shortcut/shorthand for opRGB, at least in libjxl's color_description.cc

0.454707 is the INVERSE of the gamma though

Is that expected?

??

jxl saves inverse gamma internally

up to 7 digits below decimal point iirc

Ok... and sRGB gamma is a special case because of the linear segment I guess?

yeah it's different from pure gamma

opRGB primaries are 0.64,0.33;0.21,0.71;0.15,0.06, d65 illuminant, gamma = 563/256 (or the inverse 0.454707 I guess) so it would be cool if there was a shortcut for requesting these settings since it's used relatively often in photos and printing

It doesn't need to be specially added to the on disk format but just a shortcut in the library would be mildly cool and neat

Maybe this is a really dumb question, but why does djxl produce such a dim and dull looking image? Is this a bug? `djxl --color_space=sRGB --bits_per_sample=8 ./jpegxl-home.jxl ./jpegxl-home-srgb.png` `djxl --color_space=DisplayP3 --display_nits=300 --bits_per_sample=16 .\jpegxl-home.jxl .\jpegxl-home-p3.png`

I don't understand what the point of display_nits and intensity_target are

the intensity_target in jxlinfo says 10000 nits, but if that's the case then shouldn't the decoded image not be so ridiculously dark?

<@604964375924834314> this does look like a bug, right?

Waterfox also shows the jxl as an extremely dark image just like here. So does jxl-oxide as used by jxl-winthumb. To me it seems like maybe intensity_target and display_nits affect the decoded image in a counter-intuitive way, and maybe the default value of the latter is equal to the former, and I'm not sure what they are or how they are even supposed to be useful.

Maybe I'm just misunderstanding the problem of converting an HDR image to SDR

I think you got the images backwards... The bright one has a P3 ICC, the dark one is sRGB

Yeah, when I decoded it to 8 bit sRGB it's really dark and dull

Which I don't think makes sense but maybe I'm not understanding something

That's standard for HDR to SDR without proper tone mapping. It's how I see all HDR JXLs

I guess, but ... why?

You would think it would be super bright and washed out with lots of clipping or something

Or just remapped to whatever range the display could show

Or just closer to the bright image

The default as far as I understand it is to scale everything down until the peak brightness matches what SDR can output

I don't see any fully bright pixels in the dark image though

That's because it's 16bit, cramming the values down to 8 means the entire range is used. But this is all speculation and observations on my behalf

Bit depth won't change the brightness

Mostly I don't understand if the nits/intensity knobs even make sense or serve a useful purpose.

can you share the jxl?

It's on the jpeg website 😂

ill find it eventually I guess

https://jpeg.org/jpegxl/

It looks great on my iPhone actually

[dorime](https://cdn.discordapp.com/emojis/979397787970072596.webp?size=48&quality=lossless&name=dorime)

>`https://jpeg.org/images/jpegxl-home.avif`

heresy

absolute heresy

It looks almost exactly like the p3.png I made

On my iPhone

I dunno, maybe I'm stupid, but... I fail to understand why the image looks so different when I just decode it without --display_nits

yeah using --display_nits=203 ont he srgb decode makes it look fine

this should be more or less expected behavior

I dunno, is it wrong of me to expect the image to have better tone mapping? Even HDR displays require tone mapping to view images, since no display exists that can faithfully represent the full possible range of intensities a camera can capture.

Is it wrong of me to expect an image to look not so completely and obviously different from what it's intended to look like?

you aren't doing real tonemapping rather you are creating "an HDR image with an SDR transfer"

I'm not sure what I'm doing because I have no idea what the nitknobs do

Or why they are even necessary

the display_nits is the tonemapping, when it's not set it's not doing "tonemapping", that being said, libjxl should likely implictly tonemap when doing PQ->sRGB

So either I'm dumb or they're dumb or both 😂

basically you are creating a 10k nit sRGB which is way out of spec

It looks like the default result is what would happen if I set display_nits=intensity_target=10000

note you get the exact same behavior with mpv `mpv --loop jpegxl-home.jxl --target-peak=10000`

the image is getting decoded with it's target nit value, which is... not really correct?

``` jxl-oxide jpegxl-home.jxl --target-colorspace srgb -o jpegxl-home.png ```

All images require some form of tone mapping to be displayed on any display

I wouldn't consider this a "bug" but it is for sure not desired behavior

Because displays can't represent any arbitrary brightness value

Considering most software still doesn't even have color management...

iirc jxl-oxide tonemaps implicitly to 80 nits when using sRGB right?

uh let me check

Sadly jxl-winthumb renders it super dark

80 is pretty low...

80 is per sRGB spec

it targets 255 nits

which yes, is very low

really? that looks a little blown out for that

oh no, it lines up with djxl

No, that's pretty much exactly how it looks on my iphone

well, it's not like tonemapping is an exact art sadly

I like 2446a with this image myself

The only difference between tirr's image and the original on safari is, the original is slightly deeper blue in the iris. And the edges of the iris are darker too

(jxl-oxide uses rec2408)

yeah, winthumb should be converting it to sRGB ideally

and iirc libjxl does some local tone mapping

In tirr's image the skin looks more saturated and the iris looks more washed out

But they are almost identical

one day, I will create a proper rust cms with a bunch of tonemapping stuff available [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&quality=lossless&name=av1_dogelol)

Much better than a super dark and obviously defective image... unless "obviously defective" is the look we're intending to go for but I don't think that's what people usually intend when they want to just view an image and expect it to look as close as possible across displays

oculante just displays

That's basically just as bad as the dark image

Instead of dark it's extremely dull

80 nits isnt even that dim

I guess not since brightness is not a linear thing

if you have a pixel 7, it's 66% brightness

on samsung it's about 25%

gonna compile oculante requesting an sRGB image

https://tenor.com/view/easy-button-that-was-easy-red-button-press-the-button-gif-17005111

BTW, Affinity Photo 2 open is extermely dark as well.

`intensity_target` only has to be an upper bound on the brightness found in the image, it’s allowed for it not to be the smallest possible upper bound

as it happens, it seems to not actually reach 10 000 nits

and most of the image is considerably below that

without `--display_nits`, `djxl` currently won’t perform any tone mapping and will therefore map 0-`intensity_target` to 0-1 in the output

whereas `--display_nits=N` will cause it to tone-map from 0-`intensity_target` to 0-`N`, and then map 0-`N` to 0-1 in the output

(iirc, there is actually even code to tone-map to a non-0 lower bound, but it’s not exposed in `djxl`)

I think when you set display_nits, it tries to approximate the appearance of a target_intensity screen on a display_nits screen

This seems wrong. Why does the image look normal on an HDR display even though most HDR displays don't get anywhere near 10000 nits? Even my iphone on low brightness shows a normal looking image.

Isn't there a better way to show a more consistent looking picture across different displays?

because HDR displays will receive a PQ signal and they know how to display that

including their own tone mapping, usually

So the tone mapping has to be implemented in software if the hardware doesn't support it?

yes (e.g. a MacBook Pro set to the reference “HDR video” mode, which leaves everything under 1000 cd/m² unchanged and just hard-clips at 1000, instead of the usual tone mapping)

Would it be more correct to encode a PQ image that only goes up to 4000 nits with intensity_target 4000 than to keep it at the default? So it will go all the way to 1 when tone mapping instead of keeping some range for intensities that don't actually occur?

Because then maybe we should change the default encode behavior from "10000 if the input is PQ" to "find actual max and signal that", no?

Since many PQ images will not go brighter than 4000 nits, or even 1000 nits...

Sounds more logical to me, assuming it doesn't break anything on HDR displays...

On HDR displays it shouldn't change anything iirc, it would just affect the default tone mapping.

What even is the point of target_intensity otherwise if it doesn't contain any entropy. Isn't it part of the on disk format?

does it actually *have* to be an upper bound? the png spec suggests using the approach from this paper: <https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9508136> which, for still images, just means taking the 99.99% percentile of the maximum light level ( = `max(R, G, B)`) value

but i agree that it would be good default behavior to calculate it automatically

shouldn't intensity target be handled similarly to HDR10's mastering display nits? EDIT: when working with PQ

can you elaborate on what you mean by that?

PQ has an "absolute nit scale" but that doesn't mean displays can always render it, for instance say you have a 1k nit display and a 2k nit picture, what does the display do? It tone maps it down, instead of clipping. so intensity_target=1000 can signal "Ok this image was made on a 1knit display, so signals that are above 1000nits, are most likely not actually above 1000 nits. EDIT: or rather it's more like, this is what stuff was tonemapped to, so take this into consideration

I can imagine sometime in the future someone takes a picture of the sun, saves it in jpeg xl, then sends it to someone to decode... and their computer explodes :)

as far as I can tell, there is no real single defined way on how to handle HDR10 metadata

I thought PQ is just a transfer function. Why can't a PQ image just be decoded to linear and re-encoded with a different transfer function?

I mean, you *can* do that, PQ is just a transfer function, but it's one that defines luminance absolutely

1.0 will always == 10000 nits with PQ

What's so special and complicated about it? And why does it feel like, whenever reading about HDR, they reinvent everything to be way more fragile and needlessly complicated instead of reusing and logically extending existing technology?

well reference nits

I see... so the problem is that other transfer curves don't absolutely define luminance

Or at least, a SMALL part of the problem is that

well kinda, but you will always have an issue of mapping intended luminance. this is the reason old sRGB images look bad on modern displays

Really it shouldn't even be a problem... and compared to the behavior of actual HDR display hardware, simply decoding a PQ image that way without any tone mapping seems incorrect and inconsistent compared to how an HDR screen would behave

inconsistent yes, incorrect no

well, it depends on what you want to do with it

If you're displaying it on a non HDR monitor or converting it to sRGB then it should mimic what an HDR screen would do and tone map it in a similar way

well we have a lot more sophisticated and better tonemappers

Well anything is better than the current super dark output 😂

mpv's spline isn't bad, bt.2446a, bt.2390, hable, marty, etc

oh yeah, libjxl should imo for sure do implicit tonemapping when converting to an SDR transfer

Well, yeah, because tonemapping is always implicitly happening when an HDR image is being displayed properly on an HDR display, it should be no different when re-encoding/remapping to an SDR transfer curve...

It would only be consistent

In my opinion...

is there any way to tell if a jxl image is hdr ?

checking intensity target is probably the easiest way

that or the intended transfer

A transfer curve based on absolute luminance?

consistent but not necessarily accurate, you can for sure convert between colorspaces without doing any tonemapping, and there may be cases for that for sure

but ofc, like I said, I do agree that it should be the default

But there's no such thing as accurate when it comes to displaying an HDR image on any screen, even HDR screen, that can't reproduce the brightest pixel

that;s not true, this is why we have reference nits

we working in reference which is the "theoretical best case" so we have a common ground, and everything revolves around them

are there actually any 10k nit displays?

or is that stupid

for example, the "common ground" for mastering right now is 1000 reference nits, but some folk are pushing towards 2k or even 3k I think it is now

I know of like 1 or 2 that have been showed of at CES before

with HDR is the point that the display always shows the pixels at the same nits

TCL and LG iirc

how does that interact with screen brightness controls?

not true

this is in fact almost never true

then how is things defined with absolute nits?

with absolute nits, you have display nits vs reference nits, by default displays will almost always tonemap, but you can stop this behavior with calibration

actually I think most HDR1000 displays track PQ, then tonemap when needed now

not like it matters because you get retarded BS like this anyways lmao

As far as I know, content is always adapted/tone-mapped to the screen it's displayed on.

"we can track PQ in hardware, but you will never be able to do this because we in firmware put a power savings mode on that you can't disable unless you get into manufacturer control mode"

oh dear

And so ideally images should look as consistent as possible regardless of the transfer curve encoding or the type of monitor

And there shouldn't be inconsistent rendering depending on things like that

Rtings actually do PQ EOTF tracking tests iirc, so you would want to check them before you buy a HDR display

https://www.rtings.com/tv/tests/picture-quality/pq-eotf

as long as you have good tracking within 1knit you are probably fine

NPC...

4knit tracking is great for people with a lot of cash to burn, and it's probably nice? I don't actually have experience with displays that high end, but I would assume it's nice

realistically for a budget display, you want good PQ tracking to 600, because past 600 nits you are mostly into the "highlights" sections unless you are watching terribly mastered content

or content that just really wants to push PQ

The brighter something is the less precisely your eyes care how bright it is

but yeah, a good display will track whatever nit range it reliably can, then tonemap the rest. so if you have a "1000 nit mastering display" (HDR10 metada) you can assume that you roughly have 1000 nits of properly graded content, then content above that has likely been tonemapped on the mastering display. This means that you only *really* need to care about properly tonemapping that 1000 nit range to your mediocre display assumedly {8,6,4}00 nits display

I dunno how to handle HDR10+ btw, I know there is a dedicated tonemapping spec, but iirc it's paywalled

If there's no headroom then there's no tone-mapping, just clipping.

"map" implies headroom/space to map it

there's always tonemapping

Within a range the display is accurately able to "track?"

No, especially if there is no HDR headroom, tone mapping is needed

I was responding to this. If a display is capable of tracking up to 1000 nits, it needs to start tonemapping before 1000 nits.

If it doesn't start tonemapping before it runs out of headroom then it will clip

yes, thats why good displays will try to track PQ as far as possible before having a very strong tonemap for everything else

they will try to track

by tonemapping

everything tonemaps that's how displays works

?

the entire point of what is being said is that displays will try to maintain 1:1 with PQ as far as they reliable can before the tonemapping kicks in

this means if you have a PQ video that never goes higher then say 600 nits, and your TV can reliable display 1000 nits, the signal will be un touched, the displays tonemapper will pretty much just do noop

nuh uh

1:1 makes zero sense since the viewing environment matters a lot. Or does PQ mandate a particular viewing environment for mastering, making it really not all that different from sRGB?

Honestly very little about the theory of HDR makes sense to me :D

You would think it would be as simple and obvious as it is in audio with changing the bit depth. But noooo. There's a whole bunch of extra assumptions and technologies attached to it and the separation between different technologies is muddled together rather than kept logically separated.

audio has nonlinear "transfer functions" too, like mu-law encoding. It's called "companding" in that domain.

They don't introduce a whole bunch of extra assumptions and unexpected complexity because completely different technologies and techniques are kept separate and distinct and are easy to understand and work with and mix/match.

I wish image processing was as simple and logical as audio processing 😂

Do I just not "get it?"

audio isnt simple either

1:1 in the sense of, "If you put a calibration tool to your display, in a reference viewing environment, send a PQ signal of 0.35, the display will output the nits equivalent of that"

they all do

you can't make a "reference" without controlling the environment

well, all in the sense of stuff that has a "reference"

So sRGB and PQ are the same and supposed nit levels are entirely ignorable

what? no

PQ has an absolute reference nit value

ofc reference is reference. but the "absolute intent" is there

can only speak for LG TVs: oled light and contrast set to the default 100 gives you accurate pq tracking, lowering either one makes it undertrack

this is caused by ABL, which is just based on the average picture level (APL). tuning the ABL curve for very high brightness in tiny windows necessarily makes it start dimming/undertracking at a rather low APL

afaik every consumer panel that currently exists uses just uses APL for the ABL implementation

[Hmm](https://cdn.discordapp.com/emojis/1113499891314991275.webp?size=48&quality=lossless&name=Hmm)

last I checked you often needed manufacturer mode to disable energy efficency stuff

e.g. full field gray/white in hdr on my lg c1 oled starts undertracking at around 60 nits

despite the panel being able to show like 120 nits full field

also because oled is really unstable at tracking and you can tell the difference in 0-100 nits

the math is fairly simple: assuming you have a panel that does 1000 nits in a 3% window (and starts dimming at a >3% window size), that means ABL kicks in at 30 nits APL

Undertrack meaning just divide brightness by x?

should be just that, yes

oled light controls the "backlight" of the panel, while contrast is like a multiplied that gets applied to the digital signal sent to the panel, but because the panel is actually running in 2.2 gamma both operations should be equivalent to dividing the luminance

How should I make this image smaller with -d >0? Now I could make it smaller only with -d 0

-d 7.2 works

That's only 2.5% smaller than -d 0

i think lossless works better because the alpha channel can be coupled with the image data

It's also interesting that -d 7.2 resulted in ssimulacra2 at 90.20265327

it's because of the alpha

It is extremely straightforward... Definitely way more obvious and logical... I never felt overwhelmed with terminology and multiple different ideas being mixed together and conflated for one another like there is in images/video

yeah but have you heard about loudness

Sure, it's often measured in decibels...

that's for sound

<:clueless:1186118225046540309>

It's never been a big deal or a crazy mess like it is in video

most digital audio is only measured as a sort of relative loudness

loudness curve is like tonemapping

It can get complicated, but when it does it's usually because of one specific thing at a time and not because of multiple completely unrelated things interacting with each other badly all at the same time

8bit palette with only 46 colors, so even with Alpha aside it's much more suited for lossless

And the completely unrelated things are never conflated or assumed to be related

Now PNG is even slightly smaller than JXL <:FeelsAmazingMan:808826295768449054>

Not surprised, the palette coding still needs a lot of work to even beat GIF most of the time

audio isn't simple? clearly never heard of "just use opus" :D

(/s)

just use opus ! 😄

just use mp3 /s

im curious about https://github.com/libjxl/libjxl/discussions/3634

is there compression between frames?

the encoder doesn't really do it, but there could be

Reading that post, is this lossless or high quality lossy? Because lossless uses square groups with 128, 256, 512 and 1024 pixel sizes, so that might be why compression didn't increase with merged images. We've had discussions about Satellite imagery with similar ideas, but they ended up using extra channels which could then reference each other to save space, I'm not sure if there's anything to do the same with frames yet. One idea to test is using APNG as an intermediary run though optipng to make use of identical areas, as cjxl can then reuse the cropped frames/layered transparency during encoding.