JPEG XL

I now see that there is test vectors for jxl, and two ways to have alpha 😦

sorry for the late response, but I noticed the issue is not visible for non-signed in users. Is it still visible to you guys?

I just get this 404

What should I do? Close it and make a new issue?

give it a try

maybe wait first before doing that since it could be something the core devs are able to solve

As it turns out, making a new account can trigger Github's antibot detection. I'm trying to get my account unflagged. 😔

Got my account unbanned. Just to make sure the issue is visible, I closed the old one and opened a new issue: https://github.com/libjxl/libjxl/issues/3517

<@794205442175402004> I know you guys use Aurora AV1 for AVIF at Cloudinary - what do you use for AV1 video?

We used to use aurora for both. We recently switched to svt-av1 for video and libaom for avif. According to our testing, the open source encoders have caught up with aurora, and there is no point anymore in paying for aurora licensing.

Aurora used to have a relatively good speed/compression trade-off compared to the rest, which were too slow for us (or not good enough, at their faster settings). But the open source encoders improved faster than aurora did, and are now as good or maybe even better.

How much can you talk about your testing methodology to us? I think it'd be interesting to know at least some of the surface level details

For this we mostly just look at metrics (rate/distortion) and speed measurements.

Gotcha. Which metrics are u guys most concerned with?

our video guys are mostly looking at vmaf, for still images I mostly use ssimulacra2 and aimos (our own proprietary ai-based metric made by our ai team)

Gotcha. Subjectively, how effective is aimos?

tbh, I don't really know. On CID22 it correlates almost perfectly, but it was trained on that set (they did of course only use only the training set and not the validation set for training, but still).

the validation set still presumably influenced hyperparameter choice

it probably did, but also: they contain different images, but the set of encoders/settings was the same between training and validation sets. So my main concern is that it might be overfitting for those specific encoder versions and may not generalize well to different encoders / encoder versions / settings...

what is the to—do list for jpeg xl until the moment when you can say: well, it's done, we can release it.

More or less this: https://github.com/libjxl/libjxl/labels/v1.0

Hi devs, is it normal to have `Quantization table with index 0 not found` error when decoding a normal JPEG with the jpegli decoder? No error occurs when decoding a jpegli-encoded jpeg.

The jpegli decoder is being too strict, I think this has already been fixed.

Thanks, I was using the jxl v0.10.2 release version. I'll try from the main branch.

I've had some test on the main branch jpegli decoder, it's still too strict. Like: ``` Unexpected marker 0xd7 ```

The same jpeg can be decoded by other decoder.

Maybe they should turn that into warning instead of error

``` // error from the libjxl decoder with version v0.8.1 libjxl/lib/jxl/modular/transform/transform.h:99: JXL_FAILURE: Invalid transform RCT type libjxl/lib/jxl/fields.cc:619: JXL_RETURN_IF_ERROR code=1: visitor.Visit(fields) libjxl/lib/jxl/modular/encoding/encoding.cc:458: JXL_RETURN_IF_ERROR code=1: status libjxl/lib/jxl/modular/encoding/encoding.cc:597: JXL_RETURN_IF_ERROR code=1: dec_status libjxl/lib/jxl/dec_modular.cc:413: JXL_FAILURE: Failed to decode modular DC group libjxl/lib/jxl/dec_frame.cc:309: JXL_RETURN_IF_ERROR code=1: modular_frame_decoder_.DecodeVarDCTDC(dc_group_id, br, dec_state_) libjxl/lib/jxl/modular/encoding/encoding.cc:542: JXL_FAILURE: No global tree available but one was requested libjxl/lib/jxl/modular/encoding/encoding.cc:597: JXL_RETURN_IF_ERROR code=1: dec_status libjxl/lib/jxl/dec_modular.cc:413: JXL_FAILURE: Failed to decode modular DC group libjxl/lib/jxl/dec_frame.cc:309: JXL_RETURN_IF_ERROR code=1: modular_frame_decoder_.DecodeVarDCTDC(dc_group_id, br, dec_state_) libjxl/lib/jxl/dec_frame.cc:644: JXL_FAILURE: Error in DC group libjxl/lib/jxl/decode.cc:1142: frame processing failed ``` Hi devs, I'm using the encoder from v0.10.2 and a decoder from v0.8.1 The decoding can cause this error. The decoder is from a third party, so I cannot change that. Is it possible to overcome this issue in the encoding part? --- ``` // from jxlinfo JPEG XL image, 7424x4936, lossy, 16-bit Grayscale num_color_channels: 1 num_extra_channels: 0 intensity_target: 1000.000000 nits min_nits: 0.000000 relative_to_max_display: 0 linear_below: 0.000000 have_preview: 0 have_animation: 0 Intrinsic dimensions: 7424x4936 Orientation: 1 (Normal) Color space: Grayscale, D65, Linear transfer function, rendering intent: Relative ```

Interesting, I wonder what causes this. Does it only happen on grayscale images?

In my several tests, YES. Some Grayscale images are fine to be decoded, some are not. But all RGB images can be decoded with no issue. But I only did several tests, maybe it's not enough.

And does it decode with 0.9?

I don't know, haven't tried yet.

I don't remember it but could be some bug in specifically grayscale image decoding that has been fixed in the meantime. If upgrading the decoder is not an option, then perhaps you can circumvent by encoding all images as RGB (e.g. convert PGM to PPM before encoding, or iirc in ImageMagick you can convert to png24:bla.png instead of just bla.png). Not an ideal solution but if it's really a decoder bug that meanwhile has been fixed, that might be the best you can do for now...

Thanks for the advice. I'll try to figure out the version that fixes this issue.

I've tested v0.8.2 and v0.9.0 djxl from v0.9.0 can decode the jxl file to exr. but the djxl from v0.8.2 failed.

Hi devs, if I'd like to split a large image to tiles and encoding them with libjxl separately, what's the optimal tile size for the balance between quality and file size? I've tried chunked encoding with libjxl, it seems to use a 1008x1008 size tile. So is 1008x1008 the optimal size?

what is your motivation for doing that in the first place? you risk getting boundary artifacts that you wouldn’t get by encoding as just one image

They could be using lossless, but it is a strange tile size for chunked encoding... 1024 would make more sense to match max group size

Better to use the chunked encode API than to manually split an image.

Is there a reason the chunked encode uses 1008 instead of 1024? (Assuming they're correct)

I’ve done some tests, low distance like 0.1 can make the tile edge artifacts unnoticeable.

Both lossless and lossy are used in my app.

You are right, I should use chunked encoding instead of manually splitting to minimize edge artifacts. Thank you~ BTW, is there a method to specify the size of a chunk? In the C API callback function, it just pass the with and height to me.

For lossy, the ideal chunk size is 2048x2048 (one DC group)

Do you mean in manually splitting the ideal size is 2048x2048? Because I cannot find a way to set chunk size when using `JxlEncoderAddChunkedFrame`. The `xsize` and `ysize` are passed in value, I didn't find a way to set these values. https://libjxl.readthedocs.io/en/latest/api_encoder.html#_CPPv4N26JxlChunkedFrameInputSource25get_color_channel_data_atE

<@179701849576833024> can clarify, but iirc you cannot choose the chunks, the encoder will ask for chunks and you have to feed them to the encoder. And it will ask for 2048x2048 chunks (except at the right and bottom where they will be smaller depending on the image dimensions).

Thank you for explaining this.

Yup

hi <@179701849576833024> in file gif.cc, there is a comment "TODO(veluca): this could in principle be implemented.". about "GIF with dispose-to-0" It's about the issue 3134 : https://github.com/libjxl/libjxl/issues/3134 when I remove the code, the JXL file is created and it's not glitched. If someone know what this mean 🙂

Is it normal for 16bit grayscale source doing lossless encoding, effort 3 always produced the smallest files? I've tried 5+ samples, they all follow this rule.

Sounds like some encoder choices are not very good then for such images, at the higher effort settings. Maybe open an issue about it on GitHub so we can at some point try to improve this

I always use effort 1 2 and 3 for lossless

Anything higher is a waste of time (for lossless!) and the diminishing returns are really bad. Tbf I'm still disappointed that 7 is the default (for lossless still), it's an extremely bad deal both for encoding and decoding time.

(For lossy mode, literally none of this applies and 7 is perfectly fine)

no prob, I'll open an issue for this.

https://github.com/libjxl/libjxl/issues/3713 it's posted here. And I also uploaded an sample file

I'm now using 3 for lossless too.

I don't think people realize just how bad it is, for decoding. The amount of time and power it takes to decode a lossless e7 file is enormous compared to near-instant decoding of lossy or lower-effort lossless.

Hey, how often are new libjxl versions released? Would be great to see this change in particular: https://github.com/libjxl/libjxl/pull/3639

Should be out by the end of this month https://github.com/libjxl/libjxl/issues/3700#issuecomment-2217476773

Hmm, you know what, effort 3 doesn't seem to decode any faster than effort 9 does, at least on windows 0.10.3... Only effort 1 and 2 decode faster. The rest of the effort levels decode at roughly the same speed, so I was wrong about the decode speeds.

Right now don't bother with anything higher than effort 4, there's a really not-good bug right now with effort 5+ https://github.com/libjxl/libjxl/issues/3717

0.0 interesting, ```ps ➜ firepal ffmpeg -hide_banner -loglevel error -i 351279248-4dcfec20-9f3b-448a-9f75-af03db2086e1.png -c:v ppm -f image2pipe - | b3sum 4bd5df8aa400d9de6b4d15d3a802ad72857e410e0b8a1464917c0454e3621a30 - ➜ firepal ffmpeg -hide_banner -loglevel error -i 351279248-4dcfec20-9f3b-448a-9f75-af03db2086e1.png.jxl -c:v ppm -f image2pipe - | b3sum 58cd0a688733c546c33976a695df5f161e9fc08365c74fe42063b6f24dc6528c - ➜ firepal ffmpeg -hide_banner -loglevel error -i out.jxl.png -c:v ppm -f image2pipe - | b3sum 58cd0a688733c546c33976a695df5f161e9fc08365c74fe42063b6f24dc6528c - ➜ firepal ffmpeg -hide_banner -loglevel error -i out.oxide.png -c:v ppm -f image2pipe - | b3sum 58cd0a688733c546c33976a695df5f161e9fc08365c74fe42063b6f24dc6528c - ```

Must be an encode error, tho it's fascinating to see it's compliant JPEG XL that decodes the same over djxl and oxide

Is it a regression or always been a problem?

I don't know. I will try older releases. Also this slowmode is really annoying. I'm using <#794206170445119489> next time..

See https://discord.com/channels/794206087879852103/794206170445119489/1265276547804299295

Wow. Really? Lossless bugs are pretty serious for a codec.

it must be some extremely rare edge case, otherwise it would have been caught way earlier maybe there should be some in built verification system (it attempts to decode and checks the pixel hashes, like what libFLAC's frontend does)

I tried with the released static versions 0.8.2 and 0.10.2 (as distributed on github) for the 0.10.2, the square is green 😄

Green means all tests are passed ok now 😅

Hi <@794205442175402004> , About this PR https://github.com/libjxl/libjxl/pull/3563 It seems for one generation, results from distance 0.2 to 0.5 without Gaborish are slight worse than the results with Gaborish. So If I'm sure my jxl file won't be encoded twice, I should enable Gaborish from 0.2 to 0.5 for better image quality manually, am i right?

I don't think the results are really slightly worse at d0.2-0.5 for no-gaborish. The metrics are slightly worse but the bitrates are also a bit lower, for identical distance settings. The overall bitrate/distortion curve does look better for no-gaborish.

That's great. Thank you.

I've tested the gaborish problem. Left: Reference Middle: distance 0.46 with no gaborish, size: 4,401,833 bytes Right: distance 0.5 with gaborish, size: 4,358,112 bytes According to the file size, the d0.46-no-gab should be better than d0.5-gab. But the middle one is a little blur than the right one. Is it normal? <@794205442175402004>

It is normal that there is a difference

OK then there is no prob

A builtin verification would be nice. I'm already doing verification myself.

I thought I was excessively cautious but now I see it's worth it.

You mean an option (or default) in cjxl to decode the image after lossless encode and check that it's equal to the input?

I wonder what is causing this bug, but it's quite bad to have lossless not being lossless, especially if it's already for a few releases now

``` t.png Encoding kPixels Bytes BPP E MP/s D MP/s Max norm SSIMULACRA2 PSNR pnorm BPP*pnorm QABPP Bugs ------------------------------------------------------------------------------------------------------------------------------------------- jxl:5:d0:buf0 4376 1056367 1.9310034 0.446 12.111 nan 100.00000000 99.99 0.00000000 0.000000000000 1.931 0 jxl:5:d0:buf1 4376 1053216 1.9252435 2.488 14.505 92.71890411 36.27901621 45.83 17.75226269 34.177428099747 178.506 0 Aggregate: 4376 1054790 1.9281213 1.053 13.254 92.71890411 68.13950810 67.69 17.75226269 34.177428099747 18.566 0 ```

12 total threads, 1 tasks, 0 threads, 0 inner threads ``` t.png Encoding kPixels Bytes BPP E MP/s D MP/s Max norm SSIMULACRA2 PSNR pnorm BPP*pnorm QABPP Bugs ---------------------------------------------------------------------------------------------------------------------------------------- jxl:5:d0 4376 1052948 1.9247536 0.597 3.396 nan 100.00000000 99.99 0.00000000 0.000000000000 1.925 0 Aggregate: 4376 1052948 1.9247536 0.597 3.396 0.00000000 100.00000000 99.99 0.00000000 0.000000000000 1.925 0 ``` 12 total threads, 1 tasks, 0 threads, 2 inner threads ``` t.png Encoding kPixels Bytes BPP E MP/s D MP/s Max norm SSIMULACRA2 PSNR pnorm BPP*pnorm QABPP Bugs ---------------------------------------------------------------------------------------------------------------------------------------- jxl:5:d0 4376 1053081 1.9249967 1.162 6.678 14.70520673 78.02987646 58.81 3.13979607 6.044097097064 28.307 0 Aggregate: 4376 1053081 1.9249967 1.162 6.678 14.70520673 78.02987646 58.81 3.13979607 6.044097097064 28.307 0 ```

so it seems to happen specifically when using multiple threads and doing streaming encoding

I'm doing two kinds of verification. Comparing pixels after lossless image endcode and compare files after lossless jpeg transcode.

I’m worried now because I encoded all my images and don’t have the originals, is there an automated way to test for them

The thing is, you are *required* to have the original pixel data, or at least the hash of the original pixel data, to verify the integrity of the JXL lossless file. I don't think JXL has a built-in mechanism to store the hash of the original pixel data in the metadata. In the back of my mind I was anxious about the possibility of lossless mode having implementation errors, and this issue is very sobering. Good thing I only did this on publicly-available music cover art for the sake of archival. I might make a little wrapper script that checks the output pixel data every time...

Encoder bugs that are not obvious (no crash, no invalid bitstream, just wrong) are the worst...

I think I could improve my verification by decoding pixels using djxl and jxl-oxide and comparing both of them with the source image. That should guarantee the image is encoded flawlessly.

I know there's some basic benchmarking run in Github Actions to test obvious discrepancies from PRs, but maybe expanding it to cover the newer features, such as chunked encoding, and running a full test on releases would be a good idea

There are plenty of tests already, but most are on small images to keep the time for CI reasonable.

I guess for lossless we should add some slow test on a large corpus, maybe to run only before releases

Yeah, similar to the CL:full label, an opt in full test to make sure there's no regressions before releases

I have tuned everything to work together with gaborish -- all the quantization matrices are chosen in a way that they work best with gaborish. If we turned it off, we could re-optimize the quantization matrices to have more high frequencies.

Or just have a `flac -V` mode that verifies round trip sample value equivalence/reversibility, and maybe have it enabled by default or have some kind highly visible advice to enable it when doing lossless mode?

Hi <@794205442175402004>, I watch this video and wondered if you saw it: https://youtu.be/FlWjf8asI4Y

Haven't watched it yet but saw it mentioned on <#822105409312653333>

Sounds reasonable. I tuned gaborish ON when distance is greater than 0.2 for my app.

<@794205442175402004> does Cloudinary use SVT-AV1 or libaom for AVIF encoding?

We used to use Visionular's Aurora encoder, now libaom for AVIF and svt for AV1 video.

And you guys test AVIF encoding with SSIMULACRA2 right?

Not only.

just wanted to share some interesting results we managed to get out of SVT-AV1-PSY recently. was wondering if you guys had moved to mainline SVT-AV1 for everything, because SVT's allintra is becoming more competitive (especially PSY) which definitely surprised me; I didn't expect these results at all (source image was https://i.slow.pics/jxFMVAIZ.webp). haven't tested any other metrics yet, but it'd be cool to know what Cloudinary uses on the image side to help work on this stuff & make more perceptually relevant changes

anyhow, I know u guys use the internal AI metric, but what else for images? if u can disclose

Mostly internal AI metric and ssimu2, also we look a bit at other metrics like butteraugli 3norm or dssim.

Those results for psy svt do look good, question is if it generalizes to more images...

Hello! I have two requests...

Extensions should be `.lossy.jxl` and `.lossless.jxl` when possible. Lossless images are more universal and important during editing. Lossy images are more specific. This way we **don't need** to compute it every time. Filenames are quite good...

The most important settings should be appended by most tools as filenames when possible. Filenames have the best backwards compatibility... Filenames are 0 code workarounds. `ABC (q=0.9; effort=0; N colors).lossy.jxl` `ABC (q=0.9; effort=2; M colors).lossy.jxl` If original filenames had ( then jxl should use [ symbol. If original filenames had ( and [ then jxl should use { symbol. If original filenames had ( and [ and { then...

I saw visually better results **but slightly bigger** filesizes with AVIF for some lossy icons with q=0.4 (2-8 colors)... Maybe this is due to Google Chrome, Android 13 or the Squoosh app...

Discord is far better than mailing lists but Google and others can't index it... Paid indexer bots options exist but are overpriced IMO...

Where security reports should go? Please update websites with private channels... https://github.com/libjxl/libjxl?#contact https://jpeg.org/jpegxl/ https://jpegxl.info/

What about CUDA acceleration?

What about mostly hardware encoders and decoders?

What about energy efficiency?

<@387224746133291022> You know, you can ask the exact same questions, without the slowdown limitation of this channel, on any other channel. Devs aren't aristocrats, they do talk with the peasants in <#794206170445119489> 😁

we're going to look into it. is the CID22 dataset public?

Only for 49/250 of the source images, the validation set, I got authorization to make it public

can point me in their direction?

https://cloudinary.com/labs/cid22

Under downloads, you can get the whole image set but only annotations for the validation set

I also think it would be nice to have an easier convention to tell apart lossless and lossy encoding with the filename. You can name your files anything you want and personally I like the convention of naming lossless files with the `.jxll` suffix. And jpeg transcodes `.jjxl`

The problem with the single `l` is that new people can confuse lossy and lossless.

It's doubled up, though. jxll

JXL is a format. Single L afterwards. Some could misinterpret things pretty easy IMO. Maybe `.jxll` and `.jxlll` are better... They are easier to mistype by hand... Remember that ordinary people and younger people aren't familiar with lossy and lossless terminology... L and ll are oblivious when you know the words.

Bit rot and bit flipping happens more frequently with bigger strings... Strings are more Google scale proof and space-proof by being redundant IMO... I don't like 3-5 character strings without redundancy because of this... If extensions and filenames have low edit distance bitwise then I like them less. I prefer `A` and `O` over `a` and `b` from ASCII and so on...

If you want one for lossy, then maybe `.ljxl`

But personally I would just use `.jxl` for both lossy and unknown encoding, and only use .jjxl and .jxll to help identify files that are lossless transcodes. I don't think you need a separate extension for lossy jxl, but if you really wanted one then I think `.ljxl` would be good to contrast with `.jxll`. One stands for "lossy jxl" and the other stands for "**j**peg **x**l **l**oss**l**ess."

doubled up L's versus a singular L. Makes sense and seems convenient to me. Certainly less lengthy than `.lossless.jxl` or `.jpg.jxl`

JPEG are always lossy and don't have this kind of ambiguity. Most people don't know how to save time instead of calling commands every time...

I will file the same suggestion with the arguments about just `.webp`.

Yes, when they receive webp or avif, and can not open it 😅 My sister is among them

I'd stick to jxl, jxl**l** and jxl**j** for lossy, lossless, and jxl'ed jpeg respectively, because this way they are grouped together in file list

Yeah, people do complain about that lol, but on the web file extensions don't matter, only mime type does. and webp certainly isn't a model that everyone strives to imitate... I still prefer `.jjxl` though, I just think it looks cooler and sounds better... I don't mind if some file explorers sort it slightly differently because I never ever sort by file extension anyways...

Do people use different file names / extensions for lossless PNG than for lossy PNG? If it helps your workflow to use naming conventions, then of course you're free to do that. This is outside the scope of what can/should be standardized though. Concepts like "lossless" and "lossy" are actually surprisingly hard to define. Where exactly do you draw the boundary? PNG cannot represent any camera picture _really_ losslessly, since you first need to at least debayer, do basic white balance adjustment, and quantize the RGB values to integers before you have the data in a format that PNG can represent.

For me it is just quick indication where\what format this picture came from. I.e. what it was before jxl, and\or how I compressed it. For jpeg-jxl it is just faster than asking jxlinfo if there is a jpeg reconstruction data, or not, and for true jxl it indicates, for example, how it should be treated when editing - will it suffer from multiple resaves or not. But I agree that in general all these custom helper extensions should not be the part of standard, maybe with exception of jpeg-jxl, but not really.

but after you upload it becomes urosbvh27rvx9.jpg (webp)

I don't think it's outside of the scope and I don't think it's as hard to define as you say. Lossless compression is very obviously distinct from lossy compression and most people I image would want less ambiguity and more convenient ways of distinguishing the two.

Lossless and lossy compression techniques are different, and to some extent jxlinfo can tell you what was used. You could dig further and also check if it is modular or vardct, and what modular transforms are used. But that still doesn't tell you much about the overall workflow. You can losslessly compress a decoded low-quality avif, or a heavily downscaled thumbnail, or an HDR image quantized to 8-bit RGB and full of banding, and these will all be much worse than a high-fidelity lossy version of the image. For representing image provenance information, an approach that is better than filename conventions is using C2PA metadata (aka JPEG Trust), which you can embed in JXL files using a JUMBF box (with or without brotli compression).

How does JPEG trust work with recompressing images? Also can I add jpeg trust to an image that wasn’t originally encoded with it?

JPEG Trust is a metadata spec that allows making assertions of the form "actor X makes claim Y about this image with hash Z, which is derived from source images A and B by applying operations P, Q and R". Whether you believe the (chain of) actors who make claims is up to you, but you can validate the chain of claims.

Doesn’t that add a lot of bloat to the file?

Doesn't have to, it can be just something like Exif/XMP, or it can be very detailed, you can choose what kind of granularity of information you want to include.

Not to mention Jpegxl's ability compress metadata. So you can append paragraphs of exif junk and it won't bloat file size compared to other image formats. Since Adobe products and cameras tend to add a lot of metadata, I think this feature should be advertised more. 😁

highly support the idea of dedicated filenames especially `.jjxl`, useful for web servers to serve JPEG-compatible JXL files transparently when ready

sounds like `.m4a` and `.m4v` to `.mp4`, and `.mka` to `.mkv` there isn't a need to implement it in JPEG XL or to make the convention to become mandatory, but it would be nice to have an optional scheme for quickly distinguishing lossy, lossless and JPEG-fronted JPEG XL files

Knowing what compression technique was used is good enough for most people. Cryptographic signing is for a different audience entirely. For some people, it's convenient to use a filename convention to identify lossless files or files with jpeg data in them. But applications shouldn't care. To applications, a file is just jxl. It's like how jpeg files can be .jpeg, .jpg, .jfif etc but it's just JPEG...

jxlinfo can tell you whether there is jpeg bitstream reconstruction data, and whether the image is xyb, so lossy, or not (so possibly lossless, or a recompressed jpeg). That's more reliable than filename conventions.

Yes, it is. Filename conventions are only for someone who wants to willingly identify for themselves and their own convenience. I think .jxl should be used most of the time, but if someone wanted to keep track of which files are lossless and which files are former jpegs, by renaming the extension, I think that's a convenient idea, and hopefully most software isn't stupid enough to get confused by the extension... (spoiler alert: windows is dumb)

I'm sure there was a github issue open for this. In the end filenames are the responsibility of the user and the applications that use libjxl. You can name it whatever you want, but don't be surprised if everyone and everything doesn't understand it

At the end of the day, though, the most reliable thing is reading the file, because it's just a filename. And a jxl decoder shouldn't care what the filename is or anything other than producing a pixel buffer from the compressed data.

If it's just for myself, I use filenames like bla.ppm.jxld2e6

Or sometimes stuff like bla.png.d1e7.nopatches.gab0.jxl

It might make sense to at some point add an option to libjxl to automatically add a JPEG Trust claim containing the encode options that were used...

jxly and jxls?

.thisjpegxlfileisprobablylossless Related issues https://github.com/libjxl/libjxl/issues/432 https://github.com/libjxl/libjxl/issues/2507

Most jxl files are raw, uncontainerized jxl with no metadata. If you want to embed metadata then you need to use the jxl container format. Most people probably don't want a few extra bytes to store what version and settings of encoder was used. But some people might, in certain rare cases.

I don’t think most people would care between a 546.9 kB and a 546.7 kB file

Think extremely small images, like 16x16 icons

Maybe make it so that if it is already including metadata, then add encode parameters, otherwise don’t. Small images like icons won’t usually contain any metadata while bigger images (like taken on a camera) probably will

Fair...

That's the entire point with filenames... You don't have to open files if metadata is good. The only way to get metadata in a most backwards compatible way is filenames. DOS names were completely stupid...

URLs have filenames. This is relevant for the Internet as well. We can know technical details from filenames even with really ancient browsers... We don't even need to leave ancient browsers.

There are too many dumb OSes and web servers though, that don't identify files by signature but by extension. On a dumb host, I have to rename jxl files to .jpg so they get served with a response header saying image/jpeg (which will cause browsers to display it as an image when opened directly in a tab), since for .jxl they will use the generic response header application/octet-stream (which will cause browsers to just download it, when opened directly in a tab).

there are a lot of web servers using nginx and having this as a problem since the default mime type config does not include JXL https://github.com/nginx/nginx/blob/master/conf/mime.types. it seems like there's a good chance that if a patch was submitted to the development mailing list that it would be accepted since 3 years ago the person who submitted the AVIF patch was just some random'ish person who at first had to be redirected from the GitHub to the mailing list. - for nginx: https://nginx.org/en/docs/contributing_changes.html - for free nginx (a fork, not sure how many people actually use this yet): https://freenginx.org/en/docs/contributing_changes.html

identifying files by contents is significantly more cpu and disk intensive than by filename. and if you don't recognize the `.jxl` extension, you're unlikely to recognize `0xff0a` anyway...

this is a good idea and I just sent a patch to the ML

Depends a lot on filesystem design. Aren't there filesystems where the first few bytes are stored inside the directory structures, e.g. sharing the space for the filename, timestamps, permissions etc with some initial content bytes? (also making very small files cheaper)

ext4 does that, but only for *small* files IIRC

Yeah looks like it only does inlining if it can fit the whole file in the inode. Maybe there are no filesystems that do what I described. It would make sense though, e.g. have fixed 512 byte records per file, which can almost all be used for the filename if it's a long filename, but if the filename is short then you could fit a few hundred header bytes in...

But the server is still providing the wrong mime type in that case...

Sure, there is no good solution if the server is dumb/old. Responding with image/whatever is better than responding with application/octet-stream though. You just need to get into the image decode codepath, and with application/octet-stream you don't get there... Unless the browser does sniffing based on the file content, which they all do for jpeg/png/webp/avif but not for jxl because they had to draw the line somewhere.

See also this discussion, which felt somewhat Kafkaesque to me tbh: https://github.com/w3ctag/design-reviews/issues/633

https://mailman.nginx.org/pipermail/nginx-devel/2024-August/UIGU3MEWNVQQ4BAT63CJDIAXR6AETCHC.html

that wouldn't work, because of hardlinks. Filename is stored entirely separately from data. Most filesystems store filenames inside the directory data, as name:number pairs, where the number then tells you where to go look for the file (meta)data.

I forgot about [CSS attribute selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_selectors) like `a[href$=".lossless" i]` and several formats... It should be `.jxl.lossy` and `.jxl.lossless`. This way we can download all lossless multimedia without RegEx if others follow the naming convention... DOS filenames were super constrained and inadequate for humans...

It **could** be. For you, or anybody else. But it should not be the standard

Some rare and sophisticated filesystems are designed to efficiently store small files in the inode metadata

but that's separate! The directory contains the file name and the inode number, the latter of which tells you where to find the metadata

Yes, that's exactly right. The filenames and inode are separate.

I need to get delta frames from full frames. Tools could use alpha channel as 1-bit mask channel. I also like to have bitwise lossy deltas instead of bitwise lossless. I would like to choose from several human-aware metrics...

I like APIs and commands with varargs for this. This way users don't have to script anything.

1-bit channels should be efficient and compressed... 8 and 16 bit alpha is a waste if only 1 bit is used.

I also like to have a permuting API where all permutations of complete files and their adjacent deltas are ranked. The lowest sum of **all** deltas should be first by default.

Funny person

The core devs don't need donations, most have a job at Google and I have a job at Cloudinary and I think our wages suffice (and donations will not make a difference in how much time we can devote to libjxl). If there are other people who want to work on libjxl or on other jxl implementations and could use donations, feel free to say so here.

I think you might want to post these as feature requests on the github, this channel is more for asking questions or seeking help with major issues. A few of those even already have open issues (You can also edit messages to concate thoughts through the slow mode)

sponsoring 'bounties' for say implementation in 3rdparty desktop/mobile apps or games would be relatively high impact. odd this wasnt a thing yet

I probably would pay to have better animation support/mjpeg to jxl support, ngl

But pay who? Main developers can't stretch their weeks and work more than they are doing now... And they are already paid. The problem is more about finding devs which could contribute

pay volunteers so they can dedicate more time to it?

Or pay bounties for features or debug?

Id say leave the jxl authors out of that, they got enough on their plate and already accomplish huge breakthroughs

Traneptora's work is very cool

Besides <@853026420792360980>, also <@206628065147748352> and others.

I guess the main thing you can do right now to support jxl is file bugreports at applications that don't support jxl yet or only in a broken way. And if you can, also submit pull requests to fix things, or try to find people who can. Even just identifying where the gaps are and reporting it in <#803574970180829194> can be helpful.

Maybe you know somebody in Adobe, to report wrong jxl handling in ACR?

id add that creating pull requests that get merged should be the *ideal* outcome but folks should not shy away from producing jxl-capable builds or even derivatives of that software if if brings em sooner - sometimes its easier to maintain that way for everyone if upstream is woefully unfamiliar with jxl

The issue with bounties is that they rarely ever get to the point where it is viable for a dev to actually tackle it, it's nice for some extra pocket money, but rarely is it ever "oh hey, I want that money"

who needs sponsoring? there might be ways to do some very light sponsoring without a lot of bureaucracy

Are u asking who here feels they need a sponsor?

https://discord.com/channels/794206087879852103/1256302117379903498/1256305400102391898

Hi devs, is it normal to see color banding issue in the 32bit lossless encoded jxl?

It has 249 unique colors, so banding is quite expected. How did you get this file?

<@625718249262284800> please continue this discussion in <#794206170445119489> so we don't have to deal with the slowmode

no prob.

I'm doing progressive rendering testing, is there a specific reason that lossless Modular doesn't support progressive but lossy Modular does?

Where are you doing your progressive testing? I'm using https://google.github.io/attention-center/ to test my images and lossy and modular images works fine for me.

I'm just using a shell script I wrote ```sh while [ $BYTES -le $TOTAL_BYTES ] do head -c $BYTES "$JXL_FILE" > "trunc/$BYTES.jxl" jxl-oxide "trunc/$BYTES.jxl" -o "trunc_png/$BYTES.png" -q BYTES=$(($BYTES + 200)) echo "$BYTES" done ```

Never mind, it's just a sense of scale. With `-d 1 -m 1 -p`, I get an image at 2 kB out of 200 kB (1%) but with `-d 0 -p` I get an image at 40 kB out of 5 MB (~1% again)

Progressive lossless is supported, but it is less effective than non-progressive since coding tools like (local) palette don't mix (well) with progressive encoding.

is jxl progressive lossless still better than adam7 png?

Depends on the image content. For photos, yes. For nonphoto, maybe.

for non-photo how should i get progressive decoding?

Tbh, for lossless it may be better to use the preview frame feature of jxl, and store the main image non-progressively. We should probably make it a bit easier to use those.

forgot about preview frames tbh, i need to go read the spec

Instead of `-p` on lossless, try `--progressive_dc=1` or whatever it's called

What does progressive ac/dc specifically do?

(Again, still really need to read the spec)

It's specific to VarDCT mode. Progressive AC is like how progressive jpeg works: the passes are between 1:8 and 1:1. Progressive DC is about how the 1:8 image itself is encoded: it can be encoded recursively as a progressive frame, either using progressive modular or using progressive vardct, giving you 1:64, 1:32, 1:16 previews...

any idea why maximum (lossless) compression seemingly gets worse with every new version of the encoder: ```cjxl -e 9 -E 3 -I 1 -q 100 --keep_invisible=0 ~/Pictures/testimg.png ~/Pictures/testimg.png.jxl JPEG XL encoder v0.6.1 0.6.1 [SSE4] Read 2048x1280 image, 17.8 MP/s Encoding [Container | Modular, lossless, tortoise | 1785-byte XMP], 4 threads. Compressed to 2173041 bytes (6.632 bpp). 2048 x 1280, 0.01 MP/s [0.01, 0.01], 1 reps, 4 threads. Including container: 2174874 bytes (6.637 bpp). cjxl -e 9 -E 3 -I 1 -q 100 --keep_invisible=0 ~/Pictures/testimg.png ~/Pictures/testimg.png2.jxl JPEG XL encoder v0.7.0 0.7.0 [SSE4,SSSE3] Read 2048x1280 image, 3438892 bytes, 117.3 MP/s Encoding [Container | Modular, lossless, effort: 9 | 1785-byte XMP], Compressed to 2290294 bytes including container (6.989 bpp). 2048 x 1280, 0.10 MP/s [0.10, 0.10], 1 reps, 8 threads. cjxl -e 9 -E 3 -I 1 -q 100 --keep_invisible=0 ~/Pictures/testimg.png ~/Pictures/testimg.png0.10.3.e9.jxl JPEG XL encoder v0.10.3 0.10.3 [AVX2,SSE4,SSE2] Encoding [Modular, lossless, effort: 9] Compressed to 2291.8 kB including container (6.994 bpp). 2048 x 1280, 0.259 MP/s [0.26, 0.26], , 1 reps, 12 threads. cjxl -e 10 -E 3 -I 1 -q 100 --keep_invisible=0 ~/Pictures/testimg.png ~/Pictures/testimg.png0.10.3.e10.jxl JPEG XL encoder v0.10.3 0.10.3 [AVX2,SSE4,SSE2] Encoding [Modular, lossless, effort: 10] Compressed to 2269.8 kB including container (6.927 bpp). 2048 x 1280, 0.166 MP/s [0.17, 0.17], , 1 reps, 12 threads. % ls -l ~/Pictures/*.jxl -rw-r--r-- 1 user staff 2269768 Aug 15 20:59 ~/Pictures/testimg.png0.10.3.e10.jxl -rw-r--r-- 1 user staff 2291769 Aug 15 20:59 ~/Pictures/testimg.png0.10.3.e9.jxl``` e10 with 0.10.3 is worse than e9 with 0.6.1 or what I am doing wrong?

We sacrificed a bit of compression for a lot of speed, generally speaking. Though there may also be some regressions. Also, some heuristics changed in ways that give better results for most images but worse for some images.

You might want to try `-I 100` instead of `-I 1` too

Ah yes, at some point the scale for -I changed from 0..1 to 0..100

Can we have good test files like these please? https://entropymine.com/jason/bmpsuite/bmpsuite/html/bmpsuite.html

Here's some data on base efforts across diverse images: https://loathra.com/jxl/bench/20240506.html#cjxl-releases-single FWIW, 0.10 achieves the most individual minimum outputs despite not achieving the best overall density. Also, it's specifically 0.7 and up where the scale of `I` changes.

Thanks! I tried: `-e 10 -q 100 -I 100 --keep_invisible=0` and got `2214959` bytes so better but still far from `2174874` v0.6.1 was producing Plus I don't recall 0.6.1 was taking that much time… I would love to get 0.6.1 back to run more tests but `brew upgrade` wiped it out (silly me didn't pin it) and I am unable to compile it from source (MacOS Catalina 10.15)

`E3`?

> Plus I don't recall 0.6.1 was taking that much time... According to your results: 0.6.1 - 0.01 MP/s 0.10.3 - 0.166 MP/s You didn't notice it was *16 times slower?* I think a 1560% speed up over a 4.19% loss in compression is still considered an improvement. Not to mention it's not even the "maximum lossless compression" possible, you have yet to try e11.

delete this

For the love of your computer and your lifetime, do not run e11

e11 or nothing™️

e11 is sometimes necessary to beat WebP 😄

Thanks <@207980494892040194> for pointing out MP/s difference. It was like 18months ago when I had 0.6.1 (and only saved the console output) but I sincerely don't remember waiting 16 times longer than I did yesterday with 0.10.3 Anyway, it seems like I simply can't compare it between different versions since [the core logic has changed](https://discord.com/channels/794206087879852103/848189884614705192/1277724721240084664) as well as the interpretation of the input parameters. OK. So [based on this](https://loathra.com/jxl/bench/20240506.html#cjxl-releases-single:~:text=566757570) it seems to me 0.6.1 had best overall compression ever available (??). Now with 0.10.3 it offers significant speed compression improvements but also, maybe not best but still decent, compression(). While I haven't tried that e11 (still not sure if it's a joke) there seems to be a combination of some advanced 0.10.3 e10 options available that generally squeeze the file to the bare minimum - but is there anyone that could help me decipher what [cjxl_0.10.2-d_d0e10E4I100g3patches0](https://loathra.com/jxl/bench/20240819.html#:~:text=cjxl_0.10.2%2Dd_d0e10E4I100g3patches0) means?

`2129083 (6.497 bpp)` - nice! thanks a lot 🙏 <@456226577798135808>!!!

what about this: `cjxl_0.10.2-d_d0e10P0E4I0g3X0patches0` or this: `cjxl_0.10.2-d_d0e10E4I100g3patches0wp_tree_mode2` ? I am guessing `-d 0 -e 10 -P 0 -E 4 -I 0 -g 3 --patches 0` but no idea on `-X` what does `-X !,0 -Y !,0 --patches !,0 --wp_tree_mode !,2` notation mean? bash says `bash: !,2: event not found`

iswhat are the `!`s for?

replace `!` by all possible values I guess it's some bruteforce thing

Indeed, for the first one, it's `-d 0 -e 10 -P 0 -E 4 -I 0 -g 3 -X 0 --patches 0`. The second one is not actually representable in cjxl CLI, since it relies on the non-existent argument `--wp_tree_mode`, but you could modify your libjxl if you want that. `-X !,0 -Y !,0 --patches !,0 --wp_tree_mode !,2` condenses 16 sets of arguments. Read `-X !,0` as trying each of default X (represented by comma preceded by no option) and `-X 0` for each of the following arguments. Read `-I !0,100` as trying `-I 0` and `-I 100`. Read `-g !0-3` as trying `-g 0`, `-g 1`, `-g 2` and `-g 3`. (And no, it's not intended as direct arguments to cjxl, it's how I concisely represent commands in my benchmark config file.)

How AVIF works at q=40; effort=10? Can JPEG XL allow us to use AVIF`s tricks as command line flag? AVIF has the best subjective quality and the smallest filesize pretty often with my very basic testing using Squoosh...

q=0.4 ? what is the scale ?

It's normalized... No. I was wrong about 100. Just 40 should be more straightforward.

You want to know how AVIF works at that quality, and use those coding tools in JXL?

If I remember well, jpeg xl don't compete very well (yet?) at very low qualities. That's why there is space for both formats. Avif for botched video previews and jpegxl for pictures (existing corpus of online pictures is more on the q=90 side IIRC)

Do you think JPEG XL could be competitive at low qualities if someone custom tuned an encoder specifically for low quality ?

There is certainly still room for improvement and a lot of unexplored bitstream expressivity. But the coding tools of jxl were designed for the distance 0-4 range, so I think that's where its inherent strengths will always be. Just like the coding tools for video codecs are designed for a much lower quality range, and they will inherently be better at that range.

Anyone can make an image coder that smooths and blurs the hell out of anything until there's hardly anything left to compress :)

Some JPEG XL version from Squoosh needs at least quality=90 in order to better preserve inscription text for further OCR... I suggest introducing areas with different encoding settings if possible. This can be useful for compressing people and characters rather than background. Some WebP 2 version has a similar problem... Some AVIF version is crazily good at such low quality but has some problems with colors...

AVIF has the best subjective quality and the lowest file sizes in most cases which makes it a codec for the Internet in my testing. It has good enough visual quality for lossy compression for many images IMO. AVIF may alter colors but never for foreground in my limited testing. I underestimated AVIF before... AVIF will cure JPEG-like artifacts in most cases... AVIF is good enough at FullHD resolution (one of the most common screen sizes in 2024). JPEG-like progressive was/is missing in AVIF. Thumbnails are missing now but should be good enough. https://github.com/AOMediaCodec/libavif/issues/605

q=90 is widespread because of JPEG and it's defaults IMO. I doubt that users manually set q=90... q=90 is overkill even with JPEG for some images.

Don't you think you've spammed this server enough? This isn't really the appropriate place. AVIF fan club is that way ----->

No problem discussing avif but not sure how the above is relevant to this channel. The devs you can contact here are jxl devs.

Maybe put this in <#805176455658733570>

How many times has he said "AVIF has the best subjective quality and the smallest filesize"?

While recompressing heavily artifacted jpegs

They mentioned it over 8 times...

6 hits for avif has the best... 1 hit for webp2 has the best... lol

fab from the av1 for dummies server?

i just know them from the AV1 for dummies server

<@794205442175402004> The Early Adopter role gives you access to "Create Expressions" and "Manage Expressions". Just checking, is this intentional?

I used to like avif. But then the avif tech lead unilaterally declared that the whole world isn't interested in jxl (because he speaks for the entire world, as its fairly appointed speaker) and that there's only room for avif and webp now (which were both his personal pet projects). So, I guess now that the avif/webp guy literally declared war on all competing formats, now I hate avif and want it to fail.

same, AVIF isn't inherently a terrible format, but the industry has room for more than 1 format, also maybe we should move channels

It seems like nginx are moving over to GitHub for contributions, should I submit a pull request and mention the patch you submitted to the mailing list or would you prefer to do so yourself?

I use q=94 when I need to compress a personal image well

I use JXL tbh

you win

No. You must provide A REASON why everyone have to 1. open `abc.jxl` and `xyz.jxl`; 2. parse some metadata in order to provide basic differences...

Why am I being pinged? Feel free to use a different channel to bypass slow mode

Apply your policies to your own data, you can trust that. You can't trust images found in the wild to have original data in any capacity. When uploaded to sharing sites, you expect metadata stripping, recompression, and name mangling.

True but my approach saves ~2 system calls. You don't provide any better alternative.

Also the image could have been previously compressed before you even got it, so your “lossless JXL” can still look terrible cause it’s from a low quality AVIF that was converted to PNG Also let’s move channels to not deal with slow mode

I said that your naming convention is only your, and should not be forced to everyone

I have a real scenario where CLI or any C/C++/Java/JS isn't accessible but I can script based on filenames. This just proves my point of filenames as a backward compatible workaround. Except for DOS and clones, which are way less than 0.1%...

Why your scenario limitations should have impact on all users? And as it was said - nothing prohibits someone to recompress lossy source to lossless jxl. Is this jxl lossless or lossy?

> converts Pinterest photo to 480p AVIF q=0 > converts that to q=10 jpeg > converts jpeg to png > converts png to lossless JXL > names it “meme.lossless.jxl”

Even `libjxl` can't help with this... Yes, both approaches are prone to this misuse at the moment...

*names it "artifacts.lossless.jxl"

Filenames should be cryptographically signed. This isn't specific to JPEG XL. Most filesystems are stupid.

So, you want JPEG Trust

English Wikipedia says something about ZFS and CryFS... It's unclear if and how unauthorized renames are handled by them... https://en.wikipedia.org/w/index.php?title=Filesystem-level_encryption&oldid=1224485789

Another page on English Wikipedia doesn't cover unauthorized renames as well... https://en.wikipedia.org/w/index.php?title=Disk_encryption&oldid=1243925010

Yet another page on English Wikipedia which doesn't cover unauthorized renames explicitly. https://en.wikipedia.org/w/index.php?title=Hardware-based_full_disk_encryption&oldid=1242249694

I can't find public implementations anywhere...

Maybe go to <#806898911091753051> filename based filtering and cryptography isn't really related to JXL

Real apps and users deal with such questions... Not the codec itself, yes.

the user is obviously authorized to make changes to their own files anyway...

im sharing a folder of images with someone but they're all in JXL, what is the current best easy way for someone to get JXL support on Windows?

IrfanView + plugin or XnView MP, I believe

maybe also migrate all jpegli issues to the jpegli repository? <https://github.com/libjxl/libjxl/issues?q=jpegli+is%3Aopen> <https://github.com/google/jpegli/issues>

Hi Jon <@794205442175402004> The digital photography industry suffer from a major technical debt when created the basic options on 8bit or 16bit/channel (de facto, Photoshop the moment we open any pure 16bit image, it converts it to 15bits/channel and Adobe is silent about it) In the meantime we have to deal with HDR 10bit screens and I found that Eric Chan (Adobe) tried a solution hard to implement and not a true and efective solution to the public. https://helpx.adobe.com/content/dam/help/en/camera-raw/using/gain-map/jcr_content/root/content/flex/items/position/position-par/table/row-3u03dx0-column-4a63daf/download_section/download-1/Gain_Map_1_0d12.pdf Can you explain me your personal vision about this 8bit/16/32bit <=> HDR 10bit future solution? How can a JPG XL embed a double image to both Standard and HDR screens that would be easily used from the general public?

My view: double images (i.e. gain maps) 'solves' a problem by making it more complicated. Gain maps allow the SDR and HDR renditions to be completely different, which in my view is too much. I prefer it to be well defined what actually "the image" is, and that becomes a problem in the gain map approach: effectively there are now two different images and any editing operation will have to take that into account. You can't just do the edits on the HDR image and then regenerate an SDR image algorithmically or the other way around. I would prefer to have an approach where only the HDR image is stored, plus some metadata (a new kind of "rendering intent") to describe how to do the tone mapping (just like we have the regular rendering intent to describe how to do gamut mapping). There could/should be some artistic control made possible with such metadata, e.g. to adjust the curves of global tone mapping or to set parameters of local tone mapping algorithms, and it should also be well-specified so the actual rendering you get out of it is consistent across implementations — I do think these are important things to have, and gain maps are addressing those issues but in my view in a way that is too explicit given that nearly always, the actual tone mapping will be done algorithmically (possibly influenced by a few adjustable parameters). So I think the main thing we need right now, is a standardized way in which HDR-to-SDR tone mapping is done (possibly with some parameters), and then we can just use HDR images and be sure how they will be rendered on SDR screens too. If more control is needed than what is allowed by this standardized tone mapping, then you can always just use two separate images (on a website you can use media queries to select the right image).

I see gain maps in JPEG XL more as something for authoring tools to conveniently and efficiently store custom local tone mappings, than as something that should be promoted for web delivery.

> some metadata (a new kind of "rendering intent") to describe how to do the tone mapping don't ICC profiles (as used by libjxl when outputting a HDR PNG) pretty much already do that?

in a very rigid way (same curve for the whole image)

ICC profiles as they are now can only describe global tone mappings; what I'm hinting at is a rendering intent that allows you to select a particular local tone mapping algorithm with some particular parameters (e.g. to adjust balance between saturation preservation and luminosity preservation)

Maybe it suffices to just repurpose the existing rendering intent field in ICC profiles (which was intended for gamut mapping but it could be interpreted more generically), and just define what tone mapping to do in each case: e.g. `Perceptual` could mean "apply local tone mapping algorithm X with the saturation-preservation parameter set to 0.4", `Saturation` could mean "apply local tone mapping algorithm Y with saturation-preservation set to 0.9", `Relative` could mean "apply global tone mapping Z" and `Absolute` could mean "just clip", or whatever.

Do you think the four rendering intents we are having for years in Adobe (Perceptual, Saturation, Perceptual and Absolute colorimetric) need a revision or adding new ones?

I'll leave that up to the ICC consortium to decide, but to me it would make sense to interpret this field in a way that also includes tone mapping intent, not just gamut mapping intent, and to actually define a standardized way to implement Perceptual and Saturation (currently they're "vendor-specific").

We do have room for a few more rendering intents in JXL, the value is stored as an Enum so we can have 64 different values (currently only using 4 of them)

My personal take on dual image is that it is a huge mistake. There should be one HDR image and it should be displayed on SDR using "best effort" fast local tone mapping algorithms -- not by a tone mapping algorithm chosen at encoding time. I believe there is no reason to actively distract the dual-image community -- it is so bad that it will eventually collapse on its own. I believe that once it propagates minimally and people start to slowly understand how it works, it will be forbidden to use in many communities -- and remain unsupported by most tools.

I think I would rather just actually store two separate images

and maybe pick which one to display using CSS or something

Well I do think some degree of artistic control over the tone mapping can be desirable — but I think it suffices to have just a few adjustable parameters for that (perhaps as little as two bits of information, maybe a bit more). And I also think it's not a good idea to just let everyone do its own "vendor-specific" tone mapping, since that means there is no consistency between platforms (macOS will do something different from Windows, etc etc) and no way to know in advance how it will look. If you want full control: use two separate images. If the standardized tone mapping is good enough: use one image.

CSS already has [media query for dynamic range](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/dynamic-range) which can be used for resource condition of `<picture>` element, and it has high browser coverage

bonus: you only need to download one image

It would be nice if we could also get a corresponding Client Hint in the request headers so we can also use it server-side to vary responses based on it. That makes it possible to have a single <img> tag which will give you different images depending on your screen cabilities (and on the decoders you support, via the Accept header, and on the viewport width, via an already existing Client Hint). The picture srcset approach is nice in some cases, but if you want to have 5 different sizes depending on viewport width, 4 different formats depending on what is available (e.g. jxl, avif, webp, jpeg) and then also 2 different dynamic ranges, that's 40 different variants and the markup gets pretty ugly pretty fast.

makes archiving easier, though

archiving in what sense? fetching a whole webpage?

yes, or website

with something like httrack or the like

or what web.archive.org does

yes, I guess http lacks some mechanism for getting an overview of all the variant responses

it is all rather complicated -- if you take an SDR display into a darkly lit room, suddenly it is more HDR than SDR -- the same if you bring an HDR display outside to the balcony or park, it is more of an SDR display from the perception viewpoint

That is also true. But presumably user agents can take into account the current HDR headroom, which will be variable when the display is auto-adjusting to the ambient light.

Re: different formats Isn’t that what accept headers are for

Yes, for codec selection there is a good mechanism already with the accept header. For image dimension selection there is also a good mechanism already with client hints for viewport width, unfortunately only implemented by Chrome for now. For HDR/SDR selection, there currently is no way to signal that in a request header, afaik.

also, the same image can be displayed in different roles on the same machine -- like an image rendered in SDR when it is in a gallery, and then incrementally upgraded to HDR when viewed individually

The more image storage the more you pay. Not feasible if you have hundreds of thousands or millions of images.

For most web delivery use cases (excluding things like personal photo storage), bandwidth is a bigger cost than storage. But yes, avoiding variants if possible is better...

Allways turned off this on my devices. It always show me strange colors. Tried Mac and android

<:BanHammer:805396864639565834>

I've been running some benchmarks on images from imagecompression.info and I noticed that for lossless compression `-e 7` and `-e 10` give very similar results, `-e 7` gave a smaller filesize in 7/13 images I tested. I'm just running `cjxl -d 0.0 -e 7 INPUT OUTPUT`. Example: ``` e7: Compressed to 37163.9 kB (24.528 bpp). 4256 x 2848, 0.622 MP/s [0.62, 0.62], , 1 reps, 28 threads. e10: Compressed to 37529.0 kB (24.769 bpp). 4256 x 2848, 0.017 MP/s [0.02, 0.02], , 1 reps, 28 threads. ``` Has anyone else encountered this? I'm running libjxl compiled ~2 months ago, so maybe this has changed (I looked through the commits since then, but didn't find anything relevant)

It is not surprising that you would ever find a lower effort producing smaller output than a higher effort. It is surprising when e7 is better in general, but a larger sample size should prove quite in favor of e10. That said, e10 is not efficient and is particularly susceptible to tweaked e7 settings on photographic content.

thanks, do you have any suggestions what type of images might clearly give the upper hand to e10 or at least be less susceptible than photographs?

there should be a larger gap with sufficiently non-photo stuff in general, like ui, pixel art, or algorithmically generated images.

Hi Jon <@794205442175402004> I posted on LinkedIn a short article about SDR/HDR need for general standardization, specially for the e-com imagery. I agree with your opinion on regard but I was not expecting one of my colleagues invoking that for Android there is already a solution. I thinking about real standardization world wide not just from and Android dev. Here is what he posted: Well, you seem to forget the Ultra HDR (also known as JPEG_R) image format: https://developer.android.com/media/platform/hdr-image-format. It is mandatory and the default format for android 15 performance class https://www.androidpolice.com/google-android-15-performance-class-ultra-hdr-support/ (more info on performance classes https://developer.android.com/topic/performance/performance-class) And following this announcement https://www.business-standard.com/technology/tech-news/google-extends-ultrahdr-support-to-3rd-party-apps-what-is-it-how-it-works-124071500671_1.html it will probably become the default for all apps on android. Considering that more and more phones will support this image format, and that chrome and edge already support it as well as some other image editing software such as Lightroom https://petapixel.com/2023/10/11/lightroom-on-android-now-supports-ultra-hdr-photo-editing/, I would suggest that this image format is the way to go.

It's a solution yes, but not a very good one imo, at least not in the long term. If you are going to demand that everyone adds support for something new, I think it's better to use something state of the art for that. To get decent quality out of UltraHDR, you probably need about 3x the bytes of jxl. That's a high price to pay, and the only thing it buys you is graceful degradation (that is, many people will see just an SDR image without even knowing the image is supposed to be HDR).

also the argument about "creative control" doesn't apply because the SDR+Gainmap are both just being created from an algorithm, so the algorithm can just be done in reverse, and any calculated parameters can be stored instead of an entire gainmap

I have some sympathy for the “creative control” argument

neither image intrinsically _has_ to be derived algorithmically from the other

someone with dedication could conceivably edit the photo twice

(but then, I don’t know if interpolating between them is a good idea)

I would be curious to see what happens when the SDR and HDR versions take qualitatively different approaches to making the image look good, and then you just interpolate between those like UltraHDR does

For authoring tools it can make sense to have a single file that describes multiple variants of the image, and gain maps can be a way to make that effective. But for general image workflows (including web delivery) things become very messy if an image is no longer just one image but multiple variants. SDR vs HDR is just one way in which you may want to have variants, another could be color vs grayscale vs black & white, or different crops for various aspect ratios, high-contrast versions for vision impaired people, versions for print and for screen display, etc etc. Again: it may be useful for authoring tools to have a way of "bundling" all these variants. But I don't think it makes sense to expect that all image workflows will somehow manage to deal with the concept of an image object that is really a set of variants of an image.

And yes, just interpolating may not be enough for good artistic control. You may want to have an image for SDR, for 1 stop of HDR headroom, 2 stops, 4 stops, and the middle ones may not necessarily be just interpolated versions of the extreme ones.

In practice, I think you'll usually just want to use a single HDR image with some kind of algorithmic tone mapping (ideally with some signaled parameters so you have some amount of control over what they will do), and if you really want creative control, you make different images for SDR (or even for different amounts of HDR headroom).

can JXL store multiple versions of the same image like that?

Just like most websites just use the same image for screen display and for printing, i.e. preparing the image for screen display and thrusting that something sensible will happen if someone chooses to print the website. If you want to have more control, you can make separate images for printing and use a CSS media query, but usually it will not really be needed.

We did add gain map support to libjxl, it hasn't been officially added to the standard yet though. For other kinds of "other versions of the same image" we don't have anything, but it seems feasible for authoring tools to use jxl files for that if they want, e.g. using multi-layered images and some box with metadata describing how to compose the various variants using selections of layers, or something like that. It's not really in the scope of jxl itself to define such things though, I think. The only well-defined image is the "main" image you get by using a conforming decoder, anything else is left for applications to deal with.

I noticed Adobe Camera Raw can now export JXL and Adobe adopted 3 options for HDR color space. Can you explain P3 and Rec 2020?

Those refer to the color gamut, i.e. the position of the primary R,G,B colors. Basically sRGB < P3 < Rec2020 in terms of the size of the gamut, and many displays can now do P3.

i think of it as WCG -> more dynamic between colours HDR -> more dynamic between light and dark

For JXL, I don't really see any reason to use anything but Rec2020 (in this ACR/Lightroom export dialog), since either it will use 16-bit lossless (which is plenty for Rec2020 with PQ/HLG) or it will use lossy (where the colorspace tag doesn't impact the actual image data, it's just a suggestion for what to decode to). For more limited formats like AVIF, it could be a good idea to use a narrower gamut space if your image doesn't actually go outside that gamut, since you'll get better precision that way. AVIF is limited to 12 bit which is enough for Rec2020 PQ/HLG but it is a bit tight.

HDR is no more then a marketing term unless it's explained, you can't assume that HDR directly equates to luminance when you see it

Oh yeah, HDR as a selling point means nothing, I’m just talking about what I use the terms to mean

We need to be careful about the 16-bit lossless because the moment we open it in Photoshop, it quietly converts it to 15-bit (we need a good third-party histogram map to verify it). Old tech debt Adobe is silent with. Not 65536/channel but instead we are dealing with 32768/channel. That might be the reason they use CameraRaw export instead...?

Can lossless handle out of gamut?

It can, but not when using uint16 buffers

That's why I would recommend using Rec2020 in that HDR export dialog, better safe than sorry when it comes to gamut clipping

~~export using aces AP0~~ oh how I wish

given that current displays don’t really go beyond P3, that’s what I would tend to recommend, to make sure that you don’t accidentally export more saturated colours than you can see on your own screen

if you do get gamut clipping because of it, it’s gamut clipping that you also saw on your display

That's also a fair point. Not sure at what point Lightroom gives gamut clipping warnings though.

<@179701849576833024> with the `jxl-rs` wouldn't it make more sense for `alpha_associated` `spot_color` and `cfa_channel` to be stored as entries in the `ExtraChannel` instead of all being `Option<T>`s? ref: <https://github.com/libjxl/jxl-rs/blob/main/jxl/src/headers/extra_channels.rs>

probably, but that would require extending the derive macro 😄

<@794205442175402004> A digital image newbie asked me a question that made me stop and think before answering because it has its logic for a newbie to think like that, so direct and fair. But it is hard for you when he is not aware of the history of technical debt coming from the industry early adoption of 8/16/32 bit. He asked: > **- If we can easily convert an image from 8 to 16 or 32-bits in Photoshop, why can't Adobe add the 10-bits to the menu and move on?** It seams easy to list a bunch of reasons, but hard to resume it clear and short: What would be the argument short list you would use to help a newbie (or someone weak wisdom on digital image and color) get the clear ideia why it is not so simple as converting to 10-bits and having to deal with gain maps.

Editors and processing workflows (on general purpose CPUs) always use 8, 16 or 32 bit buffers because the basic data types CPUs can handle efficiently (in terms of registers and SIMD instructions) are things like uint8, uint16 and float32. The only way to do 10-bit or 12-bit in a practical way is to use 16-bit buffers in memory. In a file format you can avoid these padding bits, but even there: PPM, TIFF, PNG etc all require byte-aligned sample data since it's easier to deal with. Also note that converting low bit depth to higher bit depth does not magically recover the missing precision.

Super! Simple and short 👍 and aligned with the past/present/future 💯

Do you guys have a preferred dev environment for browsing and making sense of the C++ source tree and figuring out where things like `Status` is defined and whether it's a macro or a class? libjxl uses a confusing mix of different C++ features that make a lot of it look really ambiguous... (This is exactly why I hate C++ by the way... Way too many ways of doing the same thing that can all be mixed together at the same time making it hell to figure out what the actual data structures are...)

I use grep.

clangd

Like notepad?

No, like Corel WordPerfect

Hi <@794205442175402004> Hi Jon A e-com studio digitech guy mention that the future of ICC is CICP implemented in JPGXL and AVIF. I have lots of questions about it. Amongst those the costs and practical things to e-com studios (only firmware? new cameras having JPEG XL, new monitors, etc) Color calibration of a studio camera or a monitor will change a lot?

as far as I’m aware, it shouldn’t complicate colour management

for SDR, it’s just a more convenient way of specifying the source colour space among a subset of those that could already be expressed as ICC profiles

for HDR, ICC profiles were already not suitable anyway

so any HDR colour management approach that relied on knowing that the source is in Rec. 2020 + PQ can keep doing that and ignore the CICP

I always use CICP if possible because it's much simpler, smaller, and I don't really understand ICC.

While I do agree that ICC sucks to work with and has a lot of really stupid limitations and stuff, CICP is nowhere close to a valid alternative to it. ICCMax did fix a lot of the HDR issues iirc, but does anyone actually use ICCmax?

iccv4, iccmax, but calibration still on iccv2

In fact, CICP makes me ask more questions than answers because I represent the point of view of the photo studio Digitech, which needs to upgrade many photography studios and standard screens used by many teams, especially the color quality control team. - what hardware will need to change? Costs? - Will photo camera brands adopt it only for new cameras, or can we upgrade some firmware? - When would be the right moment to do it in the future? Is it a must for all e-commerce imagery? I would prefer to set up a dedicated channel for #ecommercedigitech once it interests a few in the e-com photo industry.

The E-commerce image industry never used HDR gain maps (eventually, a few high-end fine watch brands alike). The super big quantity of images to deal with doesn't allow it to streamline the process. The studios are color-calibrated, but some colors and materials (saturated or reflective fabrics, etc) are impossible to match on the screen and even in the HDR or Ultra.

Was this ever implemented? [Improvement of JPEG XL Lossy Image Coding Using Region Adaptive DCT Block Partitioning Structure](https://www.semanticscholar.org/paper/Improvement-of-JPEG-XL-Lossy-Image-Coding-Using-DCT-Cho-Kwon/9f76f21fac5be5146208a3d04900d392b0faca03)

CICP is just a standard for color metadata, from the video world.

Does ICC not work anymore?

I came from the e-commerce industry and learned to respect video pros highly because the video industry and TV broadcasting are the grandfathers and creators of digital image and compression. Lately, I have been in contact with some video experts who are helping me with image color control and getting more wisdom from them. They are sometimes perfectionists I need to adapt and transfer to my industry, which doesn't have the time and millions of images to deal with rapidly. There is a balance to do and adjust.

Still does, of course, but for end delivery, ICC is kinda unoptimal, overkill. Basically, CICP metadata is simpler, smaller, better standardized, potentially faster to render, and already supports HDR transfer functions well.

the CICP tag in an ICC profile just means that in addition to the usual ICC representation of that colour space, there are also a couple of bytes that specify the same colour space via its primaries/whitepoint and transfer function, in accordance with https://www.itu.int/rec/T-REC-H.273-202309-T/en

CICP is also capable of representing a few colour spaces that ICC couldn’t (those that use HLG or PQ), in which case “the usual ICC representation” might be a fallback representation

but so, those are use cases for which ICC was already inadequate

so an ICC-based colour management workflow is not made any less adequate than it already was by not handling the new CICP tag

Is CICP that 9-18-9

yes, 9 = Rec. 2020 primaries with D65 white point, 18 = HLG, 9 = Rec. 2020 matrix coefficients, iirc

(why do I know this)

technically, there’s also one more byte that tells you whether it’s full range or limited range

I just used that cause that’s what iPhones shoot in

If I am a fashion brand where the online e-com 95% of my sales come from HDR cellphones, in this (rare) case, adopting a streamlined process (calibration, cameras, monitors, photoshop enabling HDR settings) should be a must. The majority of e-com cases should be ICC-based.

CICP is extremely basic metadata defining transfer, primaries and matrix and thats it. https://github.com/AOMediaCodec/libavif/wiki/CICP#terminology

how the color metadata is stored does not matter much, as long as it's carried or correctly processed through the entire pipeline CICP usually makes that easier, though, compared to ICC

but does cicp support luts

no. It's literally **just** matrix / transfer / primaries. It's not even technically strictly enough to properly define a colorspace

depends on what you mean by that, but if your source has a CICP tag that says it’s in Rec. 2020 / PQ, and you have a LUT that converts from Rec. 2020 / PQ to Display-P3, you can certainly apply that LUT to the pixel data and tag the result as Display-P3

there isn’t really anything in CICP that would particularly help you with that, but it doesn’t preclude it either

cICP isn't understood by everything, so haveing an ICC profile fallback is not a bad idea

maybe also add <https://github.com/google/jpegli> <https://github.com/libjxl/jxl-rs> to <#847067365891244042>?

There is an issue that I have experienced for a long time with ssimulacra2_rs, which is that input bit depth matters in ways that shouldn't, with identical sources stored in bit depth > 8 returning slightly but quite significantly different scores than with the same source stored in a higher bit depth... But, I just noticed that this same issue exists with ssimulacra2 and butteraugli as well. To reproduce this, take any 8-bit PNG, and convert it to 16-bit with something like `magick convert source_8bit.png -depth 16 source_8bit-16bit.png`, and then calculate an ssimulacra2 or butteraugli score between them. The scores should show loss. But there is no loss, both files are identical. This can be confirmed with `magick compare -metric SSIM source_8bit.png source_8bit-16bit.png null:`, or converting the 16-bit version back to 8-bit with error diffusion dithering enabled. Someone should definitely figure out why this is happening across ssimulacra2 implementations, and butteraugli as well.

might it be that magick is doing the conversion wrong?

and then of course agreeing with itself when calculating SSIM

I suppose this hypothesis could be tested by doing the conversion via lossless jxl instead

Ffmpeg zscale and swscale produces the same. Vapoursynth Resize as well, which uses zimg like zscale. With ssimulacra2_rs video I only get the issue when I compare 8-bit with 10-bit, not 10-bit with 12-bit, all converted the same way. Also, when visually comparing them I don't see any difference, while I do when doing a near-lossless lossy encode that produces a higher ssimulacra2 score (higher than the ~95 I get when comparing identical 8-bit and and high bit depth content).

I think the issue here is limitations of float arithmetic precision, causing x / 255.f != x * 257 / 65535.f or something like that

Are you using a ssimulacra2 from before or after https://github.com/libjxl/libjxl/pull/3626 ?

Note that 8-bit can be losslessly converted to 16-bit, but not to 10-bit: e.g. 1/255 is not the same number as 4/1023.

I don't think this is correct. 8-bit to 10-bit means you have four times the values, and as I see it, the old value range can be perfectly scaled onto the new one, without even any sub-value shifting. See: ``` | | | | |||||||||||||||| ``` Perfectly spaced.

Well the convention is to interpret 8-bit values with an implicit denominator of 255, and 10-bit with a denominator 1023. Sadly 1023 is not 4*255

that's 4 to 16, they match perfectly because x/3 equal to 5x/15

Yes, 2-bit can be converted to 4-bit losslessly because 2^2-1 is a divisor of 2^4-1

but not with 256 to 1024 as <@794205442175402004> said

oops, you're right

Basically the issue is that we want to have 2^n integer values mapped to [0,1] inclusive. If we would be ok with mapping to [0,1) and having denominator 2^n, you could just change bitdepths by shifting and it would be lossless. But we want to have perfect white at all bit depths...

After. I just built freshly from git again to make sure. ``` ❯ ssimulacra2 /home/damian101/Pictures/sc2/sc2.png /home/damian101/Pictures/sc2/sc2-16bit.png 95.54799654 ``` ``` ❯ butteraugli /home/damian101/Pictures/sc2/sc2.png /home/damian101/Pictures/sc2/sc2-16bit.png 0.7919309139 3-norm: 0.312337 ```

What does ImageMagick compare -verbose -metric pae say?

``` ❯ magick compare -verbose -metric pae /home/damian101/Pictures/sc2/sc2.png /home/damian101/Pictures/sc2/sc2-16bit.png null: /home/damian101/Pictures/sc2/sc2.png PNG 3840x2160 3840x2160+0+0 8-bit TrueColor sRGB 11.1443MiB 0.310u 0:00.309 /home/damian101/Pictures/sc2/sc2-16bit.png PNG 3840x2160 3840x2160+0+0 16-bit TrueColor sRGB 25.7138MiB 0.350u 0:00.352 Image: /home/damian101/Pictures/sc2/sc2.png Channel distortion: PAE red: 287 (0.00437934) green: 256 (0.00390631) blue: 286 (0.00436408) all: 287 (0.00437934) /home/damian101/Pictures/sc2/sc2.png=> PNG 3840x2160 8-bit sRGB 11.1443MiB 1.290u 0:00.303 ``` ``` ❯ magick compare -verbose -metric pae /home/damian101/Pictures/sc2/sc2-16bit.png /home/damian101/Pictures/sc2/sc2.png null: /home/damian101/Pictures/sc2/sc2-16bit.png PNG 3840x2160 3840x2160+0+0 16-bit TrueColor sRGB 25.7138MiB 0.370u 0:00.366 /home/damian101/Pictures/sc2/sc2.png PNG 3840x2160 3840x2160+0+0 8-bit TrueColor sRGB 11.1443MiB 0.280u 0:00.277 Image: /home/damian101/Pictures/sc2/sc2-16bit.png Channel distortion: PAE red: 287 (0.00437934) green: 256 (0.00390631) blue: 286 (0.00436408) all: 287 (0.00437934) /home/damian101/Pictures/sc2/sc2-16bit.png=> PNG 3840x2160 16-bit sRGB 25.7138MiB 1.410u 0:00.322 ```

So there is a difference then, of up to 287/65535 in some sample values

Yes, that one was actually converted to 16 bit with ffmpeg zscale it seems... must have messed up my files. Imagemagick computes an ssim score of 0.999988 for that.... ``` ❯ magick compare -verbose -metric pae /home/damian101/Pictures/sc2/sc2.png /home/damian101/Pictures/sc2/sc2-16bit_imagemagick.png null: /home/damian101/Pictures/sc2/sc2.png PNG 3840x2160 3840x2160+0+0 8-bit TrueColor sRGB 11.1443MiB 0.280u 0:00.275 /home/damian101/Pictures/sc2/sc2-16bit_imagemagick.png PNG 3840x2160 3840x2160+0+0 8-bit TrueColor sRGB 11.576MiB 0.260u 0:00.264 Image: /home/damian101/Pictures/sc2/sc2.png Channel distortion: PAE red: 0 (0) green: 0 (0) blue: 0 (0) all: 0 (0) /home/damian101/Pictures/sc2/sc2.png=> PNG 3840x2160 8-bit sRGB 11.1443MiB 1.380u 0:00.314 ``` ``` ❯ magick compare -metric ssim /home/damian101/Pictures/sc2/sc2.png /home/damian101/Pictures/sc2/sc2-16bit_imagemagick.png null: 1⏎ ``` ``` ❯ ssimulacra2 /home/damian101/Pictures/sc2/sc2.png /home/damian101/Pictures/sc2/sc2-16bit_imagemagick.png 100.00000000 ``` ``` ❯ butteraugli /home/damian101/Pictures/sc2/sc2.png /home/damian101/Pictures/sc2/sc2-16bit_imagemagick.png 0.0000000000 3-norm: 0.000000 ``` ``` ❯ ssimulacra2_rs image /home/damian101/Pictures/sc2/sc2.png /home/damian101/Pictures/sc2/sc2-16bit_imagemagick.png Score: 100.00000000 ``` So, the conversion might have been the issue all along? Still weird to see butteraugli print out scores that normally indicate visible loss, on images that appear indistinguishable to me...

Yes, that seems like a bad butteraugli score for at most off-by-one errors in 8-bit...

How does SSIMU2 fare with image consistency? For example, if a 512x512 image has a bad 64x64 region, how much would SSIMU2 penalize that region when calculating a (whole-image) score? Have there been studies around evaluating SSIMU2 local consistency vs. metrics like Butteraugli or DSSIM?

ssimu2 uses 4-norm so should be able to penalize worst-case well enough, but not as strongly as a max-norm like Butteraugli-max

makes sense, thanks! good to hear ssimu2 takes local image quality into account

Butteraugli apparently decodes Bt.709 using the inverse scene-referred OETF. But most Bt.709 content is encoded using inverse gamma 2.4, and decoded as such, so the scores are way off in dark regions... But even worse, I think all Butteraugli scores might technically be off, because my perception of an image decoded with gamma 2.2 aligns better with Butteraugil distmap when Butteraugli decodes it using the inverse sRGB OETF than when it does so using gamma 2.2. Which makes me wonder, might the original "sRGB" training data have been performed on gamma 2.2 monitors but incorrectly decoded and mapped using the inverse sRGB OETF? However, correcting this would actually result in reduced perceptual alignment for most people, since many if not most "sRGB" images are meant to be decoded as gamma 2.2, and are usually watched just so anyway, due to most displays using gamma 2.2.

I really don't understand how this specification mess wasn't cleared up at least two decades ago...

Or was sRGB always meant to be decoded with pure gamma 2.2? But "approximately 2.2" could apply to the inverse sRGB OETF as well, although the difference is huge in dark regions.

pure 2.2, this is one of the few publically availible snippets of the actual sRGB spec EDIT: Cont <#1288790874016190505>

oh, where did you get that...

https://github.com/libjxl/libjxl/pull/3905 I'm updating the documentation here and I have a couple questions for the devs. 1) Does VarDCT e10 do anything special relative to e9? It seems to perform the same (or even slightly worse than e9). I just wrote, "e9 + I don't know" 😂 . I've been digging around in the source code and I found a couple interesting things but I don't quite know what to make of it. Like in `enc_ans_params.h`, there are ANS parameters for e9, stuff like histograms/hybridUint/lz77/clustering and so on. It's already mentioned in `encode_effort.md`: > "For the entropy coding (context clustering, lz77 search, hybriduint configuration): slower/more exhaustive search as effort goes up." I couldn't find anything worth mentioning for ANS VarDCT e10 specifically. I also found this in `common.h`: ``` kGlacier = 0, // Turns on FindBestQuantizationHQ loop. Equivalent to "guetzli" mode.``` I can't seem to find this special "FindBestQuantizationHQ" loop, just the regular "FindBestQuantization" loop for =>e8 and a special trigger which adds additional butteraugli iterations for e9. BUT, does it trigger for e10? I found this code block in `enc_adaptive_quantization.cc`. ``` int iters = kMaxButteraugliIters; if (cparams.speed_tier != SpeedTier::kTortoise) { iters = 2; } ``` I don't code in C++ but if I'm reading the logic correctly, if it's not e9, do 2 iterations, If it's __only__ e9, do 4 iterations due to `constexpr int kMaxButteraugliIters = 4;`. Doesn't that also exclude e10 (kGlacier)? Because that could explain why e10 is faster than e9, and usually gives worse results than e9. It's only doing 2 iterations instead of 4!😅 Did I find a bug? Or am I simply reading it wrong? 2) In benchmark_xl, what does QABPP mean?

Also it seems like streaming for lossy is disabled at e8. Lossless disables streaming at e10, am I correct? PS: I also found out that distances higher than 3 also disable streaming: https://discord.com/channels/794206087879852103/803645746661425173/1298409491989139458 Is this intentional?

<@&1092213695108628581>

is it possible to connect two jpeg xl images without transcoding, raw, along the border of the blocks. for example, 64x64 + 64x64. vardct is there a difference. is there a block to take from the center of the picture? or if the 64x64 image is the full image file

you could just use patches

Yes. JXL is a chunky format and it can be chunked up without transcoding

but this is true only for the maximum unit 256x256. I am sure of this. but I'm not at all sure that the blocks inside this unit are independent enough to be "moved" or changed (vardct)

Correct. Current libjxl respects 64x64 tiles too, but there is no guarantee that this is the case, the bitstream does allow having blocks that cross the 64x64 grid...

is the 64x64 grid actually do anything spec wise or is it just libjxl hasnt implemented it? like is the 64x64 grid mentioned in the spec

Spec only has 256x256 groups. Libjxl happens to do things also aligned to the 64x64 CfL grid, and only uses block sizes up to 64x64, but that's an encoder choice, not something that is guaranteed to always be like that.

Keep in mind that you can crop and stich several frames together after roughly throwing away the unneeded groups.

you can just use patches

I have no idea About

since jxl-rs was recently mentioned, another ping

the jpeg xl standard makes it possible to create images use combined modular and vardct blocks?

every frame is either vardct or modular, but a single image can consist of multiple frames that mix the two modes (and if those frames have zero duration, the decoder will do the frame blending so from the point of view of an outside application it's a single still image)

one caveat though is that the whole image has to be either XYB or not XYB, so you cannot mix lossless RGB with the usual lossy VarDCT which uses XYB.

(though you can do lossless XYB + lossy XYB or lossless RGB + lossy RGB/YCbCr)

Is XYB lossless?

won't it be a cycle of changing the display? as I understand it, it is necessary to make an alpha channel on all blocks that should be in others mode on next images...

not quite, it depends on what quantization you use to turn XYB into integers but it's not a transform that is designed to be reversible without needing a lot of extra bits. Though for doing lossless compression, arguably it could be a better color space to work in than sRGB...

So you can’t mix lossless and VarDCT

you don't need an alpha channel to make effective use of multiple layers. There's also a blend mode which just adds the values (so regions that are black will not make any changes), or you can have frames that are smaller than the image canvas which also leaves the other regions untouched.

animation layers 😔 crutch

If your source is RGB and you want to preserve the RGB values exactly, and you want to combine that with VarDCT, then you'll have to use YCbCr in the VarDCT parts.

<@794205442175402004> I made a mistake here. While (some) zimg RGB conversion is indeed broken (https://github.com/sekrit-twc/zimg/issues/213) , I didn't know that in ImageMagick you have to prepend the output file with `PNG48:` to actually produce a 16-bit PNG. Now things look different: ``` ❯ magick 100.png -depth 16 PNG48:100-rgb48.png ❯ ssimulacra2 100.png 100-rgb48.png 92.85603931 ❯ butteraugli 100.png 100-rgb48.png 0.0000911159 3-norm: 0.000028 ❯ ssimulacra2_rs image 100.png 100-rgb48.png Score: 100.00000000 ❯ magick compare -metric ssim 100.png 100-rgb48.png :null 1 ``` The ssimulacra2 score is concerningly low.

Scores above 90 are very high. You would expect an even higher score?

<@794205442175402004> was this video https://youtu.be/w7UDJUCMTng?si=EledMXrv1YE9T0l8 done with lossy or lossless compression?

lossy?

Lossy, of course. Lossless has no generation loss.

That behavior of jpeg was some ImageMagick bug though

what's the quality used? 80-90?

Lossless and ADDS the noise

Maybe you should campaign for xyb to become a standard encoding similar to srgb

Eventually it would make sense for image editors to use XYB as an internal working space instead of RGB or Lab. It would automatically give you nicer color pickers, gradients, operations like hue rotation or doing sharpening/denoising differently for luma than for chroma, etc. And then it would make sense to store images in XYB too during authoring workflows.

Technically jxl had no generation-loss advantages over regular JPEG, from what I understand... Wait, actually, no, there are some near-lossless encoding techniques possible with jxl that aren't possible with jpeg. But libjxl in lossy mode has either the same or slightly less generation loss than regular jpeg, and further improvements to generation resilience are easily within reach.

There are some inconveniences though, such as having to use float32, at least in the way XYB is currently defined. So that means more memory consumption.

In my view, default varDCT has some disadvantages over regular 4:4:4 JPEG when it comes to generation loss, since variable block sizes (and deblocking filters) mean that artifacts can propagate while in JPEG "what happens in the block stays in the block". These can of course be avoided by not using the features that cause this, if generation loss is expected to be a potential issue (i.e. when using lossy in an authoring workflow). In any case it does look like generation loss is mild compared to what you get with video codecs like webp/heic/avif, which have more aggressive deblocking and also directional prediction which causes additional artifact spreading in a south-east direction.

I thought there was a RCT integer variant of XYB as well... Besides, float32 working space is not uncommon when editing. And yeah, color pickers are typically so very bad... XYB and Lab seem the same to me honestly. I can't explain the difference. Different transfer curve maybe? Primaries?

The main difference between XYB and Lab in my view is that Lab was based on experiments where people compare large color disks (or rectangles) to find the thresholds of where two colors can still be distinguished or not, while XYB is meant to model pixel-level color perception. That makes a difference since for large color areas the S cones play a bigger role than for pixel-sized colors. Also Lab is pretty old and more recent experiments have found it to be not as perceptually uniform as it should, even for the task it is supposed to solve (large-area color similarity).

But it's a relatively subtle difference, for most practical purposes it doesn't matter that much if you use Lab, XYB, OKLab, CIECAM or whatever — it will in any case be a lot more perceptually relevant than RGB or HSV or YCbCr or any of those more simplistic color spaces that are close to RGB.

Does that mean new experimental data was used with pixel scale colors?

The experiments that lead to Butteraugli (and XYB) consisted of comparisons between small image patches, most of which with small pixel-level details in them. This is a different type of experiment than one where you compare large patches of solid color.

Basically the deltaE color difference you get with Lab (or later models like CIECAM) is about things like the color of a wall, or a car, or pants: there you want to be able to define say a Pantone color and have thresholds for how much difference can be tolerated. While the deltaE color difference you would get with XYB is more about whether you can see that a pixel changes color (which of course also depends on other things like contrast masking, so you cannot just use XYB-deltaE as a perceptual metric by itself).

Yes, I would expect a score of 100.

That error is roughly the same as the one that swscale and zimg introduce...

100 means mathematically lossless. How large is the error here, say if you do `compare -verbose -metric pae 100.png 100-rgb48.png null:` ?

0

that should be impossible. Then the images are identical? Or are you using an 8-bit build of imagemagick?

oh wait I see now what you did, the two images are actually supposed to be identical

except they're somehow not when ssimulacra2 gets them as float32 input

this is probably related to this: https://github.com/libjxl/libjxl/pull/3626

basically the issue is that in general `x * (1.0f / 255) != x * 257 * (1.0f / 65535)` even though mathematically that equality should hold since 1 / 255 = 257 / 65535

I wonder how rust handles this math

I guess it's just floating point shenanigan

it's a pretty annoying property of floating point arithmetic that you get these kind of issues... so even if 8-bit images can be converted exactly to 16-bit images (this is not true for all conversions to a higher bitdepth but it is true for this one), there is still a difference in the values you get when converting them to floats. The difference is only in the lsb of the floats, e.g. the 16-bit image has a float value of 0x3F7EFEFF where the 8-bit one has 0x3F7EFF00, and both will convert back to ints just fine, but they're still different...

threw in `c` for verification, rust doesn't like mixing floats, seems like rust likes to play nice, this kind of error won't show up when crunched back down to 8bit int probably?

I suppose you could make a `double` version of ssimulacra2 and butteraugli and then maybe it wouldn't be a problem

ultimately the problem is that uint samples have this implicit denominator of `2^n - 1` which is understandable but also very awkward

does `/ 65535.f` instead of `* (1.f / 65535)` improve things at all?

no, but doing `/ 255.f` does give more accurate results than doing `* (1.f / 255)`

but I don't think we want to use divisions instead of multiplies in those conversions...

what about doing the conversions in integer arithmetic? 😄

https://google.github.io/highway/en/master/_downloads/7570796eb9b1e4f3e025626833f22c51/instruction_matrix.pdf

actually if that is correct then multiplication and division is the same for f32?

in that case we should probably be using division instead of multiplying by the reciprocal

oh definitely not the same

11 latency / 5 rcpt for div, 4 latency / 0.5 rcpt for mul

oh right that table just says how many _instructions_ it is, but that doesn't mean anything about how fast that instruction is

so fp div (in simd) is 10 times as slow as multiplication, in terms of throughput?

yup

ssimulacra2 of 100.00 means lossless, is this what it's supposed to be?

yes

Are there any known compatibility issues with jpegli in combination with libtiff? For context, I'm currently investigating <https://github.com/libvips/build-win64-mxe/issues/69>.