JPEG XL

sRGB does not define an EOTF persay, only an oetf and a "reference display". note that there are very good historical reasons for the mismatch In encoding that made sense when CRTs where a thing, Because fundamentally sRGB is a great color space for CRTs. It's just retro now.

but... I DID encode everything in sRGB... I even downloaded the sRGB ICCv4 profile from color.org :(

I dunno what I did wrong :(

<#1288790874016190505> Dx

time for the nuclear option, strip the icc profile

they dont because you are supposed to calibrate

The `-r`, `-t` and `-p` switches are all NOOP/deprecated, see: <https://github.com/libvips/libvips/blob/v8.15.3/tools/vipsthumbnail.c#L220-L222> <https://github.com/libvips/libvips/blob/v8.15.3/tools/vipsthumbnail.c#L202-L204> <https://github.com/libvips/libvips/blob/v8.15.3/tools/vipsthumbnail.c#L217-L219>

... so with all deprecated options removed, this is what remains: ```console $ vipsthumbnail input.jpg --linear --import-profile srgb --size 1000x --export-profile srgb --intent perceptual -o 'tn_%s.jxl[lossless,effort=2,keep=icc]' ``` PR <https://github.com/libvips/libvips/pull/4175> improves these docs a bit.

thanks :) I didn't know that... there's no way to specify the resampling algorithm? that's kinda annoying, I wonder what the default is now

You could replace `-n` with `--intent` to be more specific too. Apparently the `-n` GOption is defined twice.

idk what the implications of that are

Seems like a bug

For reducing, `vipsthumbnail` defaults to lanczos3. For upsizing, it defaults to bicubic. You can use `vips resize` to pick another interpolator, see for example: <https://github.com/libvips/libvips/discussions/3083>.

Ah, indeed, that `-n` option is defined twice, hopefully it picks the first defined one instead of the deprecated option. I'll investigate.

ffprobe

Tried that yesterday, but it made no difference

No wait, I only tried cjxl, not djxl

there's a strong brightness shift between the original and the d1 here, that seems odd

this is just a visualization of what I got from doing benchmark_xl --input=102.png --codec=jxl:4,jxl:5,jxl:6,jxl:7,avif:q82:s3,avif:q82:s4,avif:q82:s5,avif:q82:s6,avif:q82:s7,avif:q82:s3:log2_cols=2:log2_rows=2,avif:q82:s4:log2_cols=2:log2_rows=2,avif:q82:s5:log2_cols=2:log2_rows=2,avif:q82:s6:log2_cols=2:log2_rows=2,avif:q82:s7:log2_cols=2:log2_rows=2,jpeg:q86,jpeg:q86:p1,webp:q91,jpeg:enc-jpegli:dec-jpegli:q86 --encode_reps 10 --decode_reps 30 --inner_threads 6 --num_threads 0

12 total threads, 18 tasks, 0 threads, 6 inner threads ``` 102.png Encoding kPixels Bytes BPP E MP/s D MP/s Max norm SSIMULACRA2 PSNR pnorm BPP*pnorm QABPP Bugs ----------------------------------------------------------------------------------------------------------------------------------------------------------------- jxl:4 5242 1128406 1.7218109 95.384 225.898 1.63101176 83.09969327 41.63 0.77717746 1.338152630488 2.808 0 jxl:5 5242 1080387 1.6485397 27.448 184.224 1.78047152 82.42861757 41.17 0.68999724 1.137487872987 2.935 0 jxl:6 5242 1082509 1.6517776 14.985 209.996 1.75711005 82.44657201 41.12 0.68662467 1.134151287924 2.902 0 jxl:7 5242 1081703 1.6505478 13.043 199.127 1.75711135 82.41697962 41.13 0.68662136 1.133301363062 2.900 0 avif:q82:s3 5242 1402720 2.1403809 0.620 36.119 1.57402488 84.26423263 44.20 0.70407007 1.506978093773 3.369 0 avif:q82:s4 5242 1394234 2.1274323 1.245 36.231 1.81495374 83.97075599 44.11 0.71672629 1.524786613935 3.861 0 avif:q82:s5 5242 1382879 2.1101059 1.862 37.632 1.82813460 83.70498052 44.00 0.74101071 1.563611059381 3.858 0 avif:q82:s6 5242 1339544 2.0439819 11.321 39.183 1.86672582 82.28488289 43.56 0.81168387 1.659067160046 3.816 0 avif:q82:s7 5242 1316406 2.0086761 15.067 38.712 1.85476755 82.28575566 43.35 0.81660202 1.640288993874 3.726 0 ```

``` avif:q82:s3:log2_cols=2:log2_rows=2 5242 1400331 2.1367355 0.723 100.492 1.67412488 84.21234765 44.17 0.70739257 1.511510836647 3.577 0 avif:q82:s4:log2_cols=2:log2_rows=2 5242 1392431 2.1246811 1.459 101.384 1.65663747 83.94402059 44.10 0.71619845 1.521693302133 3.520 0 avif:q82:s5:log2_cols=2:log2_rows=2 5242 1380575 2.1065903 2.345 111.030 1.67678352 83.65033903 43.98 0.74074627 1.560448888943 3.532 0 avif:q82:s6:log2_cols=2:log2_rows=2 5242 1340107 2.0448410 16.876 114.731 1.88479123 82.29229785 43.55 0.81171131 1.659820565445 3.854 0 avif:q82:s7:log2_cols=2:log2_rows=2 5242 1319072 2.0127441 22.527 106.265 1.85481300 82.27874745 43.33 0.81703295 1.644478286674 3.733 0 jpeg:q86 5242 1443376 2.2024170 120.301 318.321 2.27509252 82.06020682 40.04 0.85157149 1.875515526893 5.011 0 jpeg:q86:p1 5242 1431826 2.1847931 98.872 257.025 2.27509252 82.06020682 40.04 0.85157149 1.860507514888 4.971 0 webp:q91 5242 1392182 2.1243011 5.290 80.716 1.96516232 82.10737458 42.70 0.78443927 1.666385245030 4.175 0 jpeg:enc-jpegli:dec-jpegli:q86 5242 1037229 1.5826859 65.508 91.129 2.40549511 79.63608754 39.61 0.89569606 1.417605477857 3.807 0 Aggregate: 5242 1289129 1.9670547 9.607 100.423 1.85009735 82.73022769 42.51 0.76443812 1.503691618879 3.639 0 ```

I only included the effort settings for libjxl and libaom that I think make sense — s8/s9 are faster but give very poor results, s0/s1/s2 give slightly better results but are crazy slow, e3 is faster but results are too poor imo, e8+ is slightly better but not worth it

for lossless, the speeds look like this:

(truncated the bar for libjxl e1 with 6 threads since otherwise all the rest would be hard to see)

6x faster encoding than decoding, that's the power of specialised code paths

maybe there would be some place for runtime code generation

fundamentally, fast encoding is easier than fast decoding since the encoder knows the entire image from the beginning while the decoder has to reconstruct it, so in terms of data dependencies encoders have a major advantage

but yes, part of it is also specialized paths for lossless encoding of 8-16 bit images while the decoder is only specialized partially (it still always uses 32-bit sample values)

But to what?

whatever you want

ah yes, using display ICC...

?

that display ICC profile that you can load in the display color management settings in Windows...

pretty sure that was a thing...

yeah?

I don't think that should be needed

why not?

inconvenient

especially with multiple and removable displays

or if you want to switch display profiles

the profile is registered to the display

and you dont need to switch anything

there's no other way to get proper color without calibrating yourself

factory claibration can work very well

sure but they drift after a month

and that does the same thing

it's just icc inside the driver that the OS pulls out and does the same

driver?

if you have a display that has calibration, on windows, it pulls the ICC from WDF

wdf?

windows update downloads the driver

huh

and that's the profiling part of calibration

it has to be paired correctly with the calibrated part of the display

even more inconvenient

since you have to use the exact calibration setting on the display for the profile to mean anything

with manual calibration, you can skip calibration controls and calibrate/profile as-is

something you have to do for displays that dont have controls or an OSD

Afaik Windows has added an option in latest 24H2 Windows version to set display gamut, so stuff that's not color managed is then shown correctly if sRGB/BT.709.

So, seems like people with DCI-P3 display just see most things oversaturated and with wrong colors currently?

most modern IPS displays are around P3. without color management, it will basically scale everything to that gamut

but you still need to profile it with color management

calibration makes it so that the monitor behaves how you want it profiling lets your colour management system (whether it is embedded within apps, as on X11 or old Windows, or system-wide, like on macOS or new Windows) know how the monitor behaves (e.g. sRGB, Display-P3, other), after whatever calibration has been performed (including none if that’s how you roll)

calibration is neither necessary nor sufficient

profiling is arguably more important

If app does not have color management built-in - yes

and if there is no display profile yes

so if youre not using a mac pretty much

assumed so... weird how Windows only reacts now to that development that has been happening for years...

For high quality monitors it is not so obvious, but in general yes

it's also because the surface has to compete with color work stuff

And AFAIK there is a problem with wide gamut monitor calibration

only problem I know of is desktop isnt color managed, but it's solved with ACM

Not only this. I don't remember exactly but it is something about average calibrators are not capable of handling such gamut, or something like this. Read some years ago on displaycal site

there was older colorimeters not being able to handle high brightness but that's like 500 nits+

some old colorimeters had filters that didn’t perform so well with wide gamut monitors and needed a correction matrix; it’s much less true with current colorimeters

https://argyllcms.com/doc/WideGamutColmters.html

> There are three approaches to addressing this problem: > […] > The third approach is to make a colorimeter that has filters that are closer to the standard observer curves, reducing the calibration needed for the instrument, and making it less dependent on the exact type of display technology. The X-Rite i1 DisplayPro, Pantone ColorMunki Display and possibly the Spyder 4 may have such an improvement.

same concept but without the weird luma shift

at 1:1, eyes approximately 1500 pixels from the screen, both the e3 and e7 look pretty good, *but*, in the dark area on the left, the the quantization is very easy to notice with the compare_images tool, while the quantization in the light part in the middle is entirely imperceptible. The quantization is harder to spot in the dark area on the right owing to light adaptation from the surrounding area, but it's still actually visible. edit: sorry, meant e7 not e8. posted the wrong jxl, though the png above was made from the right file. double edit: meant *e*, not *d*, jeez im typoing a lot today.

<@532010383041363969> / <@179701849576833024> is there some adaptive quantization heuristic to bump up the quality in regions that are locally dark (i.e. large enough patch of darkness to allow local adaptation)? Say if the surrounding 3x3 DC values for Y are all below some threshold...

If not, then I think we should add something like that.

not as far as I know but only <@532010383041363969> truly knows

there are a few things to consider, like e8 and e9 may counter great heuristics if butteraugli disagrees with the heuristics, IIRC butteraugli iters can make quantization 40 % worse than the original heuristics decided one way forward would be to start by creating a test corpus with 3-10 images where the phenomenon is visible

also, there is the annoying thing that other codecs don't bother at all with the blacks

if we want to remain roughly competitive in other areas -- if we spend our bit budgets in black areas -- then those users/reviewers who have their monitors dark or have bright viewing environments are just going to call AVIF the better codec

last, butteraugli doesn't compensate for the opening of the pupil -- if there are some bright areas or not, it can modulate the pupil to give 4x more light which makes the blacks way more visible

if someone creates an issue and uploads three images, I will spend 222 minutes trying to fix or mitigate this

I will look for two more test images

222 !

I guess the issue is mostly if there's a large area (or even the whole image) that is dark but still has some detail. If the area is large enough (so you can look at it without getting 'blinded' by neighboring bright areas), brightness adaptation of the HVS does become relevant. This is also what happens if people pixel peep and zoom in on the dark parts (to the point that they no longer have any bright parts on screen).

I think the main issue at hand is that the color profile I was using was causing libjxl to choke, for some reason.

Maybe if that issue is fixed, there won't be any problem with retaining detail in dark shadows

This issue seems to resurface a lot, there's lots of duplicated bug reports that seem to demonstrate the same issue.

It's hard to determine what else could be wrong until that issue at least is fixed first.

Maybe it's a combination of issues.

first, turning srgb images into linear int...

Maybe if that issue is (actually) fixed (for real this time), the other issues will go away.

does it make any difference if you build libjxl with lcms2 instead of skcms?

Good idea, I'll give it a shot.

Ugh, why does 0.12 have a different soname...

latest libjxl git still craps the bed when built with lcms2 on this image.

Here's what they look like when decoded as so: `djxl --color_space=sRGB d1_wood.{jxl,png}` `djxl --color_space=sRGB source_wood.{jxl,png}` Notice how `d1_` is noticeably darker than `source` despite both being decoded to the same color space and d1 being created by `cjxl source.jxl d1.jxl`

Something is obviously very wrong here.

A, very good. This means my theory was correct.

For some reason, firefox shows source_wood.png as being WAY brighter than gwenview...

The second set looks identical, though?

Huh. Last I checked, Firefox converted everything tagged to sRGB, including transfer function. Maybe they changed that to gamma 2.2, then this would make sense.

im currently downloading a bunch of images from Apple Support website to see how smaller they could be in jxl

there seems to be something wrong with the ICC profile in that original png

maybe it's one of those incorrect ones that cjxl keeps mentioning

I'm pretty sure this is the actual original file.

Pashi told me that they re-encoded it using the ICCv4 sRGB profile from color.org

why wouldn't you just use the cICP sRGB?

why add an sRGB ICC?

sometimes ICC profiles will be used in the wild so related bugs do need to be fixed

I hadn't seen such a big icc profile yet that is supposed to be sRGB

something weird happens with it, e.g. imagemagick refuses to extract it from that png for some reason

are `A2B0` and `B2A0` meant to be 30 kB?

When I had a lossy JXL with that profile attached, djxl decoded it to a PNG with a linear light profile 😄

also cjxl doesn't seem to be able to correctly parse it, it ends up storing the entire icc profile which is something it wouldn't do if it would recognize that it's just sRGB

I am going to file an issue for that

how does it recognise colour profiles?

maybe it's slightly wrong sRGB

in lossy mode, it looks at the primaries and transfer curve and it allows for some wiggle room since not all profiles round stuff the same way, and if it's close enough to something that can be done as an CICP-style enum, it does that

When viewing the two in mpv, they do look identical, save for the d1 image being significantly blurred way too much for d1

``` % cjxl -x icc_pathname=sRGB_v4_ICC_preference.icc -d 1 -e 3 stainedwood.jpg.ppm{,.icc.d1e3.jxl} % jxlinfo stainedwood.jpg.ppm.icc.d1e3.jxl JPEG XL image, 2560x1600, lossy, 8-bit RGB Color space: 60960-byte ICC profile, CMM type: "", color space: "RGB ", rendering intent: 0 % umbrielpng -v stainedwood.jpg.ppm.icc.d1e3.jxl.png PNG signature found: stainedwood.jpg.ppm.icc.d1e3.jxl.png Chunk: IHDR, Size: 25, Offset: 8, CRC32: 3ace8c97 Chunk: gAMA, Size: 16, Offset: 33, CRC32: 31e8965f Chunk: iCCP, Size: 345, Offset: 49, CRC32: 6d1c0317 Chunk: cHRM, Size: 44, Offset: 394, CRC32: 1b1c9a7f Chunk: cICP, Size: 16, Offset: 438, CRC32: 9aa2f9d9 Chunk: IDAT, Size: 8204, Offset: 454, CRC32: cc4f90af Chunk: 215 more IDAT chunks Chunk: IEND, Size: 12, Offset: 1767824, CRC32: ae426082 Size: 2560x1600, Color: 8-bit RGB gAMA: 100000 ICC Profile Length: 536 cHRM: wp: 31270, 32900, r: 64000, 33001, g: 30000, 60000, b: 15000, 6000 cICP: BT709, LinearLight, RGB, Full ```

(the `-x icc_pathname=` option doesn't appear to work with jpeg input, so I converted to ppm first)

mpv seems to display both images with the exact same transfer function. But I'm not sure if it's because the color profile support IS broken or NOT broken.

yeah so for some reason cjxl does not recognize the transfer curve to be sRGB, and instead signals it to be linear in its enum colorencoding while putting the whole icc profile in the bitstream. Then when viewing or decoding, it cannot convert the result to the ICC profile space and it converts it to the enum colorencoding, which is a pretty bad idea (8-bit linear is not pretty).

for some reason if I do `convert source_wood.png profile.icc`, I get ``` convert: no color profile is available `profile.icc' @ error/meta.c/WriteMETAImage/2397. ``` while normally that should just extract the icc profile from the png

Is the ICC stored correctly within the PNG?

I can extract it with exiftool, and Apple's ColorSync opens that icc profile without complaining

but _something_ is not happy with it, clearly

It’s odd it s saying it doesn’t exist instead of saying it’s corrupt

Does PNG store the ICC verbatim? Or does it do like JXL and compress / modify it

yeah but I've seen it say that before, I think it's what happens if libpng doesn't like the profile (it doesn't return it, so imagemagick thinks there is none)

I think PNG uses DEFLATE for icc profiles, like it does for most things

lotsa blur

At least it's not as bad as discord webp :)

Does it do any ICC specific processing or just generic compression. Otherwise why would libpng need to verify

it doesn't need to, but it just likes to be pedantic

I'm very confused by this stuff: https://color.org/ICC_White_Paper_26_Using_the_V4_sRGB_ICC_profile.pdf

I don't get why they feel the need to come up with a 60 KB profile for sRGB that supposedly provides "more pleasing results for most images"

I for one am not pleased by the results 🙂

How big is a normal sRGB profile?

So... It's *not* sRGB

500 bytes or so

as far as I understand, it mainly has to do with the absolute and perceptual rendering intents

also, > 1. The ICC-absolute and media-relative colorimetric rendering intent transforms are not black point scaled.

I think it is, just one that is bundled with various suggested gamut mappings to convert to and from sRGB in ways that are not just the normal mappings.

But this is not the kind of thing you want to tag an image with

Yeah, reading further it seems like it was made to replace multiple v2 icc files with a single v4

uh, so we write a linear Rec. 709 CICP for a jxl with a colour space encoded as an ICC profile?

yeah

doesn’t sound good at all

issue filed https://github.com/libjxl/libjxl/issues/3874

it seems we didn’t use to add a cICP when decoding a jxl like that, up until a commit titled “Always add cICP chucks to djxl generated pngs if possible.”

which I suppose makes sense

(and suggests I did my bisection correctly)

You're not just adding such a cICP tag, the other color tags and the emitted ICC profile are also for linear RGB.

And the actual image data is of course in linear RGB.

So convert from linear RGB to sRGB and it'll look correctish, of course with visible artifacts from the fact that 8 bits isn't enough precision for linear RGB.

right, the cICP thing is maybe just another manifestation of the same underlying problem

still, the fact seems to be that before that commit, at least as far as cICP was concerned, we “knew” that the image wasn’t really linear sRGB

ah

so perhaps this points to what might be wrong about the logic elsewhere as well

hm, it seems to be that before, we only added cICP `if (ppf.icc.empty())`

now we do it even if it isn’t, which we probably shouldn’t

and so it’s probably something similar somewhere about using `ppf.color_encoding` despite it having an ICC profile (which points to it _not_ having a sensible enum representation)

decode_oneshot also produces an ICC profile that is synthesized from the color_encoding rather than the big one

correction: it’s not having an ICC that points to that (as it could be a fallback ICC we generate), but (probably) `primary_color_representation == kIccIsPrimary`

correction (bis): somehow, when we decode that file, the ppf has `primary_color_representation == kColorEncodingIsPrimary`

which seems like a bug

Maybe we should add a conformance test case, it's a bit surprising to me that conformance testing didn't catch this...

do we run all conformance tests at high precision? that would explain it

it’s fine with 16 bits

Oh, right. Yeah they output float32 even

But there should also be a test in there checking the ICC profile reconstruction, so that would have caught it too. If there is an example that triggers the buggy behavior, which may not be the case...

has anyone tested with a more typical ICC profile?

I only tested with that odd profile from color.org

In the typical case, the profile matches an enum space and doesn't get stored explicitly when doing lossy.

https://github.com/libjxl/libjxl/blob/main/lib/extras/dec/color_description.cc Would be cool if someone added some shortcuts for "adobe" opRGB

Since it's a very common and popular standard

And it could be called "opRGB (compatible with Adobe RGB)"

<@263300458888691714> This site seems pretty good for forming a pixel art corpus. Can't download directly in 1:1 scale, but they do list the original sizes for every image so scaling it back down is simple. https://www.pixilart.com/

nice. I like these for 1:1 in the wild: <https://lospec.com/> <https://pixeljoint.com/>

Here is the committee draft for AIC-3 (subjective quality assessment). If you have any comments, feel free to send them to me.

it looks like much of this is just a formalization of already established methodology, with the exception that this is really geared toward high fidelity... though I don't necessarily want to wade through all the other standards to see the specifics

For larger images, distances of 3 or higher __doubles__ encoding times and __triples__ ram usage 💀 ``` wintime64 -- cjxl b.png b.jxl -d 2.9 --disable_output Compressed to 1390.6 kB (0.596 bpp). 5760 x 3240, 8.856 MP/s [8.86, 8.86], , 1 reps, 12 threads. PageFaultCount: 532016 PeakWorkingSetSize: 368.8 MiB QuotaPeakPagedPoolUsage: 33.05 KiB QuotaPeakNonPagedPoolUsage: 19.52 KiB PeakPagefileUsage: 489.1 MiB Creation time 2024/10/22 15:09:29.969 Exit time 2024/10/22 15:09:32.255 Wall time: 00:00:02.285 (2.29 seconds) User time: 00:00:00.984 (0.98 seconds) Kernel time: 00:00:14.531 (14.53 seconds)`````` wintime64 -- cjxl b.png b.jxl -d 3 --disable_output Compressed to 1370.3 kB (0.587 bpp). 5760 x 3240, 4.008 MP/s [4.01, 4.01], , 1 reps, 12 threads. PageFaultCount: 1360046 PeakWorkingSetSize: 1.584 GiB QuotaPeakPagedPoolUsage: 33.05 KiB QuotaPeakNonPagedPoolUsage: 45.02 KiB PeakPagefileUsage: 1.888 GiB Creation time 2024/10/22 15:10:14.812 Exit time 2024/10/22 15:10:19.669 Wall time: 00:00:04.856 (4.86 seconds) User time: 00:00:01.546 (1.55 seconds) Kernel time: 00:00:18.953 (18.95 seconds)```__To avoid this use streaming:__ ``` wintime64 -- cjxl b.png b.jxl -d 3 --disable_output --streaming_input Compressed to 1351.2 kB (0.579 bpp). 5760 x 3240, 8.874 MP/s [8.87, 8.87], , 1 reps, 12 threads. PageFaultCount: 531897 PeakWorkingSetSize: 368.2 MiB QuotaPeakPagedPoolUsage: 35.89 KiB QuotaPeakNonPagedPoolUsage: 19.52 KiB PeakPagefileUsage: 488.3 MiB Creation time 2024/10/22 15:11:10.567 Exit time 2024/10/22 15:11:12.851 Wall time: 0 days, 00:00:02.283 (2.28 seconds) User time: 0 days, 00:00:00.609 (0.61 seconds) Kernel time: 0 days, 00:00:14.968 (14.97 seconds)```

This doesn't happen on my 1080 screenshots btw

Implicit streaming until distance 3 at effort 7?

I hope this isn't a bug...

Strange that streaming does anything, since PNG streaming isn't supported

v0.11 added chunked encoding support ~~for non-interlaced pngs~~, the warning the CLI throws out is an outdated lie from 0.10

Scratch that, chunked encoding supports pngs (interlaced or not) and jpeg inputs (with -j 0). Probably more input types as well.

Chunked encoding and streamed input are separate things, with streamed adding an even higher density/memory tradeoff

are they? I forgot the difference

Got an image that's bigger lossless at effort 3 than 2 or 1 (e2 and e3 are bigger than the original png too). ```JPEG XL encoder v0.12.0 c6355600 [AVX2,SSE4,SSE2] Encoding [Modular, lossless, effort: 1] Compressed to 345.3 kB including container (0.005 bpp). 22800 x 22500, 433.904 MP/s [433.90, 433.90], , 1 reps, 16 threads. Encoding [Modular, lossless, effort: 2] Compressed to 830.9 kB including container (0.013 bpp). Encoding [Modular, lossless, effort: 3] Compressed to 906.9 kB including container (0.014 bpp). Encoding [Modular, lossless, effort: 4] Compressed to 341.7 kB including container (0.005 bpp). ```

It happens sometimes

The methods between effort 1 and 4 are quite different so there are no guarantees that higher effort is smaller

Perhaps we should make higher effort slower and make it try lower efforts too

Either way, nothing will compare to the speed of e1, so at minimum it could be run

e1 is unbeatable for screenshots etc. much faster than PNG too

And still delivering smaller filesize

yeah, e1 is fantastic when you need speed

honestly it's fine as it is, as long as people understand it's a possibility

it is a bit unintuitive that e3 performs well only on photo-like stuff. it is not really safe to use in general if you are processing non-photo content, where e2 (and even e1) can be faster and denser. that said, e3 is really great for photo. if there is a smart, fast way for the encoder to decide between e3 or e2 strategies, that would elevate the utility and mitigate unfortunate situations.

photo film 3d digital_painting vs not

retime with optimized build

seems updated and new sets <:FeelsReadingMan:808827102278451241> https://github.com/WangXuan95/Image-Compression-Benchmark

https://github.com/WangXuan95/NBLI

> It can also beat the SOTA JPEG-XL SOTA?

State of the Art?

and beat it *how*? If it's just in pure compression efficiency, that's not terribly interesting unless it's at least as fast

> JPEG-XL v0.9.0 cjxl.exe in.pnm out.jxl -q 100 -e 8 --quiet --num_threads=0 A shame, but still rather good results for a single thread

Means it's not using chunked encoding

yeah, it's single-threaded and tuned only for photos, but anyway, photos may compress much faster at pretty high efficiency (like jxl e6-8), as HALIC has also shown

Hardcoding stuff for single-threaded 8-bit RGB does come with some benefits in speed and compression, but I don't think it is an acceptable limitation in 2024 to handle only that.

> Most of them are Windows executable files (.EXE). I put them in the codec folder. ~~Wait a minute, I hope they aren't using the 0.9 binaries provided on github. They are 2-3 times slower than other releases. MSVC was misconfigured or buggy iirc ~~ nvm, I forgot the performance regression was just affected VarDCT.

Is it open source? Someone should look at how it works

https://tenor.com/view/evil-laugh-rub-hands-patrick-star-evil-plan-gif-15072939

Maybe we can port over some algorithms for 8 bit images

https://github.com/WangXuan95/NBLI/blob/main/src/NBLI/AdvancedPredictor.h it seems to have a really intricate predictor model

This is the main file https://github.com/WangXuan95/NBLI/blob/main/src/NBLI/NBLIcodec.h

They list what they took inspiration at https://github.com/WangXuan95/NBLI/blob/main/src/NBLI/NBLIcodec.h

` - And of course, Claude E. Shannon.` Nice

8-bit probably because it's pretty common for images and it's easier to get started same for single threading, easier for further experiments and improvements, while codec can still have big changes, but it's not such big problem to make multithreading later, again HALIC as an example, which later got support for multithreading and even if considering hardcoded 8-bit and and not the most optimal compilers for libjxl, it's sometimes 10x speed difference for the same efficiency

NBLI = small nb = small awesome (or cow v*lv*)

if it eventually handles MT and > 8-bit then I'll be interested

Has anybody done some comparison of packJPG and precomp vs jxl transcode of jpg? From my quick tests it seems packJPG is quite a bit more efficient for archival of jpg

I've only tested on comic books so that might be different to photos

``` The Last Human 000-032.cbz │ │ 57897 KB The Last Human 000-032.cbz.pcf │ │ 46034 KB The Last Human 000-032.d0.jxl.cbz │ │ 51485 KB The Last Human 000-032.pjg.7z │ │ 45375 KB ```

pcf = precomp, pjg.7z = packJPG

Big thing with JXL is it needs to be 1) still an image not just bytes, 2) really fast to decode

The others can probably do more fancy things like DC prediction from AC which JXL can’t

Also try setting higher effort on JXL

(Yes lossless transcode uses effort as well)

Not necessarily a fair comparison. packJPG & dropbox's lepton are just that, archival formats. They don't have to worry about being an actual image format with all the responsibilities that come with it.

Like actually being decoded and displayed for example

Yeah I'm just testing out stuff

Don't mean to cast shade on jxl 😉

Try seeing how well lossless AVIF compares lol

Speaking of jpeg re-packers, the best one so far was lepton I think

actually, jxl beat lepton in a few edge cases I remember

I would be curious if lossless transcoding of jpg comic books to multi page jxl could make use of some similarities

I assume no such encoder exists yet, and it would be a pretty rare use case

Ah cool... so there is packJPG, brunsli and lepton?

Those are the main ones I am aware of

Does lossless JPEG even support multi frame images)

What about just jpegultrascan to optimise the JPEGs

No, that would be comic book jpgs in a zip (or rar) file, converted into a single jxl multi page

So not being able to restore the original JPEGs?

Well (I imagine) theoretically you could write an encoder that way, to restore the original zip file bitexact from a multi page jxl

`-e 10 -I 100 -E 3 --brotli_effort=11` can't remember exactly, `-E 3` probably does nothing in newer versions and yeah, jxl is better because it's normal images with all the features, not just an archive

But really there are only two cases: 1. archival to restore zip archives bitexact 2. compress comic book archive visually lossless

And for 1 it's better to have something like precomp

I would like to write a comic book reader app though that can just open precomp archives

yes, 0.10 and after `-E 3` does nothing. Also effort 9 is better than effort 10. It's a known issue

https://github.com/libjxl/libjxl/issues/3888#issuecomment-2450725691

then `-e 10 -I 100 --brotli_effort=11` or `-e 9 -I 100 --brotli_effort=11` and if needed + `--allow_jpeg_reconstruction=0` and + `-x strip=exif` because after tests on some jpegs, `-e 10` can still be better, so `-e 9` is not always best and `-e 11` is not implemented for jpegs, as far as I know

Yeah effort 11 isn't a thing yet for lossless jpeg reconstruction. I find that effort 9 is usually smaller on average but it doesn't hurt to try both

But --brotli_effort=11 should do something? What is the default? (never mind, default is 9)

yeah, if there's any metadata

What is Ben 10 effort

Brotli effort

no we are talking about regular encoding effort: `-e 9`. Brotli effort should always be maxed out if you want maximum compression: `brotli_effort 11`

alias cjxl=“cjxl —brotli_effort 11”

`--brotli_effort=11` better compression for exif and such

So if I strip metadata anyway it shouldn't make a difference?

Ah so like xz -1 vs xz -9e ?

Why strip metadata

probably, but if you want to archive stuff, don't strip metadata

ya know, the whole point of bit-exact reconstruction is preservation of the originals and all that

I'm sort of trying to figure out what I want to do lol. If I want to archive stuff, I'd want to be able to restore the containing zip files bitexact (e.g. torrent). So that means precomp. If I cant then I could just as well strip it

sometimes it's useless data for just viewing images, but if it's needed, then yes, better to keep it

_maximum compression for some threshold of allowed compute_

comparison from a couple years ago https://discord.com/channels/794206087879852103/803645746661425173/937627806408523777

Thanks! It seems web optimized jpg is already less compressible than most "normal" jpgs. (also good to know there isn't much difference with packjpg and lepton)

in this case, not exactly "web optimized". most images originated from the web, but all subsequently passed through significant local jpeg optimizations.

There is certainly a difference between a camera jpeg which is typically sequential with fixed Huffman tables, and a typical web jpeg, like the output of mozjpeg, which is progressive with optimized Huffman tables.

Is it ever worth it to downscale for "quality per filesize"? Like if you have a 4k image, does it ever make sense to scale down to 1080p or 720p instead of using a worse distance / quality?

Like if you want to match a filesize of say 100kb for an image

it can make sense at some threshold and libjxl already implements such judgement

Are there some benchmarks? or a term I can use to search for?

Or is there a formula, like if I half the resolution of a 4k image, the butteraugli distance increases by 4 or something

Sorry for the weird questions haha

https://github.com/libjxl/libjxl/blob/adeeccc415f7c18e6a1732d4c9d6d905ed8e2f9b/lib/jxl/enc_frame.cc#L92

That's what libjxl currently does

At d20 it instead does d6 on a 2x downscaled image

https://cdn.discordapp.com/attachments/1156999429782515723/1302549045356527666/lurk.png

https://cdn.discordapp.com/attachments/1156999429782515723/1302570142655582238/lurk_googly.png

two instances of `cjxl` failing horribly at SSIMU2 80

How large is the lossless version? Is lossy really useful for an image like this?

param: `cjxl -j 0 -d 5 -e 7 -p --progressive_dc 1`

419.5 KiB

currently both WebP and AVIF are more viable

Their lossless version is smaller or what do you mean?

I meat lossy version at SSIMU2 score 80, with WebP at 24.7% and AVIF at 9.95%. JXL is at 37.2%

the lossless versions in WebP and JPEG XL are on-par, lossless AVIF is still way worse

Ah. Yes, webp and avif have lossy encoders that are better at this kind of image. Libjxl's lossy encoder is suitable for photographic images but for non-photo it isn't really using the available coding tools in a very good way...

You could give `-m 1` a shot, should result in a different score at least

Are those percentages of the original filesize? Just to confirm

yup

Just accidentally clicked the channel and remembered I never did my testing I got SSIMU2 80 at 26% with lossy modular `-m1 -d 3`, a score of 90 at 36% with delta palette`cjxl --lossy-palette --palette=0 -m` (I can only get it working in version 0.6), but the smoothing of WebP and AVIF will win out for smooth images until JXL can take advantage of its splines

Is there an approximate buttergaugli distance / quality where avif becomes better (smaller) than jpegXL? Like is that distance at d=2 or more like d=10?

If I recall d3 is the highest recommended, although you can probably go to 5 maybe with resampling depending on the image (AVIF does better with lineart and smooth images)

Thanks. Lineart and smooth gradients seems to apply for comics and webtoons.

Oh, you could also try lossy modular like I mentioned above. Tends to do well with gradients, but adds some color fringing at edges

Is jxl expected to surpass avif at low bitrate with additional development?

AV1 is especially good at low bitrate, it becomes less detailed but creates no ugly artifacts so just looks better. Is that a fundamental advantage of avif vs jxl?

In theory yes. The techniques/algorithms used in jxl are more advanced than avif. The current encoder isn't tuned for low bit rate content, especially non-photographic media.

Also some advanced encoder tools are not being used (like splines) or could be improved (like patches). Patches and splines can really help with encoding artwork, lineart, etc.

Many people have mentioned this before on this server but Jpeg XL has *HUGE* potential.

We really are just at the tip of the iceberg honestly.

Awesome, thanks

Lineart, but with actual lines instead of blurring and DCT artifacts :P

I will say that AVIF's de-ringing is pretty clever

I'm curious how JXL and AVIF are going to compare when JXL is eventually well optimized/tuned for lower bitrates as well

In the end, it depends how low you go

More like we are still travelling to the iceberg

are there any possibility that improved patches and/or splines will come to JXL encoders soon?

Soon, I doubt. I am not aware of anyone currently working on either of those.

Patches, maybe. It's been improved before, but the code was accidentally deleted before it could be submitted https://discord.com/channels/794206087879852103/803645746661425173/1280926171835662336 (Ignore my message about decode time underneath, it's unrelated) https://discord.com/channels/794206087879852103/803645746661425173/1247714711777054730

<@386612331288723469> ^

So currently patches or modular tools are not used in VarDC mode right? What would be great is if patches worked even if they are some distance away, not just when they are perfect. Like with text with slight compression

But I guess this is basically the same hard problem as OCR

Patches are used with VarDCT, but only at either effort 10 or images under 2048 in both dimensions. I *think* it allows slight differences too, but I could be wrong on that

iirc lossy patches are used at e7 or e8, and they are using in a very simple manner, so it could definitely improved

you are indeed correct

jxl bitstream is capable of very efficient compression at low bit rates but the current encoder was designed and optimized for high fidelity and high bitrates. A clever developer could make a low bitrate optimized encoder that is specifically designed for low bitrate and it could easily surpass avif then, but there is no motivation in most developers to write an encoder that is designed to operate at low bitrates that force painful compromises to be made to image fidelity

Even though extreme compression ratios is exciting, it's also potentially dangerous and uncomfortable to design an encoder that can significantly change what's in the image...

But personally I expect that to change when people try more clever tricks.

Eventually someone is going to want to try some of the very clever tricks that are possible in jxl but still unused

Like spline encoding, layered decomposition, noise replacement, jxl delta palette tricks, perceptual noise shaping, and even near-lossless compression filters and techniques

All of those have enormous potential to decrease bitrate while maintaining visual fidelity. None of them are available in current jxl encoders

So basically it's a really future proof and outstandingly clever bitstream format but current jxl encoders like libjxl are still in the neolithic era

But I expect high fidelity/high speed encoding modes not to use most of those techniques

(Except noise shaping. That's something useful at all quality and speed levels.)

Only low bitrate and low speed encoding would bother using most of those techniques that would significantly slow down encode and decode

Maybe some can be accelerated in hardware someday...

Noise replacement and perceptual spectral masking are probably the low hanging fruit right now that I would expect libjxl to improve upon next

The others I would not expect any time soon unless an enthusiast/hobbyist tries making a new super encoder

> but there is no motivation in most developers to write an encoder that is designed to operate at low bitrates that force painful compromises to be made to image fidelity I disagree, there is still room to improve low-bitrate performance without comprising image fidelity. Look at svt-av1-psy, the improvements are downright impressive. They took a format we thought was near it's limit when it came to image fidelity and they completely flipped that on it's head. It's at a point where avif is now *beating* Jpeg XL at even medium-high quality encodes. Basically, what I'm saying is that we can have our cake and eat it too. Since Jpeg XL is still in it's infancy, there are plenty of improvements to be made which are __straight upgrades, no compromises necessary__. The more I think about it, <@386612331288723469> was right, we *are* still traveling to the iceberg.

But I don't have a crystal ball

I agree, but the problem is no one has been brave enough to try bold techniques and optimizations for the low bitrate, low speed (like avif) case.

Like I said libjxl is still only using stone age techniques when the jxl bitstream format has space age tools sitting on the shelf untouched.

Maybe someone from the svtav1psy team could "switch teams" ;) I think jxl as a format has a lot more long term viability and potential for obvious reasons.

tl;dr it's not the format, it's the encoder. jxl is a better designed and more future proof standard in my opinion but the current libjxl encoder is very conservative and limited

Brave enough? I don't think bravery is the issue. It's just a matter of time. The sheer number of potential pit falls present in writing a complex encoder is insane. Not to mention the large amounts of time spent on testing to catch regressions and minimize edge cases. It's quite a task for a small dev group, let alone one person.

The problem is libjxl is complex in terms of dependencies and c++ spaghetti but it's very conservative and limited when it comes to functionality

It's not easy, but in a way writing an encoder doesn't have to be that hard either. After all, you don't have to use all coding tools available. You could start with just a JPEG encoder (only dct8x8) and then gradually add more stuff to it, for example.

Yep! That's a great way to start, too. Bootstrapping, you could say

Nevertheless I agree, jxl is MUCH more future proof. And I think investing the time and effort will payoff a thousand fold.

Get the basics down right first

That's the best way to start any project

Then incremental improvements can always be added later as long as the basic foundation is solid and not monstrously complex

Reminds me of the development of rav1e, they started off simple and gradually added more codec techniques overtime. Laying a solid foundation helped rav1e to become the best AV1 codec of its time.

Nowadays it's quite stagnant, I don't know why it's not being developed anymore but I keep hearing about it's potential. It's quite sad. 😔

It's how it's done

It's slow, that's why

It's good quality though.

I thought there was no more funding or something like that.

Just very slow and people need to do things that put food on the table

Yeah probably

Wasn't it mozilla or xiph?

Xiph

Mozilla doesn't spend money on anything worthwhile anymore, they have a strict policy

Mozilla used to be xiph's main funding source

Mozilla only spends money on milking the brand to death and being vultures and vampires

Not engineering anymore

So I've heard

How similar are the coding tools in AV1/AVIF to JXL? I’m not that familiar with AV1

https://discord.com/channels/794206087879852103/794206170445119489/1285283645988409424 and https://discord.com/channels/794206087879852103/794206170445119489/1248537640316174337

Not very.

Basically avif has a lot of very strong pre/postprocessing filters built in for smoothing out macroblock artifacts and other DCT artifacts. JXL tries to avoid those artifacts in the first place using clever spec design, and also has multiple diverse tools that are completely different from DCT, for encoding different kinds of content, and they can be layered and combined in creative ways.

I think jxl is actually capable of a lot more unique and diverse tricks than a boring DCT video intra codec.

so AVIF is bascially VarDCT * 0.5 + Restoration filters * 9001

It has more potential tools in the toolbelt. Like I said, space age tools. Libjxl is currently only playing with sticks and stones.

Someone just needs to make an encoder that plays around with the more advanced tools. Just like how webp has near-lossless mode or gif has gifski, by comparison libjxl is super conservative and plain

welp, time to fail at writing an encoder! 😄

The super fast fjxl mode is cool though

That was very impressive

Modular mode is actually very impressive too

Including squeeze

lossy modular could do some cool stuff

So much potential there! It has not even been tuned and optimized and it looks as good or better than default DCT mode on average! Wow! That's insane!

Maybe it should be the new default? Screw DCT!

add them together \:D

oh wait, that's called JPEG XL

some kind of encoder heuristic that tells which one would be better

That would be even better but also twice as slow if every image is 2 layers

im thinking like if encoding a screenshot, encoding it in Modular but if it detects a picture, encode that using VarDCT

because lossless Modular can be smaller than lossy VarDCT in some types of images

Or an image processing kernel that divides the image into a dct "texture" layer and modular "cartoon" layer

this sounds even cooler than AV1 palette blocks

Then encodes them in parallel

now to find someone who can write the code to do this...

and who has time to

and wants to

and has the financial incentive to ..?

Av1 is a boring dct intra codec made with ancient technology when 4:2:0 was still considered cool, sorry!

avif is effectively webp++ but without the good lossless mode

It really doesn’t belong in the same league as jxl!

Unfortunately you should not judge jxl the format based on the current state of libjxl the encoder

jpeg xl is next-gen JPEG plus next-gen PNG, squared

The design of the bitstream is super clever

still sad that xyb is global instead of per-frame \;(

=[

Like Jon says, if jpeg was alien technology from the future (as it has been called before), jxl needs to be the "next gen alien technology from the future."

And it really looks like that's how the bitstream format was thoughtfully designed

that's why chrome codec team didn't want to add jpeg xl, they were scared of aliens!

It was designed with maximum future potential in mind

With possible untapped improvements looooong into the future

The future proof design really really stands out compared to all other formats

Similar to how the original jpeg is still being improved upon today

It takes everything good about JPEG, learns from it and gives it a few more well known image coding tools to play with and combine together.

While trying to avoid long established mistakes and ambiguities of the past

-# *join the jpeg xl revolution*

<:WhiteFoxSalute:689481667412623385> <a:Military:821038589571235850>

Thanks that was what I was looking for

https://github.com/libjxl/libjxl/pull/1395

Made a very dodgy comparison of jpegli for high bitdepth use https://github.com/Beep6581/RawTherapee/issues/7125#issuecomment-2538513015

Jpegli is definitely better in color channels, but blurs luma 😦

The quality was 'only' set to 96 there with 4:2:0 subsampling and AQ enabled, so it's expected that the fine details would be lost regardless

Piss... I should've re-encoded the JXL at the same quality, that was at -d 0.3

Ah well, the main point is the decoder difference, not the quality itself

honestly its soo cool how well jpegli performs

Added a quality 100 4:4:4 comparison too

Working on lossy VarDCT right now. Using a 6.25 MPx, 25 MPx, and 100 MPx image.

I'd probably do the standard monitor resolutions, 1080, 4K, 8K and then an extreme like 100 MP

yeah, I'm working with an image with similar dimensions to 4K right now.

Messy messy graph. Blue = Memory. Green = Time

gonna switch to excel, google sheets sucks

maybe they should be on seperate graphs, its kind of hard to read

Oh yeah I should mention that I'm using a nightly build, v0.12.0 2368781

there any comparisons between fpng, fpnge, and <:JXL:805850130203934781>?

ah, it looks like there are some old benchmarks on it

fjxl is the fastest 🎉

isn't fjxl rolled into the main encoder now? i.e. its code is used in the lower efforts?

for `-d 0 -e 1`, yep

4371x5828 image, on Windows

100 MP will take ages...

But I won't give up

Add -e 11 coward

Is that spike in decode time for -e 3 because effort 3 is the one that exclusively uses the self-correcting predictor?

Probably, the internals of libjxl are an enigma to me

Hmm... Wonder if effort 3 also has higher quality because of being self correcting

I guess so yep

but I wouldn't use lossy modular much tbh 😛

I use lossy modular for d < 0.15

why?

I treat it like WebP near lossless

I'd be surprised if vardct didn't do better

possibly with slightly modified flags

I haven’t actually tested it lol

I guess I’m too used to the quality ceiling that WebP has

Due to its unfortunate design of being based on VP8

any fast decode tuning?

My understanding of the self-correcting predictor is that it's a different approach to predicting entirely. On higher effort levels there's a bunch of expensive checks going on to determine which predictor is best on any given section of the image or MA tree leaf. There are no such checks when the self-correcting predictor is used, it just naively uses it everywhere. The idea is that it is generally *good enough* for all image data, but not as good as trying all of the possible predictors and going with the right one (anyone correct me if I'm wrong)

And being self-correcting is what makes it "good enough on all image data." It'll do an alright job adapting to the image data, but it's possible there was another another predictor that would have done better that wasn't evaluated

you can certainly mix the self-correcting predictor with the logic to determine a good predictor, fwiw

For sure, and there's probably some cases where the self-correcting predictor even does better than any other predictor, and I assume in that case it's just another predictor that can be chosen for a certain context

Are signals from the self-correcting predictor actually used to inform which of the other predictors to pick? Like are signals from it used to inform predictor selection? Or am I misunderstanding you

Because that's actually interesting, I don't know how predictor selection works, I thought it just tries them all and goes with the best

there's a decision tree that looks at a few things to decide which predictor to use, the decision tree is encoder-decided

If that tree is the MA tree, that matches my understanding. If not I have some more reading up to do :D

yep

Finally, 100 MP is done <:Hypers:808826266060193874> Here are the encode results:

Here are the decoding results:

For lossy modular effort 9, my guess would be that it chose a certain predictor which was very easy to decode.

huh that's very surprising

why not put the modular and varDCT on the same graph so they're easy to compare? 🤔

Because Excel wasn't having it

Excel <:PepeGlasses:878298516965982308>

I decided to have separate graphs for better readability

I'm too tired to learn Excel atm

I'll do `faster_decoding` benchmarks, probably with the 25 MP image

There are a few caveats. __Levels that are bit-exact or bugged are represented with a 0.__ 1. VarDCT: levels 2 and 3 are bit-exact. 2. Lossy modular: Levels 2, 3, and 4 are bit-exact. 3. Lossless: Levels 1 and 2 are bugged. Levels 3 and 4 are bit-exact.

In conclusion faster decoding is a footgun for anything Modular. There seems to be a case to be made for VarDCT, although I think 4 levels is overkill considering the number of bit-identical images. I think just having one level, being `--faster_decoding=2` or `--faster_decoding=4` makes more sense.

what does it even do for modular?

disable RCT?

why does bit-exact get a 0? 🤔

Because it's the exact same result, same BPP/decode speed/etc.

For example, levels 2 and 3 are bit-exact, so for level 3 just look at level 2.

It also illustrates how redundant the *four levels* of faster decoding are.

In my opinion there should only be one *maybe* two levels

I'm not actually sure what faster decoding does currently. I'd assume disable expensive predictors like weighted, but it's not guaranteed

I think all it does now is break things [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&name=av1_dogelol)

I'd guess it's literally *just* disabling weighted, so there's no prediction used instead

for dct it made some sense - stuff like turning off some filters and/or larger transforms

not sure why it was 4 levels though