JPEG XL

Openjpeg does true lossless by default

It can only do it in one way though, afaiu. So no room for encoder improvements in terms of density.

Because according to preliminary sizes, OpenJPH is better than JXL `-e 9`, WebP `-z 9` and on par with slow BMF, on very different content, but at FJXL speeds (for kakadu)? That's pretty strange 🤔

Yes, it is lossy or maybe lossy color conversion? 🤔

Ah, maybe the htj2k version isn't lossless by default

Usually this approach will not work -- the residual has two sources of entropy, the original image and the errors made in VarDCT, and the overall compression will be (likely) 10% worse than just compressing the original image with modular

I think we used to do very well with generation loss. I wonder if recent quantization changes have sabotaged the generation loss behaviour. It could be interesting to try generation loss with a year old version of libjxl. ... also it might be a good idea to turn of DCT coefficient 'dithering' for energy preservation to improve on generation loss.

the experiment I performed supports this theory

Voila:

`-d` is random between 1.00 and 2.00

This image by Stanley Lau turned out to be resistant to funny artifacts 😄

Tried it with photo this time:

Thank you!

I think the second test is a good example of how changes from past year have affected the generation loss to the worse, and at the same time how much the preservation of details has improved.

<@!111445179587624960> you may want to do a benchmark of fjxl with and without this pull request: https://github.com/libjxl/libjxl/pull/1086

Yep, as soon as I have time

There's some margin for improvement: it is using the (clamped)Gradient predictor also in the palette case while probably the Zero, Select or North predictor works better.

But it should help on images with few colors

In the QOI testset there are only about 70 images where it makes a difference, but for those images, it cuts the bytesize in half

Hmm, probably yes, other predictors would be useful, since for now, for Pixel Art and Game sets compression hasn't improved that much, and on some purely photos it's even gotten a little worse (but less than 1%)

Hmm, palette detection doesn't work on something like this?

https://i.redd.it/49ox0pmrf6c61.png

```3,644,906 - FJXL 3,771,200 - FJXL palette detection```

... why would that make any difference whatsoever?

pretty sure it has many colors

perhaps it skips huffman adaptation? <@!794205442175402004>

it shouldn't change anything when palette is not used

A lot of those pictures are missing now (from the reddit source link). Are they integrated into the benchmark itself? I don't know who runs or maintains the benchmark itself..

Integrated as in stored with the benchmark..

Okay, the benchmark isn't in the jpegxl.info website, don't remember where I got the link..

I am the creator of this benchmark, but yes, this is a problem, but I have no choice, since I do not want special test sets of free images, then I use the sets from various public places, although I have all the source images, but for obvious reasons I can not distribute links to a very wide audience or reference them in the spreadsheet itself

But, if anyone needed, a link to all sets was mentioned here https://discord.com/channels/794206087879852103/803645746661425173/923949551650500618

<@!179701849576833024> Hmm, it seems that after some recent changes the result is noticeably worse, if I'm not mistaken sometimes FPNGE was better than FPNG, even on images without transparency, I think on all photo sets

Also > fprintf(stderr, "Usage: %s in.png out.**jxl** [num_reps]\n", argv[0]); <https://github.com/veluca93/fpnge/blob/main/fpnge_main.cc#L27>

https://twitter.com/richgel999/status/1481100874483216384

that sounds about right

haha ops!

I made changes to the branch, should have more impact now

Hmm, so is it possible to use the Gradient predictor for palette images with few colors and Zero or something else when there are more colors?

that does look like a nice improvement

Yep, if further improving efficiency for Pixel Art and Game Sets, FJXL will be on par with good PNG encoders

should be reachable with custom NN upscaling ;)

Perhaps, but I think even with the proper predictors it is possible to noticeably improve compression

Btw, how can I check which one might be better for FJXL by using CJXL, choose something like `-s 4 -i 0` and different predictors?

cjxl doesn't quite do the same thing as fjxl, not with any options, but I _think_ something like `-m -s 8 -C 1 -I 0 -P 5 --palette 0 -X 0 -Y 0 --patches 0` is somewhat close to non-palette fjxl, and `-m -s 8 -C 0 -I 0 -P 5 --palette 512 -X 0 -Y 0 --patches 0` is somewhat close to palette fjxl

Yes, I understand, I rather meant which predictor would be efficient for FJXL as well (for certain images), not a full FJXL emulation

I did some quick experimentation and for some images, the Select predictor is more effective with palette

Hmm, more efficient than Zero, but it can also be pretty fast? Although WebP also uses it even on fast modes and palette images have very good compression 🤔

Select is basically the same as ClampedGradient regarding speed

Then, it might be a good idea to have it and as I said, it would be useful further for slower modes and let's say `-e 3` could be a compression beast at its speed, if properly switched between several of the most effective predictors

Just picking best of `cjxl -e 3` and fjxl is probably a pretty good fast encoder

Hmm, so Select predictor and smallest group size `-m -s 8 -C 0 -I 0 -P 4 -g 0 --palette 512 -X 0 -Y 0 --patches 0` showed better results for Pixel Art, don't know how this relates to FJXL but it might also be worth trying the smallest group size, this will also allow to use more threads for smaller images when it is needed for a bigger speed increase

smaller group size will likely only be worse in the fjxl case for density

in the -s 8 case, it is doing both local and global palette

fjxl only does global palette

Hmm, but it might still be worth experimenting and comparing the smallest and largest group size for FJXL, maybe something will give better results Also, like I said before, when the highest speed is needed, probably more tiles/groups per image would be better, if the efficiency loss isn't particularly noticeable

Yes, perhaps we could add a speed parameter to fjxl instead of aiming for a single trade-off point

In the future it would be nice, but I think one fastest and one "slowest" mode with good compression similar to the most optimized PNG or better would be enough, also it would be nice if it was strongly asymmetric and had a high decoding speed Something like a combination of `-e 3` for photos with something as efficient and fast for pixel art and some other artificial images where `-e 3` is bad

https://twitter.com/richgel999/status/1481382860498116617

updated with current versions of fpng and fjxl, and added fpnge

fjxl is the version with this PR included: https://github.com/libjxl/libjxl/pull/1086

nice! I assume fpnge is without a fixed predictor?

whatever it does by default

can you also try -DFPNGE_FIXED_PREDICTOR=2 and 4?

sure

fixed predictor 4 seems to hit the best trade-off, imo

is that paeth or which one is it?

yup paeth

2 is avgN+W or which one is that?

top iirc

The fact that fjxl produces smaller output than optipng is pretty amazing.

I figure 99% of that is the color transform and the planar encoding

and clampedgradient being better than any of the png predictors

not sure if it makes *that* much of a difference

Could make an fjxl with paeth instead to see how it performs...

could, yes, should, good question... 😛

it's about 2% less dense with paeth than with clampedgradient

so indeed not that much of a difference

screenshots are 14.714 MB with clampedgradient, 14.967 MB with paeth large photos are 49.885 MB with clampedgradient, 50.709 MB with paeth small photos are 20.400 MB with clampedgradient, 20.872 MB with paeth

~~gotta include avif to make every format seems 10 times better XD~~

Adding PIK would also be interesting, because in my comparisons it was better and faster than JXL `-e 3` (except Pixel Art)

sigh, I cannot even get pik compiled anymore

Sad, since this is roughly what a fast path can be for `-e 3` with better compression, especially if further added selection of a more efficient mode for non-photo images

And in this comparison, since FPNG is now faster, it might be nice to add FPNG -s (better compression, but a little slower) and FPNGE P4 to show how much faster FPNGE can be with still good compression https://twitter.com/jonsneyers/status/1483000547934449668

For FPNGE in the future it might be more convenient to have a choice of predictors through CLI options and add multithreading for really crazy fast encoding

for fjxl it would also in principle be possible to sacrifice some density to get more speed (skipping the palette check, the huffman code sampling, possibly skipping YCoCg or using SubtractGreen instead, possibly using a cheaper predictor, etc)

or to sacrifice some speed for more density (do more huffman code sampling, perhaps add some ctx modeling)

also some more specialization could be done, like a separate code path for RGB (as opposed to RGBA), which could make it a bit faster _and_ a bit denser for images without alpha

there is quite some room between fjxl and cjxl -e 1/2/3... perhaps i'll at some point make a qocjxl that fills that gap — e.g. something that does WP like -e 3, but still uses huffman coding and lz77 like fjxl, and not hardcoded for 8-bit like fjxl, but also not as generic as cjxl (which can do up to 32-bit), maybe limited to 12 or 14-bit (i.e. can do everything in int16_t) would be a good spot...

Yes, like adding -f (fast) and -s (slow) modes, because as I said before, having something like a balanced mode (slow for FJXL) would be even more useful for most people when the encoding speed is still fast for real use, but not as fast as the fastest modes, while having significantly better compression, which would at least beat the best optimized PNGs for most images Although the simplicity of the encoder would be lost even more (but in my opinion this is a quite dubious advantage)

Even faster mode is also interesting, but for FJXL there is a problem that decoding is not that fast and having an extremely fast encoder without the same decoding speed significantly limits use cases

I think it would be more convenient as `-s` (slow) mode for FJXL as I mentioned above than creating another completely separate encoder

yes — just codewise it wouldn't share that much code — not that that matters much

Would 12/14-bit support have minimal impact on speed and memory consumption? And continuing the theme, I would like to see something like PIK in terms of efficiency, but with higher encoding speed, because there are still complaints that JXL is very slow and not so efficient for these speeds, so I think something like based on `-e 3` + good palette mode is a fairly balanced solution Also, how much does WP affect compression, compared to the other things used in `-e 3`, does the predictor have the biggest impact? Something lighter and faster could be useful too, but I think a mode with higher compression is more needed

WP is better than ClampedGradient for photo, but it does come at a significant speed cost

what makes pik sometimes better is that it has some heuristic to select the RCT, while -e 3 just always uses YCoCg

Why so poor compression

YCoCg is a good all-purpose RCT (and pik doesn't have it), but selecting an RCT per group like pik does it is good too

not sure if 14-bit fjxl can be done, 10 and 12-bit should be doable

as for impact on speed and memory: the only thing that would really change is that the input would be twice as large (8 bytes per pixel instead of 4), other than that the remaining steps would be pretty similar

(assuming you skip palette for > 8 bit)

Because for a fast encoder, I think it's better to prioritize the most used and common images, although for photos perhaps more than 8-bit may be necessary And also yes, the fast path for RGB only without alpha (if it gives a noticeable gain)

So like having a more restricted but faster and less memory-consuming encoder that is still usable enough and suitable for most common images

Guess that's hard to find out without the actual image :p

I just use cjxlng

For me it goes down from 792KB to 665KB with -q 100 -e 7

https://github.com/libjxl/libjxl/pull/1086 trims off another 1% from the QOI test set compared to yesterday's version of that branch

maybe i've been tweaking a bit too much for the QOI test set, the improvement is smaller for other test sets

i'm curious what the effect is on these other corpuses

On the default setting? Also, maybe in the future rename Effort to something else when slower modes come out, because there could be confusion for such large values: `Effort should be between 0 and 127 (default is 2, more is slower)` And I would also like to be able to disable multithreaded mode via cli

<https://drive.google.com/drive/u/2/folders/1X_F_vhNwJFhPWSW3EeSa-gGhhikkOoYd> And also my sets which can also be used for tuning, there are a lot of images, but for fast encoders it does not take long and it is also possible to use only a certain amount of random images from each set

either remove -fopenmp when compiling, or pass the OMP_NUM_THREADS=1 environment variable

Yeah, I meant without recompiling

FJXL PD is the previous build

where is the current build?

FJXL

interesting

not a very big difference, it seems

should be somewhat faster though - it's now using only about half the amount of samples for huffman code learning than in the previous build

can set effort to 4 to get something closer to the previous build

Yes, it's a little worse in some of the photo/game sets and pixel art, I'll compare different efforts later

It seems that something higher than Effort 4 does not affect that much

Large photo

Pixel Art

it's image-dependent but yes, generally speaking more effort just makes it slower and impact on density goes down above effort 16 or so

effort 8 should still be slightly better than effort 4 in density, but it's probably beyond the third decimal of bpp

Fractals <:monkaMega:809252622900789269>

heh interesting, it's not monotonic on that image

Very good efficiency on photo sets, especially LPCB 🤔

But for something like comics, manga, and pixel art, FPNG is noticeably better <@!179701849576833024> I did not add previous FPNGE results, but overall compression is better

What changes between JXL e 3 and e 7 to get such a difference in the Pixel Art corpus?

anything grayscale or palette-izable will work horribly with fpnge - it'll be encoded as RGB888, which is bad

e3 uses a predictor that really only works for photo content, e7 tries to figure out what to use and to do some better ctx modeling

Ah, thank you.

Yes, that's why I asked about this kind of content and if it's possible to identify and compress it more efficiently

it's possible in theory

probably not even a lot of work

but I probably won't do it 😄

Also, does JXL use something special for this kind of content? Since as recently discussed on encode.su there are MRP and other codecs very efficient for grayscale and such

not really, a lot of gains come for free with planar encoding

we don't have the kind of pixel bitpacking that PNG has, though

FPNGE4 (Predictor 4)

There could be some extra speed for grayscale fjxl, but not much extra density

Same with specialized RGB without A path

Not explicitly, but I wonder if 3 horizontal squeezes and then an 8-channel Palette does not boil down to kind of the same thing as pixel bitpacking

Hmm, so as a thinking, for FPNGE would also be useful something like a palette mode, more simple than FJXL, but as I understand it is better to use Zero predictor and some compression for indexed colors and where there are few colors?

In fjxl it uses clampedgradient also for palette, maybe I should experiment a bit more with other predictors

Palette detection code could indeed also be used for fpnge

I improved density a bit in some cases (also speed, but whatever :P) in case you want to give it a try again

<:Hypers:808826266060193874> I'll retest in a while

thanks!

Also, do common PNG encoders use some sort of heuristics to fast choose predictors, or is it also sort of a brute force?

they use some heuristics

how good they are is debatable 😄

Libpng doesn't have integrated deflate, so it does predictor selection based on something that not necessarily correlates well with actual compression

Now it is almost like FPNG in density or better, except for the low-color images

<@!179701849576833024> Maybe also add the exact file size below bpp in this mini-benchmark after encoding with reps? For easier comparison, because for example FPNG uses only file size

I do that with ls xD but yeah I could

🤔 FPNGE `158.122 MP/s` `6 619 757` Only QOI is better, but slower

what's the image?

ah I think I can see why

what does fjxl make of it?

8T FJXL `419.444 MP/s` `5 147 264` <:Stonks:806137886726553651>

yeah I thought it would compress well

YCoCg really helps here, I think

so does QOI's color cache, probably

Also, strange, does FPNG use something different for palette images or is it just more heavy compression?

no clue

might just be that the better RLE gives bigger benefits

PNGCHECK comparison for some image from a pixel art set (FPNGE, FPNG, optimized PNG) 🤔

So does FPNG also use a 32k window or is that a wrong detection?

FPNGE, LodePNG, optimized PNG

Should palette and transparent chunks be separated for better compression or for something else? Ah, it's for simple transparency

It seems that LodePNG always uses Zero predictor for palette data 🤔 (however, it has at least Deflate compression)

Hmm <https://raw.githubusercontent.com/lvandeve/lodepng/master/lodepng.cpp> ```There is a heuristic called the minimum sum of absolute differences heuristic, suggested by the PNG standard: * If the image type is Palette, or the bit depth is smaller than 8, then do not filter the image (i.e. use fixed filtering, with the filter None). * (The other case) If the image type is Grayscale or RGB (with or without Alpha), and the bit depth is not smaller than 8, then use adaptive filtering heuristic as follows: independently for each row, apply all five filters and select the filter that produces the smallest sum of absolute values per row. This heuristic is used if filter strategy is LFS_MINSUM and filter_palette_zero is true. If filter_palette_zero is true and filter_strategy is not LFS_MINSUM, the above heuristic is followed, but for "the other case", whatever strategy filter_strategy is set to instead of the minimum sum heuristic is used.```

it's a reasonable heuristic, and libpng also does that, but it's not necessarily optimal

Zero filter for palette is mostly needed when you have < 256 palette colors, because it's the only way to ensure you actually have fewer literal symbols (with prediction, you end up with `min(nb_palette_colors * 2 - 1, 256)` different literal symbols)

If you have ~256 palette colors, and they are ordered well, then prediction does actually still help

So at least a strategy for a small number of colors can be useful and I noticed that for transparency also often uses Zero filter, but I do not quite understand why some encoders create separate chunks for transparency and palette, and some do not

how do you mean?

if I understand correctly, PNG always has RGB palettes in the PLTE chunk, and then optionally you can also define A values for each palette color using the tRNS chunk

palette colors are implicitly opaque and the tRNS chunk can have fewer entries than the PLTE chunk

Like this (but sometimes it's only IDAT, for the same image, but with a different encoder):

yeah AFAIU that's how it works

Ah I forgot about bit packing

If you have less than 16 colors, png actually stores two or more pixels per byte

So for really low color counts, the argument that predictors introduce new symbols doesn't hold

For > 16 and < 256 colors it does hold though

AFAIU it uses just RLE - it just says otherwise in the deflate header, but it doesn't *really* matter

so why is decoding with a float32 buffer fastest?

that seems odd

also don't mind the uint32 one, its failing to set the output buffer

codes all the same, only changes are the format parameter, and the actually buffer types

can you perhaps get a profile in some way?

i honestly don't know, im running this all in wasm

it's pretty hard to troubleshoot performance without a profile, and I definitely don't know enough to get one xD

What image is it?

If it's a case where direct uint8 output is not done, then it makes sense that uint8 is slower - it would decode to float32 and then convert to uint8, right <@179701849576833024> ?

yes, but still

the image in question

hm

testing it with the 117B file from https://jpegxl.info/ I get a similar result.

Can you try a vardct encoded image? E.g. the result of default cjxl without options

that would require me to get a function to emulate the cjxl inputs

i will eventually do that

but not now

Updated `-e 1`/`-e 2` (no comparison with the old ones, as it is quite cumbersome to do a lot of extra columns), but still

So, fast speeds also need a palette mode like for FJXL

I also wonder what would give more gain to FJXL with a moderate increase in encoding time, ANS coding, some context model or predictors

I think the first thing to try would be a different hybriduintconfig so more bits get entropy coded

Then maybe some simple fixed ctx model, something like the e2 one

ANS vs huffman likely doesn't make a huge difference when there's no symbol that has a chance > 50%, so without ctx model it's likely not worth the trouble

(remember we do have rle for the simple solid or gradient regions, so the zero symbol is not _that_ common even in nonphoto images)

When there is some ctx model, ANS probably starts to give some more advantage, and while it cannot be done as fast as huffman (simd ANS seems very tricky), it could probably be done faster than what cjxl does. Cjxl -e 1 does four scans over the samples: once to collect tokens, once to count them to make the histogram, once (in reverse order) to produce the ANS stream, and then once to write the stream in the correct order

So, only one of these things doesn't work that much 🤔 , also it looks like `-e 1` needs a serious tuning or changing to something like FJXL, because it's already mostly better

Probably you could do sampling instead of full counting, and then do one scan in reverse order to write tokens directly to a buffer that grows from right to left

Yes, e1 can already be worse than fjxl - e1 does not do palette nor rle

you can't save that much imho

Probably not much, could try e2 with ans and with huffman to see how much difference it makes. At e1 not that much, at e2 maybe a bit more, but I dunno

Hmm, yes, on average it is not very different, but on some images the gap is very noticeable, in addition, at least on my CPU, the speed difference between FPNGE4 and FPNGE34 sometimes imperceptible

And if there was also a palette mode, it would reduce the difference even more, since 34 predictors are worse than 24/1234 mostly on such images So, there can be like these speed grades: FPNGE2 -> FPNGE4/FPNGE34 (maybe even without FPNGE4) -> FPNGE (with all predictors)

Meanwhile I got an experimental version of fjxl that can do 10 and 12 bit images too (with ppm/pgm/pam input)

Additional predictors would also be interesting, since I don't really have sets with 10/12-bit images, but it's a pretty needed thing and also PPM

https://imagecompression.info/test_images/

Here are some 16-bit test images that can be converted to 10 or 12 bit with `convert -depth 10`

It would be good to have more corpora of high bit depth images, in particular ones that are rec.2100 pq or hlg

Yes, I mean a varied good sets for comparisons, also considering that it can be used for like game screenshots

(which might have different compression characteristics than just sRGB with higher precision)

Yes, hdr game screenshots would be an interesting set to test on

WP predictor can also work well on non-pixel-art game screenshots

Yes, it's a bit inherently slow though

Compared to other predictors

fpng `206.843 MP/s 8.703 bits/pixel` fpng4 `236.443 MP/s 8.862 bits/pixel` fpng24 `228.501 MP/s 8.851 bits/pixel` fpng34 `231.221 MP/s 8.706 bits/pixel`

What is the challenge for fjxl when going above 8 Bits? And why does it expand to 10/12 but not 16/32 Bits?

huh, 34 is nicer than I thought 😄

Yes, 34 can be almost a replacement for 4, although it is better to also test on some other hardware, maybe there will be more differences than on my CPU

how did you measure speed?

4 should be faster than tryall

I mean, more faster 😛

Just compared for something like 100 to 500 reps on the same image and the result was about the same

There may be more difference in other images, but I was more interested in the comparison between 34 and 4 and it usually isn't much different

fjxl uses int16 internally, also for uint8 input, since YCoCg requires 1 extra bit and predictor subtraction requires another extra bit, so the values to be encoded are 10 bit. The approach also works for uint10 and uint12 input, since you then get 12 or 14 bit values, which still fit in int16

uint16 input wouldn't work without moving to an internal representation that is 32-bit, which is of course possible (cjxl does that), but it would be slower

32-bit input is even trickier, because then you have to do predictor arithmetic in 64-bit (cjxl does that) and it would be even slower

fjxl gets a lot of its speed from SIMD: in avx2 you can do everything with 256 = 16x16 bits per instruction, so 16 values can be processed at the same time for YCoCg conversion, Gradient prediction, and even huffman coding itself (though I didn't get that working yet for >8-bit, there are some additional constraints to make SIMD huffman coding work)

https://github.com/libjxl/libjxl/pull/1124

for a set of 10-bit test images: Total png size: 560 MB (in almost 4 minutes) Total jxl size: 337 MB (in about 4 seconds)

(default imagemagick/libpng vs fjxl)

for lossless HDR screenshots, I imagine this can be pretty useful

(also this is without SIMD huffman coding; can probably get those 4 seconds down to 2 or so if we can make it work)

Now, has this also been used for grayscale? https://discord.com/channels/794206087879852103/794206170445119489/935123835399831553 Or is it too slow for FJXL?

The weak point is still some palette and pixelart images (also for FPNGE), I wonder if it is possible to make some faster WebP Z0 replica for such images? And for FPNGE (although I understand that now there is little chance that someone will improve it, but theoretically) something like what LodePNG does using Zero predictor

It would also be interesting to compare with Jpeg XR as it is used for NVIDIA capture and Windows 10/11 gamebar for HDR screenshots, but it only has a old abandoned encoder/library and a pretty buggy acceptance of input images other than 8 or 32-bit

The only reliable output format I've found is BMP, everything else is either unreadable or changes the bit depth (to 16bit)

But then nothing reads the BMP files correctly, I had to write code to convert them to 10bit PNM files

ImageMagick can also encode JXR, but if there is a JXREncApp.exe in the paths or some external libraries

However, as far as I remember lossless JXR is less efficient than J2K and probably HTJ2K too, although I am not sure about 10/12 bit images, also, this implementation should be faster than OpenJPH (but not sure about compression): <https://github.com/osamu620/OpenHTJ2K> (but, when I was testing, I didn't have this encoder compile/work correctly for Windows)

Also JXL/FJXL can have even more potential gains on non-photographic images than J2K/JXR (which may be quite useful for game and usual screenshots)

Good question how it compares to jxr. I guess I mostly need a good test set of 10-bit images...

I've got ~30 from my Xbox in JXR, no idea if I got the conversion right but here's one as a 10 bit PNM, https://flawlesscowboy.xyz/stuff/bray.pnm

Accidentally found some comparisons (including FLIF and Jpeg XL), and also once again confirming that lossless Jpeg XR is not very efficient compared to other formats

https://arxiv.org/pdf/2108.02557.pdf

Makes sense - after all, XR was Microsoft's attempt at a "QOI" version of JPEG 2000: lower complexity, less dense. J2K was too slow, JPEG was fast but feature-limited (no hdr/alpha/lossless), so they made Windows Media Photo to fill that gap

How does JPEG XS compare to these super fast encoders? Wasn't that designed for speed?

As far as I remember lossless encoding has not yet been fully finalized in the specification and it has some things with patents

Ah.

XS is indeed not royalty-free, and it's intended for hardware implementation, basically something that would be integrated in professional video equipment

so I could measure the speed of a software implementation of XS, but that's not very relevant

And XS is designed mostly for lossy and video, btw, for theoretical lossy FJXL I'm not sure what could be done much faster

For lossy 8-bit, you might be able to avoid going to 16-bit, e.g. do things in 8-bit YCbCr, but not sure if that would help much

thanks for the test image, feel free to send a few more if you have more. Any idea what colorspace they're in?

Will do, they're apparently BT.2100 PQ (according to https://github.com/13thsymphony/HDRImageViewer/blob/aef000911813daadf367acf093ab514a6e9e76e1/DXRenderer/ImageLoader.cpp#L959)

https://flawlesscowboy.xyz/stuff/Xbox%20HDR.7z < 26 images, 1.2GB in total (Man this PNM format has bad compression 😁)

Rec2100 PQ was indeed what I would assume it was, but it's good to check

Do these images have a license? E.g. can I include them in a public test set?

Personally I consider them public domain, but I've got no idea how that works with game stuff

I suppose the existence of twitch shows that it's ok

Did you make those screenshots yourself?

Yep, all taken on my Xbox

Then it's just a matter of adding a CC0 or some other CC license and I think it's ok

That's my ugly purple haired character in them

Also, qoi-lz4-avg2 is not bad (diff from average instead of previous pixel), it is only slightly slower than the regular qoi and better than the previous lz4 variation, except that PixelArt is like 30% worse

So it is quite competitive compared to at least FPNG -s, slightly slower, but with much better compression

Yes, but still inherently sequential to decode... And a new format

Yep and still FJXL is much faster and better (especially if further improving palette and pixel art compression)

``` -P K, --predictor=K [modular encoding] predictor(s) to use: 5=gradient, 6=weighted, 14=mix 5 and 6``` Hmm, does `-P 14` not work with something like this `-e 8 -C 0 -I 0 -P 14 --palette 512 -X 0 -Y 0`? Because from some of the results it looks like the gradient predictor is never selected And otherwise, yes, it seems that the Select predictor is a slightly better for palette images

So, SubtractGreen can be not much worse than YCoCg (at least for fast modes), but be noticeably faster? And also which predictor would be faster and not much worse than Gradient, something like W?

SubtractGreen and W will probably be significantly worse than YCoCg and Gradient, but it'll at least still compress somewhat. Not sure if it would actually be much faster though: I think you still need to do the arithmetic in 16-bit anyway (jxl doesn't have wraparound SubtractGreen)...

I'm just wondering if it's possible to have an encoding speed close to something like FPNGE4, but with some better compression

However, it seems that using YCoCg and Gradient is something optimal, and maybe also SubtractGreen and Gradient (as far as I understand SubtractGreen is almost the same as LOCO colorspace, and this combination is used as something fast and efficient) and it all depends on what is slower and gives less efficiency, the color space transform or the predictors

Well fpnge can do 32 samples at a time (8 rgba pixels) because png works with bytes. Fjxl can only do 16 samples at a time (16 planar samples) because jxl works with integers so the only way to avoid adding an extra bit is to not do any color conversion and not use any predictor

Also png stores interleaved RGBA directly, while jxl stores things in a planar way

We're in both cases assuming the input comes as interleaved RGBA, which in a way gives an advantage to png encoders

Also, why does JXL use gradient and weighted by default (except for all predictors) instead of select and weighted? Because photo content is not so easy to identify and gradient works better for it than select?

Maybe we should revise defaults, but I think those two are just the overall best predictors

Select is good for pixel art and crisp nonphoto, but for things like anti-aliased text like in modern screen shots, I think Gradient is already better again

But probably a better heuristic for cjxl would be to sample some small regions with different predictors and make a choice based on that

Also, could it possibly be up to 14-bit? Like to fully support some photos

Without SIMD huffman coding (like it is now for > 8-bit anyway), yes, up to 14-bit could be done

With SIMD huffman coding the way it is currently implemented for 8-bit, there's a constraint that prefix code + raw bits has to fit in 16 bits, which is probably already a problem for 12-bit input, because with ycocg and prediction, the largest residuals are 14-bit which means you need a 3-bit code for that (and 13 raw bits, first 1 is implicit)

But maybe different simd huffman coding can be done without that constraint, not much slower

yeah it is possible, just someone needs to write it

Hmm, yes, after full tests, Gradient is better on average on everything except PixelArt (on "emulated" FJXL) Perhaps also Select is better integrated with other things and more efficient in WebP

By the way, does FJXL work like `--keep_invisible=0`? And as I said before, I think it's better to make `--keep_invisible=0` for CJXL, but keep `--keep_invisible=1` for Libjxl by default

Fjxl zeroes invisible pixels

```JPEG recompression 2022-01-31 web content (mostly) (optimized) JXL = cjxl -e 9 -E 3 -I 1 BRN = cbrunsli PJG = packJPG -ver LEP = lepton-slow-best-ratio Wins are smallest results. savings % median mean wins JXL 12.92 15.73 311 BRN 15.58 16.72 408 BRN/JXL 15.76 17.30 719 PJG 16.83 17.43 1223 LEP 16.95 17.63 1743 PJG/JXL 17.00 18.04 1534 LEP/JXL 17.08 18.23 2054 LEP/BRN 17.14 17.89 2151 PJG/BRN 17.17 17.72 1631 PJG/LEP 17.17 17.77 2965 any 17.42 18.38 3684```

`-I 1` also slightly improves compression and works for Jpeg re-compression as well

welp

In general JXL wins on large and high-quality Jpeg sources, also more often where YUV444 is used, even though, if not to consider that it is also quite a different class of compressors/formats with different purposes

Hm, I haven't the same conclusion about large, high-quality yet. JXL was worst on the photos and scans I tried before this (although that was without -I).

On my quality photos and other random quality photo sets, the JXL mostly beat the Brunsli (with other compressors there may be very different results and they have different purposes, but the Brunsli is almost the same, just without various JXL features) Also, `--strip` if not needed to keep all metadata, also as far as I remember brunsli does not save them (or at least in certain versions) and because of this also has some advantage on images where they exist, so like: `-e 9 -I 1 -E 3 --strip`

Brunsli git master preserves metadata as far as I've seen (but I didn't actually roundtrip everything here).

I'm personally interested in results including metadata.

Some, examples, hmm, quite a long URL for all files, but still (because it's higher than Discord's limits): <https://transfer.sh/%28/to4UiJ/41288175385_0398187107_o.jpg.jxl,/AuTt01/41288175385_0398187107_o.jpg.br,/wzkAUL/41286909695_750263f4e3_o.jpg.jxl,/dtpbT0/41286909695_750263f4e3_o.jpg.br,/dcmWba/41288175385_0398187107_o.jpg,/7ir6Bt/41286909695_750263f4e3_o.jpg%29.zip>

I think cjxl is still keeping exif/xmp uncompressed instead of brotli'ing it

Yes, it is.

It could brotli it though. Only makes a difference if there is a large xmp blob

jxl is better at recompressing 444 than 420. Chroma subsampling was something jxl originally didn't have at all, and we reluctantly added it just to be able to support representing/recompressing 420 jpegs at all

We didn't try to make Chroma from Luma work in the non-444 case though, so that's one thing that 444 jpeg recompression can benefit from but 420 cannot

Does -E 3 actually make a difference when recompressing jpeg? Not sure if that parameter gets passed to the dc encoding

It does, as far as I recall from prior testing.

Yes, both `-E 3` and `-I 1` slightly help compression

if it does, then that would also only be mostly for 444 I suspect

In this example, Brunsli preserved the metadata and JXL did not. JXL still wins on one image, but not both when doing bit-lossless.

Yes, but JXL still wins even if excluding metadata (I checked on at least at one of these images)

Ah, that's the same thing, I just didn't read the whole message And these are just random examples that I found just now, usually on similar images the result is quite same, also I found one of the online store catalogs with medium quality images and there JXL was also better on almost all images, but in general on lower quality images Brunsli wins, but I think this is quite a good trade for additional JXL features (and since JXL is not supposed as a storage format and it is basically kind of simplified but real JXL images), plus there is still room for improvements

Also, I wonder if it is possible to create a Jpeg decoder based on JXL, is it possible to create at least a simple JXL-Jpeg encoder?

Because I do not think that the any near future the need for JPEG encoding will be gone, at least to maintain compatibility

redone JXL results

Comparison of WP and Gradient predictors on "emulated" non-palette FJXL (not sure how much this really represents the real difference)

And it looks like only predictors can't give much improvement without other things, but I think that at least these two would be pretty useful, although it would require some fast heuristics for FJXL

what are those WP/Gradient columns?

`-m -s 8 -C 0 -I 0 -P 6/5 --palette 512 -X 0 -Y 0 --patches 0`

ah, I see

fjxl does -C 1

other than that, that is probably a reasonable emulation of what fjxl would do, more or less

Ah, yes, that's a typo, but the real comparison was with the proper options to emulate a non-palette FJXL: `-m -e 8 -C 1 -I 0 -P 6/5 --palette 0 -X 0 -Y 0 --patches 0` So it's correct

<@!794205442175402004><https://www.reddit.com/r/compression/comments/scvzmf/question_im_here_as_an_ignorant_looking_for/huz0f0x/> Also, about these comparisons, what results does EMMA show? (when there is free time for it) There are links and options in this post: <https://www.reddit.com/r/jpegxl/comments/mepoun/modern_lossless_jpegxl_still_defeated_by_old/gtm8geq/> Maybe my PPMs aren't converted right or something (because I noticed that sometimes results can differ depending on which converter was used for PPMs) And because it's not the first time I've heard that the Gralic is still better, even though I've shown other compressors (but, for me EMMA usually beats almost everything except pixelart and some other artificial images) Also, as I already said, the LEA is very good for its speed, encoding speed is about 1.5 times slower than JXL `-e 3`, but compression is on par with `-e 9` or better (and it's only like 23kb uncompressed binary with some unused code to compress regular data) https://globalcompetition.compression.ru/assets/files/compressors/LEA.7z

Also, as far as I understand, EMMA is CM compressor and BMF at least in slow modes is PPM, probably that's why they show better results than others and maybe someday, if people continue experiments and research, they can be made more asymmetric and fast > **Shelwien** > CM idea is explicitly based on combining (mixing) predictions from multiple statistical models. > While PPM method appeared much earlier (wikipedia lists 1984) and can be interpreted as evolution of LZ methods (extreme ROLZ with unary length coding). > But PPM can be also easily turned into CM with linear mixing and unary weight estimation. > So CM/PPM difference is mostly historical. > For example, Ppmonstr is technically a result of PPM evolution, but practically its a CM over unary codes of symbol ranks, with SSE as a mixing function, and multiple submodels (aside from order* contexts, there're additional sparse contexts etc). > > > PPM: > > Uses an ex. '8 letter context' as the model-er of what letter comes next. > Well, the practical difference between CM and PPM _compressors_ is that PPM first encodes the choice of submodel (context), then encodes the symbol using the probability distribution of that context only. > This is also known as context switching (PPM) vs context mixing (CM).

Perhaps they were not widely used in image compression due to not enough speed (and lack of research in this direction) and the much slower hardware at the time (except for BMF and some other experimental codecs and more modern ones like EMMA, sadly they are not open source)

Also, `-e 4` uses Gradient and Weighted predictors? And is it notably slower than fast modes because it uses more heavier context modeling and entropy coding?

It does MA tree learning instead of using a fixed tree

So perhaps having another mode with a fixed tree and these two predictors would be nice. Also, some tool or feature in the benchmark with detailed info about using predictors, transform, context model, group size, etc.

Also, I wonder if making heuristics to pick predictors based on an sample of small image regions, but prioritizing the more versatile and efficient predictors and for them the region size for a sample would be larger, and for less versatile ones use a smaller sample, so theoretically it would be faster or more accurate and efficiency would hurt less in the mistakes

Hmm, it seems that Oodle Lossless Image has 5+ times faster than WebP decoding even in single-threaded mode (and much more in multi-threaded mode), with better compression (sometimes even better than FLIF), I wonder how this is achieved, although their compressors like Leviathan and Kraken are also very fast and denser than LZMA (but with slower encoding) Hopefully they'll be open source soon (as Epic claimed) and I can do my own comparisons ```Leviathan : 3.23:1 , 0.9 enc MB/s , 642.4 dec MB/s LZMA : 3.18:1 , 3.0 enc MB/s , 53.8 dec MB/s Kraken : 3.09:1 , 1.3 enc MB/s , 948.5 dec MB/s zlib : 2.33:1 , 7.7 enc MB/s , 309.0 dec MB/s``` Some more information from the blog about palettized images (it looks like the site is down, but there is a Web archive): <https://web.archive.org/web/20210107214851/https://www.jonolick.com/home/oodle-lossless-image-v147>

> Implementing this was pretty interesting in that the order of those colors in the palette matter quite a bit - and the reason is that if you get it right, then it works with the prediction filters. As in, if the palettized color indexes are linearly predictable then there is a good chance you will get significantly better compression than just a random ordering. In practice this means trying a bunch of different heuristics (since computing optimal brute force like is prohibitively expensive). > So you sort by luma, or by different channels, or by distance from the last color for example (picking the most common color as the first one). I also implemented mZeng palette ordering technique which isn't commonly in PNG compressors. Believe it or not, while this theoretically should produce really good results in most cases, sometimes the simpler heuristics win by a lot so you can't just always use a single method to decide when going for minimum file sizes

However, as already discussed, having something like WebP emulation in JXL would also be pretty good, even WebP Z0 proves very efficient, with the encoding speed on the level of fast JXL modes it is much denser and "stable" (especially on non-photo)

I'm not super surprised, those guys know what they're doing 😛

especially for speed

Yep, after some more tests, it seems that from the fastest and cheapest predictors - Left (W) is the most optimal, so theoretically for FJXL the Effort choice could be something like this (from faster to slower): ```0 - None + Zero + No palette detection + No huffman code sampling (but still possible through the option) 1 - SubtractGreen (if it makes sense) or YCoCg + Left (W) + No palette detection 2 - YCoCg + Gradient 3 - YCoCg + Gradient + Ctx modeling and ANS/Huffman 4 - YCoCg + Gradient and WP + Select (or/and Zero) for palette (and maybe lz77) + Ctx modeling and ANS/Huffman 5 - Some sort of bruteforce/heuristics of available FJXL predictors and color transforms + ANS, and the like ...``` And also keep a separate option for huffman code sampling (but not as Effort)

`13=toptop predictive average` This predictor is slower than Gradient? It also sometimes produces best results (even than Weighted), although it's not as universal

Otherwise, it seems that yes, the most universal predictors are Left (W) (among the fastest), Gradient (the most versatile), Weighted (for photo-like content, including game screenshots and complex art) and also Select and Zero for palette images, pixel-art and if there are a lot of solid color areas Other predictors may be better on certain content, but much worse on other and on average results

The thing is that you can have a predictor per MA tree context, which can make not-so-useful predictors more useful

Yes, this is just for a fast encoder, in case there is no way to choose among many different predictors and which would be the most versatile (when the result is not dramatically worse if the predictor is not the best for that content and as a good average overall result on any content)

Also, it is interesting that Oodle Lossless Image uses many Gradient-based predictors, not just one

Which ones does it have?

I don't know, because the source code is not yet open, but judging by: > The filters are the standard PNG filters plus a few new ones, namely 3 different types of gradient predictors (sorely missing in stock PNG) and a parametric linear predictor (fixed-point filter coeffs are determined at encode time and stored) And also on some old experiments on the blog: https://cbloomrants.blogspot.com/2010/06/06-20-10-filters-for-png-alike.html

Some subjective results comparing cjxl -e 7, cwebp, mozjpeg and libavif aom -s 7

Do you used a test cjxl

This is just 0.6.1

Is there a public source with context?

Results are just in, we still need to figure out the best way to analyze them

But it's based on over 200 different original images and about a million pairwise comparisons

The raw data I cannot share, it costs quite a lot of money to produce such data considering it involves humans giving opinions.

Cool, looking forward to any kind of publication on this.

We're going to collect some more data to better calibrate results across different originals, and then probably I'll write a blogpost about it.

It's going to be a quite big subjective eval of modern codecs. We're also testing the proprietary Aurora avif encoder, as well as Kakadu for j2k and libheif/x265 for HEIC.

More results: https://twitter.com/jonsneyers/status/1493531124596940800?t=CakzF8IZp9726b00Rw0GrQ&s=19

Original image: https://cloudinary-subjectify-shared.s3.us-east-1.amazonaws.com/iqa_dataset_no_saliency_accepted/preprocess/twilight/861443.png

5.7 KB JPEG: (ok that's ridiculous I know) https://res.cloudinary.com/jon/image/fetch/f_jpg,q_5/https://cloudinary-subjectify-shared.s3.us-east-1.amazonaws.com/iqa_dataset_no_saliency_accepted/preprocess/twilight/861443.png

5.4 KB JPEG 2000: https://res.cloudinary.com/jon/image/fetch/f_png,q_100/https://res.cloudinary.com/jon/image/fetch/f_jp2,q_70/https://cloudinary-subjectify-shared.s3.us-east-1.amazonaws.com/iqa_dataset_no_saliency_accepted/preprocess/twilight/861443.png

not bad

although of course gradients are a bit of an extreme case for both codecs 😄

5.3 KB AVIF: https://res.cloudinary.com/jon/image/fetch/f_png,q_100/https://res.cloudinary.com/jon/image/fetch/f_avif,q_50/https://cloudinary-subjectify-shared.s3.us-east-1.amazonaws.com/iqa_dataset_no_saliency_accepted/preprocess/twilight/861443.png

5.3 KB JXL: https://res.cloudinary.com/jon/image/fetch/f_png,q_100/https://res.cloudinary.com/jon/image/fetch/f_jxl,q_65/https://cloudinary-subjectify-shared.s3.us-east-1.amazonaws.com/iqa_dataset_no_saliency_accepted/preprocess/twilight/861443.png

(sorry had wrong link for the j2k at first)

gap between avif and jxl is pretty large for that image

anyway, I'm currently fetching the whole test set so I can compute butteraugli, ssimulacra etc and see how well they correlate with the subjective results

225 original images in the set (the above is one of them), but about 18000 compressed variants, and I'm just fetching them one by one 🙂

these are all the originals (well, jpeg-compressed here for discord)

in case you're wondering what the test images were

So mostly photographic with a few diagrams then

yes — but we did try to cover various kinds of subjects: humans, buildings, food, nature, etc

What about synthetic, abstract, line & anime art?

There are a few images like that in the set, but overall photo is way more important

<@!794205442175402004> What was the rational for the encoder settings (mostly default)?

default is typically a good trade-off between speed and quality/compression

also we're interested in practical speeds: slower than that is not really feasible in production

e.g. aom s0 performs quite well but we can't afford that kind of cpu cost, and also first-image latency would be a serious problem

sample image text

sample image jpg

https://www.hdblog.it/games/articoli/n551459/cyberpunk-2077-1-5-patch-next-gen/

https://hd2.tudocdn.net/1021037?w=660&h=325

wow without cut

anothe sample wifi seven

this last is my favouite

hee's the link https://www.hdblog.it/portatili/articoli/n551457/macbook-pro-m2-13-pollici-marzo-rumor/

text specific

Compressed image

shapness elated image

i dont want to lose anything

ed colo

eeen colo

finished

in my opinion thees should be a dedicated channel fo test images but he's i'm a guest

General phone screenshots

At least for me. People usually don't do such complex capture.

A particular image for cjxl block selection

Random near lossless selector

......

atifacted images

vey good images

illegible

good

too noisy

supe

destroyed by gaborish

fuck is shit

niko pandetta too black like nicky jam

wong colo

this i like

this impessive with the gain

<@!794205442175402004>

ten is ugly too shap

-s 7 -d 0.3

hees' impessive compession

fifteen has moee oveshapening

To me this setting would solve most of the problems here

for %i in (C:\Users\Utente\Desktop\setwo\youtube\*.png) do cjxl -s 7 -d 0.519 -I 0.202 --epf=2 --intensity_target=260 -p %i %i.jxl

for %i in (C:\Users\Utente\Desktop\setwo\youtube\*.png) do cjxl -s 7 -q 92.18 -I 0.288 --intensity_target=271 -p --patches=0 %i %i.jxl

i will wait new build

for %i in (C:\Users\Utente\Desktop\setwo\youtube\*.png) do cjxl -s 7 -d 0.456 -I 0.751 --intensity_target=271 -p --patches=0 --epf=2 %i %i.jxl

yes quality is impoving

for %i in (C:\Users\Utente\Desktop\setwo\fou\*.png) do cjxl -s 7 -d 0.456 --intensity_target=271 -p --patches=0 --epf=2 %i %i.jxl

for %i in (C:\Users\Utente\Desktop\setwo\fou\*.png) do cjxl -s 7 -d 0.456 --intensity_target=271 -p --patches=0 --epf=2 %i %i.jxl

Who is Utente?

Yes, fuck is shit

Me (User,)

Fabian having a field day

how well objective metrics work to predict the subjective quality of libaom vs [some other codec]

same but for mozjpeg vs [other codecs]

here jxl vs [other codecs]

These look like they come from the same place as https://discord.com/channels/794206087879852103/803645746661425173/943119227081072650. Is this a publicly available publication? Gonna be? (just curious, looks very interesting!)

the data itself (raw results) are likely not going to be publicly available, the aggregated results like what I'm sharing here will probably eventually become a blogpost on the cloudinary blog or something

I just know myself, it can take a while to write such a blogpost, so I'm already sharing some stuff as I'm analyzing the data

segmenting the data by quality range instead of by codec gives this:

Objective metrics appear to perform really well in first charts, but poorly the last. Do other codec vs codec comparisons show worse performance, or is there some other reason for the observation?

I think it's just easier if half the ranking is just one codec at various q settings (which is what it is in the first charts) than if it's all codecs at various q settings in a more narrow quality range (so the subjective scores are noisier too)

Ah, makes sense. I didn't really have good intuition for this kind of analysis. But speaking of noise, this makes me wonder how much in agreement human judgement was between individuals. Also, why the choice of BA 1.5-norm (Jyrki recommended BA 6-norm for human comparison in the past)?

Agreement between individuals is kind of an indication of how big the gap is between two codecs. We did only 10 opinions per pair, and then basically consider it all games in a virtual tournament, one per original image, where each (encoder,q setting) is a player and you can compute ELO scores. If all humans agree, that's 10 games won against the other codec/setting, if there's lots of disagreement then it indicates that the two 'players' are at a similar level...

When using `butteraugli_main --pnorm 1.5`, it really computes some kind if weighted sum of pnorm 1.5, pnorm 3 and pnorm 6. I also tried the (default) 3-norm and the max-norm, but they performed worse than the 1.5-norm setting.

Higher norm is useful to make sure there's no artifact anywhere, lower norm is better to predict how humans judge the overall image quality

So, the Elo ratings inform the rankings for the Spearman correlation? then the distribution comes from tournaments for each original image?

Yes