JPEG XL

Who have the binary of butteraugli for windows https://github.com/google/butteraugli (it's impossible to compile windows program when we are noob :D)

<@132637059327328256> maybe this is a better place to discuss benchmark setup and results

ah

So, do I just make my source yuv444 jpegs ?

Jpegs? No that is not quite lossless. I would just undo the YCbCr and get rgb input

But even better is to avoid yuv sources, they are effectively only 7 bit for R and B so not quite ideal source images

Which yuv sources are you looking at? Because chances are they weren't yuv originally...

Right now i'm testing with a tiff frame from netflix chimera

I also have a manga page in png

i answer me, binaries are here if you have the same question (https://encode.su/threads/3544-JXL-reference-software-version-0-2-released?p=67989&viewfull=1#post67989)

average of 260 4K lossless images, encoded losslessly in both webp and jxl:

should probably get a line running through them so it's easier to read

in short, it's on-par with webp in terms of speed/compression, until you get to speed 5 and slower, where it's better compression than `cwebp -z 9` while being about as fast as `-z 8` (for `cjxl -s 5` and `-s 6`)

cool

if encode time is not a big issue, `-s 9 -E 3 -I 1` will push compression a bit further

yeah, I didn't even bother to test `-s 9` as `-s 8` was already slower than cwebp's `-z 9`

You also seem to lose about 15% compression, so it doesn't seem worth it for it to take longer and produce a larger file for slightly better quality. I've only used `-s 8` so far (except for testing), seems to be the best tradeoff between speed, file size, and quality.

for lossy, yes, `-s 9` is not worth it and hasn't been a focus – I think we mostly have focused on making `-s 5` to `-s 8` give good trade-offs

for lossless, `-s 9` is quite a bit more exhaustive than `-s 8` so it will give better results, but it's very slow so only if you don't care about speed

It's mostly that `-s 9` is not very well multi-threaded compared to the other speeds.

If you use per-file multithreading, it actually becomes a decent bit faster on my end.

other speeds are not that great in mutithreading either (in lossless) with the exception of `-s 3`

well, in multi-process literally any app scales

S7-S8 lossless has become better in terms of MT scaling with the latest patches nicely enough.

It's speed 9 which is still very slow from my experience obviously. 😛

the part of lossless compression that decides the context model (skipped at -s 3) is single threaded so far 🙂 it should be relatively easy to fix, but we didn't prioritize it yet

which patch?

Just the patches from 6 days ago.

I'll have to recompile then, still using vanilla 0.2.0

Some interesting findings. At very low BPP with --noise=1 and --epf=3 on, modular performs quite a bit better than VARDCT. https://slow.pics/c/JTcwkABF

However, blocking is more apparent.

Has anyone done JXL benchmarks using the FLIF sampleset? https://github.com/FLIF-hub/benchmarks

I just like looking at the animations 🙂

yup - in fact we switch to modular somewhere around -q 7 or so. You might also try --resample 2 to get extremely low bitrates...

Yeah, I think --resampling=2 is the best bet for absolute highest efficiency at these absurd bitrates, because no resampling means it has the most detail, but the most blocking, while --resampling>=4 makes the image far too blurry.

What is the best, --resampling 2 or divide by 4 the mpx of the picture? ^^

On the other hand, is it possible to modify the encoder to go down to -d 0.1 like -d 0.05? It's not visually pixel transparent on my display 6bits+FRC (with srgb 14bits and 8bits input picture)

Wait what? -d 0.1 not visually transparent?

Are you zooming in a lot or something?

Haven't seen examples yet where -d 0.3 is not visually transparent

Could be another encoder bug

Yes i'm speaking about ultrazooming, not normal visualisation, only few pixel have slightly color change.

The consequence is the precision of some pixel is way under 1/256 even if it's an 14bit picture, i'm expecting a visual perfect lossy 😄

did you know a software which can give the exact value of a pixel of 16bits? gimps did not that.

i have a max deviation (gimp pipette) of 1.2/256 in a green pixel, and give 2/256 on my display :p

If you want higher lossy precision, can try bumping up the max nits target

you can also try passing --gaborish=0

Or just do lossless

-q 93 -s 6 --noise=0 ==> 982 KiB -q 93 -s 6 --noise=1 ==> 982 KiB -q 93 -s 7 --noise=0 ==> 989 KiB -q 93 -s 7 --noise=1 ==> 989 KiB -q 93 -s 8 --noise=0 ==> 0.98 MiB -q 93 -s 8 --noise=1 ==> 1.43 MiB -q 93 -s 9 --noise=0 ==> 1.02 MiB -q 93 -s 9 --noise=1 ==> 1.65 MiB Is this expected?

yes, noise and s >= 8 can do that

but please file a bug

👃

^^

Does the generated noise get subtracted from the image before encode?

so nitpicky 😛

nope

What causes the size increase then?

the butteraugli loop complains about the presence of noise that wasn't there in the original

Ah BA does see the noise, hm

Noise generation and objective metrics do not mix well

But good to know that now. -q 93 --noise=1 is now my go to setting, visually lossless except a very minimal blur when leaning forward (and comparing to the original).

I expect we'll fix it in such a way that it will be about the same size as -q 93 --noise=0

yeah i use -d 0.1 --intensity_target 65000 😄 it upgrade a little the precision 😄 (it's usefull for picture before being retouched and preserve detail in high light and low light and apply noise reduction. (it's why i'm searching extreme quality in lossy.

Look at the yellow part, you'll see chroma noise in VarDCT that doesn't exist in Modular... ...or am I crazy?

You're not crazy.

And colors in the sky, not only they're different and more variable, but they also exhibit some banding, while in Modular sky's mostly single-colored.

Yeah. At these super low bitrates, modular compression is doing some weird stuff. 😛

As some would say: it may look better and be more appealing to most eyes, but there are definitely wrong things in the image. 😛

<@!794205442175402004> I run video encoders benchmarks time to time, would it fit in this channel?

I thought this is for benchmarks relating to Jpeg XL 🤔

I can post all graphs in jxl <:whatisthis:672847617796997165>

I don't mind, but maybe better for <#805176455658733570> if it's really about inter-frame

If it's about avif it can go here

Ok

The CPU utilization hit on lossless for speeds slower than 3 is quite substantial:

It goes from using almost 700% to 202% at -s 4

I need to rerun these benchmarks with 0.3.0 though

Also, it's a bit of a memory hog compared to cwebp:

yeah, tree learning is sequential, so s >= 4 will not really be very much parallel (yet)

as for memory - we're aware of it (it's caused by keeping way too many copies of the image, with *floats* or *ints* - aka 12 bytes per pixel - around), and I'm working on improving that - but in the decoder first

Now I'm trying to compare compression on some emoji sets, I tried to take popular Twitch/BTTV/FFZ emoji (like <https://betterttv.com/emotes/top>), the problem is that most of them are animated and also to build a collection I would need to write a parser, so I gave up this idea and decided to take the Discord/Slack sets, like this: <https://emoji.gg/> <https://discord.st/emojis/> What is inconvenient is that many PNGs are actually GIF or Jpeg and have to do filtering of such, also some have very high source resolution, like 10000x10000 pixels.

😲

10000x10000 **emojis**?

animated jxl should work but probably cjxl is not doing something very good when you give it an apng as input

besides emojis, what might also be interesting are those small UI images

sometimes those are done using special fonts, but that only works if they're monochrome

Surprisingly APNG is not used for emoji on Twitch/Discord, mostly GIFs renamed to PNGs. UI images are hard to find, I tried parsing Google and other search engines, but parsers are quickly banned, also I would have to re-upload all these images and there are problems with copyrights. The free sets are very small and there isn't as much variety.

GIF renamed to PNG, why would you do that

Perhaps because Discord does not support APNG

yes but still, why rename to .png?

I know a guy who renames all audio files to *.mp3 🙂

He was complaining that some audio player didn't support his Ogg Vorbis and AAC files with *.mp3 file extension.

He just didn't like different file extensions for audio files for some reason

sounds a bit confusing to me

but ok, I don't care much about filename extensions, it's just a naming convention after all

That's why I wish JXL had a nice human readable signature.

To recognize it easily when you look at random data

Btw, Discord does not play audio with Opus extension, but renaming to ogg or mp3 works

0xFF0A is not human readable enough? 🙂

Your eye probably won't notice it when it is a part of another file.

it's one of the few things where we didn't get much choice, actually

JPEG _always_ uses markers that are of the form 0xFF [something]

Yeah, I know that it is because JXL is an ISOBMFF file.

well it's not always ISOBMFF, you can also use a naked codestream

that one starts with 0xFF0A

Oh, clear

Is there an other input file than apng to create an animate picture?

the ISOBMFF one starts with `0x0000000c` followed by "`JXL `"

gif...

ffmpeg can make apngs iirc, or you can use `apngasm`

Opus files are actually Ogg 🙂 *.ogg is usually Ogg Vorbis, and *.opus is usually Ogg Opus. Ogg is just a container.

Yep, but Opus is also a valid extension

There are YouTube tutorial videos for that <:kekw:758892021191934033>

They're just too cursed

~27000 emoji and only 120 MB total size, it will be hard for Google spreadsheets <:ReeCat:806087208678588437>

lol

are you also trying avif for those?

avif container overhead alone for 27k images with alpha would be about 10 MB 🙂

Yes, on another smaller (3630 emoji) set where there are a lot of big emoji too <https://emoji.gg/> ```129 079 046 (PNG) 144 478 511 (AVIF) 88 607 146 (FLIF) 93 595 588 (WebP) 90 341 593 (WebP v2) 114 674 943 (JXL S3) 97 612 982 (JXL S7) 90 631 695 (JXL S9) 85 545 674 (JXL S9 E3 I1) 89 144 669 (LEA) 77 566 600 (EMMA) ``` 2080 emoji set <https://discord.st/emojis/> ```54 395 097 (PNG) 66 655 300 (AVIF) 38 700 699 (FLIF) 39 465 618 (WebP) 38 376 865 (WebP v2) 52 720 966 (JXL S3) 42 305 826 (JXL S7) 38 638 890 (JXL S9) 36 264 120 (JXL S9 E3 I1) 37 223 272 (LEA) 32 501 495 (EMMA) ```

Not bad

WebP v2 is also good on such emoji (but very slow at the moment, still waiting to finish encoding for the first set)

I've found WebP v2 with effort 9 is an order of magnitude slower than the other effort values of WebP2 (at least for lossless) for a marginal (~2%) gain. Lots of room for optimization, hopefully

How is wp2 lossless at 10-bit images?

(and is it also limited to 10-bit like wp2 lossy, or does it support higher bit depths?)

16326 small emoji set (like ~28x28, ~128x128) <https://www.frankerfacez.com/emoticons/> ```163 267 422 (PNG) 175 449 368 (AVIF) 123 978 586 (FLIF) 122 555 150 (WebP) 117 540 435 (WebP v2) 137 334 778 (JXL S3) 144 491 331 (JXL S7) 135 087 171 (JXL S9) 124 852 846 (JXL S9 E3 I1) 113 288 808 (LEA) 103 821 345 (EMMA)```

Some interesting results, WebP was the winner, also `JXL -s 7` has worse compression than `-s 3`

Looks like there's room for improvement in the encoder then

What are LEA and EMMA?

s7 worse than s3 may indicate we're not properly taking some signaling costs into account

Research compressors with slow decode, unlikely to become useful as an actual image codec but nice to get better bounds at the actual information content of the images.

Ah, ok.

Are they image encoders or general data compressors?

https://encode.su/threads/2459-EMMA-Context-Mixing-Compressor

General, with specific models for images

And for other popular formats

LEA seems to be more of an encryption algorithm with compression on the side

WebP is doing well on these, also better than FLIF. Now I want to see what we're doing wrong here, could be something silly that can be avoided, judging by the diff between s3 and s7.

You scraped that website? Could you drop a zip/tarball with the set you used somewhere so I can try the same set?

aren't a lot of emoji sets atlased?

you mean like css-based sprite sheets?

uhhh, probably something like that, I think of it more from a GPU/rendering perspective

basically bundling a bunch of little images into one larger one

css sprite sheets are still used, but that's really annoying to maintain and use, and the main reason to do it is to avoid format-level header overheads and the overhead of lots of http/1 requests

with a codec that has a small header overhead, and with http/2, I hope there is no real reason to have to use sprite sheets anymore

not for web anyway

bundling little images into a big one is operationally very annoying – an image is no longer an object/file/uri anymore, but it is an object/file/uri PLUS some extra identifier and crop coordinates that need to be stored somewhere somehow

yeah, it's used in games, though typically they generate them during runtime with a packing algorithm

and then you want to add or remove little images from the big one and things get really messy

so looking at overall compression of a large atlas in any codec (other than those meant for GPU use) is kind of pointless

for games I think they often represent animation frames or other similar sprites, which means if you arrange them horizontally, PNG and WebP can get better compression because they can be relatively similar (enough to get lz77 matches)

but in that case it would actually make more sense to store them as animated png / animated webp / animated jxl

usually its decode -> pack/stitch -> compress (e.g. DXT) though

I suppose animation could help in some circumstances, really depends on the situation

for 2D games, a tile atlas can still make sense, especially if you have things like road segments etc that are cut in tiles but when arranged properly, things gets a bit more continuous (so easier to predict) than just the separate tiles

yeah, this is all for 2D

<@!794205442175402004> 3630 emoji set <https://emoji.gg/> ```129 079 046 (PNG) 144 478 511 (AVIF) 88 607 146 (FLIF) 93 595 588 (WebP) 90 341 593 (WebP v2) 114 674 943 (JXL S3) 97 612 982 (JXL S7) 90 631 695 (JXL S9) 85 545 674 (JXL S9 E3 I1) 89 144 669 (LEA) 77 566 600 (EMMA) ``` 2080 emoji set <https://discord.st/emojis/> ```54 395 097 (PNG) 66 655 300 (AVIF) 38 700 699 (FLIF) 39 465 618 (WebP) 38 376 865 (WebP v2) 52 720 966 (JXL S3) 42 305 826 (JXL S7) 38 638 890 (JXL S9) 36 264 120 (JXL S9 E3 I1) 37 223 272 (LEA) 32 501 495 (EMMA) ``` 16326 small emoji set (like ~28x28, ~128x128) <https://www.frankerfacez.com/emoticons/> ```163 267 422 (PNG) 175 449 368 (AVIF) 123 978 586 (FLIF) 122 555 150 (WebP) 117 540 435 (WebP v2) 137 334 778 (JXL S3) 144 491 331 (JXL S7) 135 087 171 (JXL S9) 124 852 846 (JXL S9 E3 I1) 113 288 808 (LEA) 103 821 345 (EMMA)```

WebP v2 is also good on small emoji

What settings are you using for WebP v2 <@!111445179587624960> ? Are you finding the encode times to be ~9-10x that of WebP for -z9?

`cwp2 -q 100 -effort 9 -mt 0` Yes, at the moment WebP v2 is unoptimized and very slow at maximum --effort

If you could send me a few example emojis where webp is better than jxl, that would be super useful

Yep, when I have free time I will add the results to a spreadsheet and pick examples where JXL performs worse

<@!794205442175402004> Emoji tab (I removed similar, mirrored and some other images and reduced the set a bit) <https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=1654200143> Since many emoji are more effective in WebP, I uploaded a whole set (it would be better to choose examples by spreadsheet) Emoji_FFZ.7z <https://drive.google.com/file/d/1bx0ZzpQgECMkDkzX_VquSNTowJlDphKP/>

Thanks!

Also, would -g 3 be better for maximum compression efficiency if not taking individual files but as an average overall result for different types of images or is it better to leave the default value?

Kind of depends on the kinds of images

Sometimes -g 0 is better, more local palette opportunities with smaller groups

For grayscale, probably -g 3 is best, at least on average

But e.g. an image with 600 colors, but only 5 different colors in many 128x128 tiles, would probably benefit from -g 0

Hmm, so this is not an option for blind selection, perhaps in the future it can be made automatic (although then the parallelism and cropped decoding may become worse) <:Thonk:805904896879493180>

Might be able to make some heuristics to select. For grayscale larger groups are kind of ok, since it's only one sample per pixel instead of 3-4

Could also of course just brute force all 4 -g settings and see what works best

Also, if an image is e.g. 300x300, it is kind of silly to split it in 256x256 groups. It will only hurt compression for almost no gain in parallelism

I added BMF for PixelArt and Emoji again (it does better on these sets than EMMA), by the way `JXL -s 3` beats even `-s 9` and `-s 9 -E 3 -I 1` on some emoji

Wut?

This is so funny, some heuristics are so much wrong

```$ ../tools/cjxl SillyW.png SillyW.png.jxl -q 100 -m -s 9 -I 1 J P E G \/ | /\ |_ e n c o d e r [v0.3.0 | SIMD supported: SSE4,Scalar] Read 64x64 image, 7.3 MP/s Encoding [Modular, lossless, tortoise], 4 threads. Compressed to 3433 bytes (6.705 bpp). 64 x 64, 0.01 MP/s [0.01, 0.01], 1 reps, 4 threads. ```

that was the worst one compared to webp

```$ ../tools/cjxl SillyW.png SillyW.png.jxl -q 100 -m -s 9 --palette 0 -I 1 J P E G \/ | /\ |_ e n c o d e r [v0.3.0 | SIMD supported: SSE4,Scalar] Read 64x64 image, 6.7 MP/s Encoding [Modular, lossless, tortoise], 4 threads. Compressed to 2024 bytes (3.953 bpp). ```

skipping palette apparently helps a lot

Some examples

no idea how to know whether palette is going to help or not – current heuristic is just to see if a palette of less than 1000 colors is enough and do that if it is possible

that is clearly not the best heuristic

Also on some `-s 9 -E 3 -I 1` is worse than only `-s 9`

Is only palette the problem? With palette=0 s 9 is better then s 3?

it depends on the image - I haven't spent any time yet trying to pick more sensible defaults for tiny images

by the way

do you check if lossless is actually lossless?

I think WebP2 forces premultiplied alpha, which means non-opaque pixels lose precision (and fully transparent ones become black)

JXL can also do that, but we don't have an option for it yet.

That puts a bit of an asterisk at the numbers for WebP2 – but please check, maybe I'm wrong and premultiplied is only done in its lossy mode or something

For WebP v2 and alpha is possible, as there have already been discussions on encode.su

Source

WebP v2

<https://encode.su/threads/3515-JPEG-XL-release-candidate?p=67656&viewfull=1#post67656> > **skal** > It is lossless (because of the -q 100 option indeed, which means 'lossless' in cwp2)... but in premultiplied world! > Means: cwp2 is discarding everything under the alpha=0 area, and pre-multiplying the other area, which could explain the difference you're seeing with 'pngdiff' if this tool is not doing the measurement in pre-multiplied space.

All I see is ¯\_(ツ)_/¯

atm cjxl preserves invisible pixels and doesn't premultiply RGB with alpha

we could add an option for premultiplied, the bitstream supports it

just for lossless I thought it was safer to just preserve everything

(for lossy we do mess up invisible pixels, which actually improves quality because invisible pixels can bleed into visible ones when doing lossy)

when comparing lossless with lossless, it's not really fair to compare premultiplied with non-premultiplied

Lossless = 0 loss

FLIF and I think cwebp are also implicitly doing tricks with invisible pixels

in flif you need to add `--keep-invisible-rgb` to preserve the A=0 pixels

and in cwebp you need to add `-exact` for that

I don't expect it to make a huge difference, just might explain some differences

the main silly thing cjxl is doing is making palettes of up to 1024 colors even for tiny images

where the cost of storing the palette is significantly larger than the gains of using a palette

going to change that default to be capped to `W * H / 8` if that is smaller than 1024

Yes, I think this is very useful, basically preserve everything is needed more for games and some specific things (and because alpha is always something artificial), but not for visually lossless or does it bring any other problems?

no it is fine to thrash invisible pixels, unless you want to have an alpha mask that you can still edit (and I mean not just shrinking the visible stuff)

so depends a bit on the use case

for web delivery or game assets, just thrash it

for authoring workflows where you still want to be able to modify the exact shape of the alpha (e.g. when you are manually cutting out a portrait and you accidentally cut off some hairs or an ear and want to put it back), thrashing RGB is not so nice 🙂

Because for normal images or smiles on the web, it's mostly an unnecessary or buggy alpha, like in that example (red dots on the edge)

yes, exactly – then premultiplied or at least destroyed invisible pixels is great

we allow both methods in jxl, premultiplied and normal alpha, it is signalled in the header

for PNG input, the encoder just always uses normal alpha at the moment, since that's what PNG also does

(for EXR input, the encoder uses premultiplied alpha, since that's what EXR does)

there's pros and cons and it's not something where you should force a choice on the end-user, imo

premultiplied means that very transparent pixels have basically no precision: A=1/255 means your originally 8-bit RGB becomes 1-bit

which can be good for compression and doesn't matter much if you're blending it in the same colorspace with 8-bit precision

premultiplied also allows "light emitting" transparent pixels (like flames or reflections) that have A=0 but RGB > 0, which is kinda cool

non-premultiplied preserves the full RGB image so you can "bring back" invisible pixels, which can be nice when editing (or can be a privacy problem in other scenarios)

This is also another problem for tests (but since, for example, my comparison is more focused on web content and casual storage and viewing, I think it is better to equate encoders to the WebP v2 behavior) <:Thonk:805904896879493180>

Makes sense

<:Thonk:805904896879493180> Also Avifenc has this option `-p,--premultiply: Premultiply color with alpha channel and signal this in the AVIF` I also did not enable it (but as long as there is no change for RGB in lossless mode, it does not help much)

I think I may have found another image that messes with the presets. S8

S9

PNG.

oh, that's funny

What does patch detection do with it?

Added some more emoji sets and yes, small images require some tweaking (faster speeds often show better results), also the option to enable premultiplied alpha would be useful (and for web content it can even be turned on by default, since this works well for webp/webp2) <https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=1654200143>

About premultiplied alpha: WebP https://i.imgur.com/gmRoaQ6.png

FLIF https://i.imgur.com/sOD7tHT.png

WebP 2 https://i.imgur.com/10IsMiF.png

AVIF https://i.imgur.com/ZYYjCub.png

WebP and FLIF only touch A=0 pixels

WebP2 and AVIF touch all A<255 pixels

which is fine, for web delivery

just need to understand there are 3 different games you can play: - keep invisible pixels (current cjxl behavior) - destroy invisible pixels (default behavior of FLIF, WebP and many PNG optimizers) - destroy invisible pixels and reduce precision of non-fully-opaque pixels (WebP2 and AVIF)

and comparing between these games is a bit apples and oranges, but you could also say cjxl should just get an option to play the other two games

Yes, I think it would be useful to have an option with different choices of how to work with alpha

In both WebP/FLIF and WebP v2 implementations, I have not found images that would differ visually, but AVIF can be too aggressive and such images no longer look lossless 🤔

Maybe they messed something up

Source

wp2

avif

I see border edges in the discord previews for wp2 and avif, not for orig

Nevermind

Now also for orig

Must be discord's fault

<:monki:750177061020106762>

Anyway, on phone now, cannot easily look, but I see a difference in the semitransparent pixels on the edge

E.g. at the monkey lips

Is that avif supposed to be lossless?

Does it perhaps need an extra flag to do the alpha lossless too or something?

It's a lossless AVIF with the `--premultiply: Premultiply color with alpha channel and signal this in the AVIF`

But I thought it would be like WebP v2

This is amazing

Monthly report: 5% better monke retention

This at the next JPEG meeting?

"So, Jyrki, what have you been working on?" "Optimizing the blocks." "Cool, and Jon?" "**Transparent Monke**"

<:Thonk:805904896879493180>

Yes, still need to add an option for premultiplied alpha

Did make some tweaks to avoid silly palettes and silly group boundaries, might help for the emoji set

Hello Scope, Could you tell me this image diff software name?

<https://github.com/joedrago/vantage>

With -s9 (-E 3 -I 1) the new build helped (mostly on the first set of emoji and less on the one added later), but on faster speeds -s 3, -s 5, -s 7 many emoji compressed worse (especially the overall size of the second set became larger)

really? that's somewhat surprising

only thing that should be different at -s 3 is that it now does -g 2 by default for some images, kind of strange that that would hurt compression at -s 3 though

I'll pick a few examples

https://tenor.com/view/mlg-illuminati-illuminati-confirmed-conspiracy-theory-gif-4887893

Strange, these are mostly Microsoft icons <:Thonk:805904896879493180>

wonder what happens here, at s3 I don't think anything changed for such small icons

at higher speeds it might now no longer do palette while it did before, which could explain the difference

Perhaps, -s 3 for the emoji set in the spreadsheet is mostly better (it's only worse on some of my additional sets where there are similar images as in the example above)

Examples where `-s 7` got worse (by 15% or more)

without monke what do we have left

its a clear path to superior imageing

somehow, splitting that one in 256x256 tiles compresses better than the new default of doing it in a single tile

<@794205442175402004> you must include monke compression in next talk you will do, please

<@!111445179587624960> which encoder/decoder did you use for avif?

<https://github.com/AOMediaCodec/libavif> `Version: 0.8.4 (dav1d [dec]:0.8.1-59-g0477fcf, aom [enc/dec]:2.0.1-1222-g886b981ef, rav1e [enc]:0.4.0 (p20210202-8-g165b9b5f))`

It seems like it premultiplies the RGB with alpha before encode, but then at decode doesn't undo that

so it produces a PNG with "premultiplied alpha" except no such thing exists

causing semitransparent stuff to become too dark

<:Thonk:805904896879493180> Btw, it would also be convenient for JXL encoder to show the git version

Like `[v0.3.1-ef3f7a62]`

looks like it's not just avifdec that does the wrong thing when decoding premultiplied alpha

also my chrome does that

<:Thonk:805904896879493180> Perhaps dav1d? (Chrome also has this decoder)

well it's the usual thing that happens

the decoder probably thinks the application will look at the field and interpret the data correctly

and the application probably thinks the decoder will have undone the premultiply and just give the usual RGBA data

There are also some issues with the YUV<->RGB conversion: <https://github.com/AOMediaCodec/libavif/pull/491> `Refine Premultiply alpha handling during conversion, update libyuv usage`

Can't we just behave like AVIF never existed? :)

AVIF is already supported in browsers, so it will have a long life (because the exclusion of any formats only happens when its support ended a long time ago and it was abandoned by almost everyone) If also consider that it uses the same decoder as AV1

Emoji has been updated with the new JXL build, and many new images have been added to the Pixiv set <https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=1654200143>

Btw, does `-I 1` affect the decoding speed?

it shouldn't (or at least not significantly)

-E 3 affects it a bit, the other modular options should have little effect on decode speed.

Ok, then I will also turn it on at maximum compression (in the next retest and when there are any noticeable changes in lossless JXL)

and for photographic

For lossy, I think --epf affects decode speed a bit, anything else <@179701849576833024> ?

--resampling

--noise

Generally speaking I think we have a relatively flat decode speed curve across encode speeds and quality targets

With default options I mean

Lossless image decoding speed is already quite good in multi-threaded mode (not as fast as PNG or lossy VarDCT, but enough for comfortable viewing)

--progressive also makes full image decode a bit slower

Typically codecs get slower the higher the quality, just because there's more data to entropy decode etc

I think in jxl it's more constant: lower quality has less entropy decode but bigger filters and bigger blocks which results in somewhat slower iDCT, higher quality has more entropy decode but faster transforms and less filtering

Unless, for now, there are no viewers that would first decode at least 1:8 previews to be able to quickly switch large images when viewing

It would make sense for viewers to do that, actually.

(we need a faster implementation of the fancy 8x upsampling btw, it's slow atm)

When browsing say 4000x3000 photos (12 megapixels), the 1:8 preview is 500x375, so doing fancy 8x upscale on that is not even needed when viewing on a typical screen - 4x is probably enough.

Does modular have a decoding speed penalty?

I notice that at similar bpp, modular is much slower to decode than VARDCT.

It's not just a penalty, it's almost two different (de)compression methods

True.

Modular is slower mostly because it has a more complicated context model, and because it needs to decode one value per sample (vardct usually has less because hf coeffs are often zeroes that are skipped)

Some more emoji examples on which `-s 3` is better (on the new build)

Pack

Also Manga page (-s 3 and FLIF/WebP/WebP2 have better compression) https://i.redd.it/5yqvn7szhhy51.png

`-g 3` helped, but still <:Thonk:805904896879493180> ```1 461 088 -s 3 1 565 266 -s 9 -E 3 -I 1 1 510 864 -s 9 -E 3 -I 1 -g 0 1 472 036 -s 9 -E 3 -I 1 -g 2 1 360 890 -s 9 -E 3 -I 1 -g 3 ```

Same: <https://i.redd.it/9s13kkm53mc41.png> ``` 993 091 -s 3 1 148 710 -s 9 -E 3 -I 1 1 137 560 -s 9 -E 3 -I 1 -g 0 1 003 257 -s 9 -E 3 -I 1 -g 2 911 118 -s 9 -E 3 -I 1 -g 3 ```

And the opposite case, on PixelArt content `-s 3` often get gigantic images https://i.redd.it/n7w9m7f7vqe41.png ``` 45 648 WebP 69 832 -s 9 -E 3 -I 1 65 113 -s 9 -E 3 -I 1 -g 0 74 709 -s 9 -E 3 -I 1 -g 2 78 280 -s 9 -E 3 -I 1 -g 3 993 091 -s 3 ```

Hello Scope in my test, if image less than 1024x1024, -g 1 and -g 2, both can get some benefit, i think -g 2 is fine, if image can lossless to png pal8(8bit), -g 2 is better, (use pingo -s0) if image bigger than 1024x1024 or grayscale, black(manga, comic), probably -g 3 is better.

that's... weird. <@!794205442175402004> any idea what could be going on there?

I guess the simplest answer is that our tree learning heuristic works worse than I expected 😄 or maybe grayscale has something off

it's not false positive patches

so must be that the fixed -s 3 predictor/tree works better than whatever tree gets learned

we don't try RCTs for grayscale, right?

i hope not

this is one of those images where you don't really want tree-based context modeling - ` -s 9 -I 0 -g 3 -P 0` gives you pretty good results (very close to webp, 47182)

In this

modular -Q90 il similiar size of -d 1 VarDCT and much better quality

can you configure your viewer to show NN upsampled pixels instead of some interpolation?

it's hard to see what are compression artifacts and what are viewer artifacts here

with the original

that's some ringing there...

About Pixel Art content, at slow speeds compression is good enough, but I mean the fastest speed and I understand that it will be worse, just now the difference can be very large, when it makes no sense to save such images in JXL. For example PixelArt-like style has a lot of indie games, and games often need to take screenshots (which usually use fast encoding modes) and it would be nice to have some fast but effective method for that type of images as well, because at the moment the difference looks like this:

And that's the kind of difference that almost always happens for content like this: https://files.catbox.moe/2mx005.png

(also pixel art can be in emoji, textures and UI elements, but on such content fast speeds are usually less often used than on game screenshots)

for pixel art, `-I 0 -P 0 -g 3` is a good choice - at `-s 8` it compresses about 1-2 MP/s, and at `-s 9` about 0.15 MP/s

maybe less actually at `-s 9`

but `-s 8` for that image is 170149 - with `-s 9` you can get it down to 140537, so about 20% less

you really need the -s 8 though, without lz77 it will be really bad

`-P 4 -I 0` is also not bad

or just `-P 4`, or `-P 5`

Yep, for slower speeds compression is acceptable, I'm mostly about the fastest speeds and also okay when the size will be 100-200% more, but now it is often 1000-5000%, which is a lot. Or it's hard to add any new methods and solutions for `-s 3` that would also work fast but be more efficient for pixelart-like content?

I get 136 KB on that image with just `-q 100 -P 4` or `-q 100 -P 5`

at 2.5 MP/s

`-s 3` is 11 MP/s but also 10x as large

Because at the moment pixelart is one of the few image types where JXL is not so effective as a replacement for PNG/WebP at fast speeds (and lossy is not good for that either)

maybe we could make a specific encoder option for this that really works well on stuff that is NN upsampled like this

that would be nice but in this case it's not actually a simple NN upscale. The sprites themselves are NN upscaled with different factors and can overlap each other.

Just call it pixelart

and make cjxl auto select it XD

Just for the context model, deviations would not be a big deal

https://files.catbox.moe/bbpl1l.png

``` 225 936 WebP 1 407 812 S3.jxl 542 861 S9.jxl 506 061 S9_E3_I1.jxl 549 390 S9_E3_I1_G0.jxl 442 273 S9_E3_I1_G2.jxl 385 637 S9_E3_I1_G3.jxl 473 473 S9_E3_I1_G3_P0.jxl 459 953 S9_E3_I1_G3_P4.jxl 1 194 044 S9_E3_I0.jxl 570 530 S9_I0_G3_P0.jxl 557 726 PNG ```

Can you try -I0 instead of -I1?

-I0 enables lz77 mode, basically

`1 194 044 S9_E3_I0.jxl`

`-I 0 -g 3 -P 0` maybe?

That created an 18MB image with that on -s 7 lol, as expected.

`570 530 S9_I0_G3_P0.jxl`

Ugh, lossless nonphoto is such a tricky game

Yep, but from what I've tested, JXL has mostly problems with pixelart and large noisy/grainy solid areas, it handles the rest of the artificial/nonphoto images quite well

For the pixel art we might need better lz77 matching decisions

What is used in WebP for pixelart can be used in JXL, or will it work badly because of the tiles/groups?

Groups reduce matching options

WebP also has a color cache thing that might help here

yeah on -s7 it would basically be uncompressed data 😄

I'm bruteforcing this, some results are so strange to me

On highly structured images, ones that can be done in < 1bpp losslessly, things do become strange

https://c.tenor.com/MoTJdGWGskcAAAAM/its-weird-weird.gif

https://c.tenor.com/sRx6btX50RcAAAAM/cow-utter.gif

Partial result

-I 0 -g 0 epic

tip: `-E 3` does nothing with `-I 0`

Thanks

If you want to brute force stuff, can try combinations with `-X 0 -Y 0`, `--palette 0`, `--palette 10000`, `--palette -10000` and see if that makes a difference

huh, what does a negative palette argument mean? I don't think it's explained in the help output.

It creates the palette in scanline image order. Default is lexicographically sorted after RCT, iirc (so sorted on ~luma in case of YCoCg).

There is potential for improvements there btw: there might be other ways to sort palettes for better compression

ahh

I'll add these

And about grain and noise, usually such photos are better compressed by WebP/WebP 2 https://i.redd.it/gpgryxvh4za41.png

```Webp 7 316 062 Webp2 6 617 432 S3.jxl 19 980 745 S9.jxl 7 565 880 S9_E3_I1.jxl 7 537 623 S9_E3_I1_G0.jxl 7 483 735 S9_E3_I1_G2.jxl 7 623 213 S9_E3_I1_G3.jxl 7 613 491 S9_E3_I1_G3_P0.jxl 7 344 182 S9_E3_I1_G3_P4.jxl 7 922 945 S9_I0_G3_P0.jxl 7 776 352 gpgryxvh4za41.png 7 878 628```

With that kind of noise prediction doesn't work very well -- better to rely on context modeling than prediction

could be that WebP is using the explicit entropy codes feature to manage this -- it is functionally equivalent to context modeling for this kind of data

it is good to note that that image does not have real sensor noise, but it is something weird that is artificially added

The winner seam

but webp win

if a large palette wins, chances are that with a better palette order and maybe with non-default entropy coding (prefix instead of ANS, so raw bits can be avoided), we can get smaller

A very smart patches finder can do amazing things in this

for lossy

palette 0 -> palette 10000 save 100KB on 320KB total

I found that a f*** big palette is even better

first time I've seen a palette size larger than 10k actually still help 🙂

https://i.redd.it/b4b0222wse441.png

```Webp 4 088 048 Webp2 3 671 143 S3.jxl 3 902 962 S9.jxl 5 178 651 S9_E3_I1.jxl 5 155 106 S9_E3_I1_G0.jxl 5 466 908 S9_E3_I1_G2.jxl 3 288 771 S9_E3_I1_G3.jxl 3 029 192 S9_E3_I1_G3_P0.jxl 3 895 939 S9_E3_I1_G3_P4.jxl 3 046 292 S9_I0_G3_P0.jxl 5 096 574``` Another example where `-s 3`/WebP/WebP2 is better than `-s 9` (and there will be two more examples with dots, noise, grain, also stars look like dots or grain)

that's very interesting and makes me wonder how much we're losing with not-so-optimal tree construction heuristics

https://i.redd.it/7ijkh752ueh41.png

```Webp 1 594 634 Webp2 1 543 869 S3.jxl 1 654 428 S9.jxl 1 988 170 S9_E3_I1.jxl 1 973 893 S9_E3_I1_G0.jxl 1 881 576 S9_E3_I1_G2.jxl 1 643 795 S9_E3_I1_G3.jxl 1 529 413 S9_E3_I1_G3_P0.jxl 1 617 225 S9_E3_I1_G3_P4.jxl 1 556 553 S9_I0_G3_P0.jxl 1 940 640```

ah - can you try with `--patches=0`?

`S9_patch0.jxl 1 988 170` <:Thonk:805904896879493180>

these cases where s3 is worse than s9 are interesting, we should make a corpus of them to be used together with more normal images when we are tweaking MA construction heuristics

https://i.redd.it/wcuh6398ts741.png

```Webp 1 820 998 Webp2 1 693 995 S3.jxl 1 824 177 S9.jxl 2 027 991 S9_E3_I1.jxl 2 001 808 S9_E3_I1_G0.jxl 2 141 836 S9_E3_I1_G2.jxl 1 709 946 S9_E3_I1_G3.jxl 1 616 329 S9_E3_I1_G3_P0.jxl 1 798 997 S9_E3_I1_G3_P4.jxl 1 641 709 S9_I0_G3_P0.jxl 2 126 845```

Also an example with solid color areas (most likely WebP/WebP2/PNG have advantages because they do not use tiles, but I often see such images with similar results) https://i.redd.it/7ztcigvejte41.png

```Webp 1 064 254 Webp2 1 010 339 PNG 1 313 281 S3.jxl 2 521 614 S9.jxl 1 406 928 S9_E3_I1.jxl 1 385 563 S9_E3_I1_G0.jxl 1 311 589 S9_E3_I1_G2.jxl 1 150 293 S9_E3_I1_G3.jxl 979 040 S9_E3_I1_G3_P0.jxl 1 233 538 S9_E3_I1_G3_P4.jxl 995 951 S9_I0_G3_P0.jxl 1 328 226``` However, -G 3 usually helps on images like this

But how it is possible?

So, for something like -s 10 would work `-s 9`+`-E 3 -I 1` and a bruteforce (or some kind of prediction) between the default `-g` and `-g 3` (choosing between the other settings rarely gives a stable improvement) Hmm, I wonder if it's possible to variable group sizes, e.g. encode large blocks and when it's more efficient split it into 2x2

nope (well, not without doing it with multiple coalesced frames)

multi frame encoding would allow variable group sizes (kAdd blended together), or I could even imagine an encoder that segments the image in different subrectangles (that can have arbitrary sizes and offsets) that are encoded with different settings

`-I 1` on this image is less efficient https://i.redd.it/dgfpiwvmghh41.png

very interesting results here 😄 here it's same algorithm but on all the data instead of a sample -> worse results...

it's not completely surprising. we don't accurately take the signaling cost of the tree into account, right? so it could well be making a better tree that makes the encoded data smaller but what is saved is less than the extra size of the tree itself

that's possible but >500 byte seems excessive here, I suspect it more likely trips the heuristics in a bad way...

how does the current tree learning work again? it's an eager optimization that looks for a property+splitval that minimizes the estimated entropy of both subgroups, then recursively does the same within each subgroup?

yup

with more compute (-s 11?) we can make it decide a few splits along the same property at the same time

it's a complicated clustering problem

we get away with some bad decisions because there's still cmap afterwards that can merge stuff again

but in general, eager clustering is not going to be optimal

not that I have any better ideas

I compared `-g 3` on the one set, if it consists of large images it gives a better overall result: <https://docs.google.com/spreadsheets/d/1ugFqODnDnyeHbww9g4kKsqk--VKvosMQ2YYd1YN5KLA/>

But, such images are worse with `-g 3` https://i.redd.it/hdspaupemqb41.png

https://i.redd.it/k82xfv6wbfi41.png

Could be more local palette opportunities in that last image

https://i.redd.it/mcjesaqusa141.png

```FLIF 1 365 303 <- EMMA 1 200 839 S3.jxl 1 455 473 <- S9.jxl 1 553 917 S9_E3_I1.jxl 1 539 684 S9_E3_I1_G0.jxl 1 517 982 S9_E3_I1_G2.jxl 1 535 278 S9_E3_I1_G3.jxl 1 559 034 S9_E3_I1_G3_P0.jxl 1 755 102 S9_E3_I1_G3_P4.jxl 1 611 902 S9_I0_G3_P0.jxl 1 990 116 S9_pal0.jxl 1 307 882 <-```

https://i.redd.it/c4o4edl7zli41.png

```FLIF 548 773 <- EMMA 507 193 S3.jxl 637 473 <- S9.jxl 650 562 S9_E3_I1.jxl 645 720 S9_E3_I1_G0.jxl 635 849 S9_E3_I1_G2.jxl 662 648 S9_E3_I1_G3.jxl 659 831 S9_E3_I1_G3_P0.jxl 712 231 S9_E3_I1_G3_P4.jxl 699 549 S9_I0_G3_P0.jxl 785 872 S9_pal0.jxl 544 944 <-```

`-S9_palette_0.jxl 1307882` <:Thonk:805904896879493180> `-S8_palette_0.jxl 1361468` `-S8_lossy_palette_0.jxl 1157640` `-S9_lossy_palette_0.jxl 928455`

For black and white pictures(like manga), it looks like --palette=0 is usually the way to go.

Now, is lossy-palette a lossless operation for monochrome images? 🤔

It doesn't auto-detect monochrome images? <:WTF:805391680538148936>

Lossy palette is not lossless

It does delta palette, which is near-lossless

Thanks for the examples, looks like modular encode makes some silly default decisions for these images.

Comparing results with `-s 9` and `-s9 -E 3` with and without `--palette=0`: <https://docs.google.com/spreadsheets/d/1ugFqODnDnyeHbww9g4kKsqk--VKvosMQ2YYd1YN5KLA/edit#gid=584197871> Some more examples (better with --palette=0): <https://i.redd.it/msakckaxtyt41.png> <https://i.redd.it/a1fok4ffenc41.png> <https://i.redd.it/a75aq91wi0251.png> <https://i.redd.it/o2gft9yymuz51.png> <https://i.redd.it/9h3c6u0d86r41.png> <https://i.redd.it/hgn92pwpfwu51.png> <https://i.redd.it/nm94tj2rwsf41.png>

-X 0 and -Y 0 might also make a difference in some cases

Those are for channel palettes

E.g. if a 8-bit image uses only 170 different values of R, then default will do a palette just for the R channel, to reduce its range from 0-255 to 0-169

-X 90 means "do this if only 90% or less of the nominal range actually occurs"

-X does this globally, -Y does it per-group