JPEG XL

The default behavior for palette and channel-palette is probably still quite stupid, especially in case of grayscale

I assume ultimately it will detect when it makes sense to use it (and enable it automatically in those situations)?

Update with `-s9 -E 3 -X 0 -Y 0`: <https://docs.google.com/spreadsheets/d/1ugFqODnDnyeHbww9g4kKsqk--VKvosMQ2YYd1YN5KLA/edit#gid=584197871> Examples (better with `-X 0 -Y 0`): <https://i.redd.it/ew0krp5u4r641.png> <https://i.redd.it/xptmu1db53o31.png> <https://i.redd.it/9snkif62a2x51.png>

JXL (30 482 507) ```Read 6720x4480 image, 27.1 MP/s Encoding [JPEG, lossless transcode, kitten], 8 threads. Compressed to 30482507 bytes (8.100 bpp). 6720 x 4480, 16.25 MP/s [16.25, 16.25], 1 reps, 8 threads.``` Brunsli (30 515 294) ```6720 x 4480, 8.20 MP/s [8.20, 8.20], 1 reps, 1 threads.``` <:Thonk:805904896879493180>

Usually brunsli will be a bit smaller because of its dedicated DCT8x8 context model, but it depends. On 4:4:4 jpegs, jxl can be a bit smaller than brunsli. JXL has potentially better DC encoding, and for 4:4:4 it can use chroma from luma also for existing jpegs, which brunsli doesn't do.

Has anyone done JXL benchmarks using these? https://github.com/FLIF-hub/benchmarks

Not afaik

Hmm, yep, that fixes AVIF premultiply alpha https://github.com/AOMediaCodec/libavif/pull/491

AVIF

AVIF (without fix)

so this there are invisible pixels

like jpeg xl

<:monki:750177061020106762>

<@!794205442175402004> FLIF has no premultiply alpha options like AVIF and WebP v2?

Although it might be better to do it in the PNG optimization step, so that all encoders are on equal footing (but I can't remember which optimizers do it right)

no, FLIF only does non-premultiplied alpha, like PNG

but it does by default not encode the RGB of A=0 pixels

so there's that

by the way, WebP2 is supposed to be named "WebP2", without space and "v"

it's not a version 2.0 implementation of WebP

Yep, in the beginning it was called that and I got used to it

it's a new codec confusingly called WebP2

at some point there might be a libwebp v2.0 and a libwp2 v1.0

things will _really_ get confusing then 🙂

The numbers are somewhat confusing, unlike letter designations like Jpeg XT and Jpeg XL (although people often get that confused as well)

yes, I don't really like JPEG's naming choices

JPEG XT was actually backwards compatible so for that one, JPEG 2 would maybe kind of make sense

XR (wdp / HD Photo) is supposed to mean "extended range" but JPEG 2000 actually supports higher range than XR

I think XT is supposed to mean "extensions"

XS is supposed to mean "extra speed" or something, as it is aiming for ultra-low-latency/complexity

and then basically it became a 'tradition' to name things JPEG X<something>

I don't know how they came up with XL

maybe it was a joke, t-shirt sizes XS - XL or something

Touradj once told me the L can mean "longterm"

Yep, L - Long-term seems logical (for the conditions that were presented for its acceptance)

it's still somewhat confusing that they call all their codecs "JPEG <something>"

and they also call themselves JPEG

Is only the contribute of jpeg group to the meme world

and there still is no easy unambiguous way to refer to the old jpeg (10918-1, or rather 18477-1 which is the formal standardization of the de facto standard)

some say "legacy JPEG"

some say "JPEG-1"

some say "JFIF"

whatsapp use .jfif

where do you see filename extensions in whatsapp?

Whatsapp web

strange, other image are .jpeg, only a lucky case?

Chrome sometimes saves JFIF instead of WebP

Maybe I should change LEA back to BMF, my idea was what results are possible with LEA's fast encoder (faster than JXL's default -s 7 and sometimes -s 5), while it is better in compression than -s 9 (-E 3), but it has symmetric decoding speed, with still not enough for practical use <:Thonk:805904896879493180>

Animation comparison for <@!532010383041363969> and <@!794205442175402004> , here is the APNG original:

jxl:

WebP (like VP8)

and VP9:

what distance did you use for the jxl?

I used rgb for VP9, since WebP already represents YUV420. Size target was 64 KiB, but I don't quite remember which exact quality number of each codec resulted in that size.

But JXL looks really nice while the other two create blocking in the out of focus parts and thin lines or fine color edges get blurred.

I see jxl ringing around the nose

I only see that if I push my nose against the screen. ^^

jxl is ringing quite strongly around many of the edges, but it does avoid banding/blockiness

the banding of vp8 and vp9, especially in the darks, is quite noticeable

video codecs really suck at dark browns

I would almost say they are racist

Better than VarDCT? (first frame only):

-m -Q 70 -s 9

Some examples of pixel art compressibility and also excessively large `-s 3`: https://i.redd.it/uo9u57i7opj41.png

```BMF 11 032 WebP 14 294 S3.jxl 1 618 449 <- S9.jxl 29 347```

https://i.redd.it/b5t7kfq0w2141.png

```BMF 4 024 FLIF 14 038 S3.jxl 253 144 S9.jxl 21 348``` BMF <:Thonk:805904896879493180>

https://i.redd.it/41xxebbzfak41.png

```BMF 792 WebP 1 170 S3.jxl 10 736 S9.jxl 3 440```

https://i.redd.it/e9vgojjx5fe41.png

```BMF 29 404 WebP 87 202 S3.jxl 790 306 <- S9.jxl 101 239```

https://i.redd.it/5ixlfk5xm7f41.png

```BMF 36 436 WebP 53 410 S3.jxl 1 289 528 <- S9.jxl 61 481```

`S9_g3_I1.0.jxl 59491`

interesting stuff

s3 does really poorly on such big-block stuff

https://openbenchmarking.org/test/pts/jpegxl-decode it would be interesting to get some ARM numbers

https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2021_02_02/index.html#air-force-academy-chapel-colorado-springs-co-04090u-original&WEBP2=l&JXL=m&subset1

vs

https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2021_02_15/index.html#air-force-academy-chapel-colorado-springs-co-04090u-original&WEBP2=l&JXL=m&subset1

small improvement on the worst image

WEBP2 is quite a lot stronger on this image

I'll probably be able to make a similar improvement in the next two weeks

This is a f**** ammount of less ringing and more details

Webp2 sky is literaly make of squares

WebP2 is compromising the sky -- it is a natural consequence of the YUV/RGB color space

we see this is we optimize neural codec against the XYB or RGB

RGB neural codec puts less bits on the sky and on red objects than an XYB neural codec

plots can be interesting, too

https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2021_02_02/plots.html

https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2021_02_15/plots.html

nothing is better than looking at the images, of course 🙂

the 2021_02_15 version includes jxl 0.3.2

2021_02_02 version includes jxl 0.3.0

jxl 0.3.2 has image improvement to 0.3.1

?

as far as I know 0.3.1 has severe problems with some colorspace mistyped as XYX (instead of XYZ), and should be avoided

i converted some files with 0.3.1

I suppose it typically should only affect lossless? In lossy we don't typically store compressed icc profiles, just a colorspace enum which is not affected

i haven't encoded anything with lossless in 0.3.1

Probably fine then - best to check if things you encoded with 0.3.1 do decode in 0.3.2, if not we could probably make a "bitstream fixer" tool

https://i.redd.it/314b01jokhh61.png ```FLIF 600 112 BMF 193 140 WEBP 196 524 S3.jxl 4 640 180 S9.jxl 994 491 S9_E3_I1.jxl 1 261 993```

Patches go brrrr

Interesting image to try better patch heuristics with

Why jxl is so bad in this?

It probably currently doesn't detect any

Because lz-based codecs will do automatic deduplication

we don't have this category of images in the test corpus

Try `-I 0 -s 9 -g 3`?

I believe that will be able to eventually use our palette coding for making complex patterns compress very well

but for those images patches could work, too

just the palette with context modeling is incredibly powerful for simple repetitive patterns

basically we could code a say whole 11x17 pixel pattern into a single palette entry

just trigger the rest by context modeling

Could be done, but patches will probably also work well here, if you can detect all occurrences

is that a crazy bunny made out of minecraft blocks

Also Web2 is worse than WebP on this kind of pixel art content (but consistently better on other types)

https://discord.com/channels/794206087879852103/803645746661425173/811316681544171590 Interestingly enough, -s 9 does worse than -s 8 on that image. `s8_g3 834915` `s9_g3 855966`

https://i.redd.it/qs0p97ahf8k51.png ```FLIF 237 281 BMF 37 948 WEBP 57 718 S3.jxl 3 755 790 S9.jxl 76 990 S9_E3_I1.jxl 76 351```

A fast test for individuate pixel art (n×n) for esample spartial color correlation (to individuate n) and then force palette of n×n pixels or paches?

Images like this mostly because I'm in the process of increasing the PixelArt set (the previous one was very small), although I don't think pixel art should be a priority right now (especially if it requires big changes), there are more important things, but this is also one of the weaknesses of JXL's lossless mode so far, especially at fast speeds, and lossy compression is not the best choice for this content either

https://i.redd.it/ze1j32m9lka61.png ```BMF 103 108 WEBP 131 228 WEBP2 140 481 S3.jxl 646 053 S9.jxl 309 170 S9_E3_I1.jxl 247 157```

We need to add a fast mode for non-photo that maybe doesn't do great on these but at least not terrible

https://i.redd.it/ky96oyujjtg61.png ``` BMF 3 400 WEBP 1 160 WEBP2 2 935 AVIF 3 380 PNG 2 293 <- S3.jxl 146 817 S9.jxl 6 176 S9_E3_I1.jxl 6 137 S9_I0_G3_P0.jxl 2 153```

https://i.redd.it/mkqqi5ly31c61.png ``` BMF 20 144 WEBP 14 868 WEBP2 36 852 AVIF 15 772 PNG 27 310 S3.jxl 66 631 S9.jxl 29 918 S9_E3_I1.jxl 29 923 S9_I0_G3_P0.jxl 22 959```

Repetitive stuff is great for compression but we mostly don't play that game yet. Bitstream has many ways of doing it though.

Try `-P 0 -I 0 -g 3 -s 9` maybe?

Sometimes even just -g 3 helps on such content (it would be nice to find signs of its efficiency on the image automatically without bruteforcing, at higher resolutions it works more often, but also not always)

https://i.redd.it/nikixial60s21.png ``` BMF 148 WEBP 1 166 PNG 1 036 S3.jxl 4 287 S9_E3.jxl 361 S9_E3_I1.jxl 184 ```

`-I 1` <:Thonk:805904896879493180>

https://i.redd.it/uszh7pkxda141.png ```WEBP 70 676 WEBP2 347 475 PNG 119 814 S3.jxl 4 367 097 S9_E3.jxl 34 254 S9_E3_I1.jxl 14 477```

where is this image from?

I mean, how is it generated

Idk, /r/fractals or something like that, from reddit

I guess it's generated by some simple rule, but can't tell on the top of my head 🙂

Tree learning will figure out the rule

Also https://i.imgur.com/UL1fXgB.png ```WEBP 14 410 WEBP2 50 011 BMF 12 940 PNG 113 542 S3.jxl 1 157 680 S9_E3.jxl 54 340 S9_E3_I1.jxl 54 605```

that's one where turning on/off patches and -I 0 will make a big difference (but I don't really know in what direction :P)

In this webp is sick!

Yep, also Lossless WebP in general is well tuned over these 10 years (it has very stable results on all kinds of content)

-I 0 is bad in this

Jyrki made a really great job with WebP Lossless...

But JPEG XL has *WAAAYYYY* more potential

just pay more Jyrki

He turned to the dark ||lossy || side

On average `-s 9 -E 3 -I 1` gives 0.25% better efficiency than just `-s 9 -E 3` 🤔

not unexpected, by default we learn the tree with 50% of the pixels, so I don't expect *huge* gains from doubling the samples

And up to 2-3% on something like pixel art

changing the group size and/or activating lz77 will probably make a bigger difference

<@!179701849576833024> do we have a way to do *both* lz77 and MA trees in the current encoder? or is it one or the other atm?

well.. in theory we currently learn a MA tree, and then try to use lz77 to figure out if there is any entropy left to reduce. This is in many ways not optimal at all though...

Thank you :-D, warms my heart to hear this!

I'm sure that's desirable in Finland right now.

But really, back when JPEG XL wasn't well developed yet, I was really planning to run `cwebp -z 9` on ALL of my ~15k PNG phone screenshots because of storage space savings, i could save GIGABYTES of space. Unfortunately I didn't have time and resources so I didn't encode more than a test suite from most recent screenshots.

It seems like it was a good decision to wait though, JPEG XL's patches probably gonna save even more space because those screenshots are text and images, and the former compresses incredibly well with patches.

Actually, that reminds me.

We need MTed lossless modular. 😛

I want to start using JXL for screenshots.

Wait, it's single threaded currently?

From what I've seen, -s 4 and above are not threaded nearly as well as -s 3.

if you want to compress a large set of images, you're likely better off with per-image parallelism - just saying 🙂

otherwise, yes, tree learning is single threaded

I mean, it's not an issue with post-processing(I can just use GNU parallel for that), but I'd like to take them in JXL in the 1st place.

one of those things that are not *too* hard to fix, but they take time that I currently don't really have

Ok, at least it works 😃

However, VARDCT already works well for screenshots, and it is quite speedy.

I'd love it if Spectacle(the KDE screenshot utility) allowed you to change encoder settings though.

That'd be nice.

For lossy ones, yes. Current lossless PNG screenshots are quite big, but I've got an idea to compress that thing better.

Let's say I'm doing a Facebook post screenshot. A JPG photo and some text.

If we *somehow* tap into JPG decoder, then we probably can directly copy-paste JPG DCT coeffs as VarDCT 8x8 and encode the text with patches.

Will it work?

Btw, on older builds (before major changes in lossless and smaller group sizes) lossless JXL paralleled very well, but the encoding speed was much slower and it took me weeks to update comparison results for all sets

Yeah, I remember the sub 0.1MP/s speeds with S9.

*in theory* it should work. Writing an encoder for that might be really hard though...

Yep, I'm afraid that the lossless VarDCT coeffs copy-paste will be the hardest.

at some point in the past tree learning was ran per group and not per image. This meant a lot more parallelism but also more compute overall 🙂

that's more or less already the jpeg recompression encoder path, so that should be doable - splitting text and non-text, on the other hand...

Splitting text should be easy if the OEM integrates the JPEG XL encoder deep into the OS (e.g. Samsung is known for deep integrations, like S Pen software and One UI in general)

For example this screenshot:

It's gonna be really tricky to grab the original JPEG file (shown in the middle), but if deeply OS-integrated encoder *somehow* manages to do that, then it's gonna be easy

Just grabbing the JPG will be a nightmare

I guess you could integrate it in the system graphic libraries, but that's not going to be trivial either 🙂 and I don't see that happening anytime soon...

The text above the image? The OS should be able to forward the info to the encoder "this is not the image, this is text". But even if it doesn't, encoder should manage to detect those patches on its own, it's quite easy case.

Profile picture (near `Jacek Gladysiak` text)? Either VarDCT (lossy) or Modular (lossless). Unfortunately the picture is processed way too much to be losslessly recompressed. Icons (below the image and in other places)? Modular.

So, what you essentially want to do is Screen Content Detection?

I think this is overly complicated at different levels of integration, but even having just lossless/lossy mix compression is good https://youtu.be/MBVBfLdh984?list=LL&t=1336

Yep. So the encoder doesn't have to do all the work from scratch, but simply grab already prepared info from the OS in order to improve e.g. patches detection (just grab the literal text strings from the OS) and the image quality (e.g. for JPEGs shown at 1:1 scale and with little to none processing)

Another thing to encode on that screenshot: the crop of the previous image just below the group top bar.

Of course we can re-encode the thing, but is it possible to grab the original JPEG file, extract just the lowest blocks (the one we see here, **including those we see partially**), encode them losslessly and then overlay the top bar over the partially visible 8x8 blocks (because it's unlikely for 8x8 blocks to *always* align their boundaries with the top bar)?

But, this can be a task for various even third-party pre-processors whose main task is to select the needed parts of the data and e.g. images, which then compress in the best way for them

I've got *lots* of usage ideas for JPEG XL. Remember that image with potential for patches. I'm about to ask lots of questions for core devs about patches and then write yet another lengthy essay about possible improvements!

But how can you perfectly restore the original JPEG coefficients? It's already quite hard, and when 8x8 blocks are only partially visible (e.g. the previous image in that screenshot just below the group's top bar), the task becomes literally impossible without the data grabbed from the app or OS.

patch detection at the moment is pretty much a horrible hack 🙂 there's lots of room for improvements

I know. But that's a topic for another related essay: possibility for manual construction of perfect JPEG XL files current encoder can't create on its own *yet*.

it's pretty hard to undo ycbcr and idct and get back the dct coefficients from it in a lossless way without having access to the JPEG directly - especially if there was anything like rescaling done it's probably impossible

That was the thing I was talking about to Scope. We need deep OS integration for grabbing original JPEG files.

Yep, it's just as challenging for any other application or utility as it is for JXL and requires deeper integration to know exactly what type of data it is

But don't worry, it's possible. Samsung somehow integrated S Pen deeply into OS.

I'm living in Switzerland for 14 years... for the first four years I didn't even have a winter jacket, but now I'm acclimatised

Ah, nice. I've lived in Zurich for 4 years (now I'm in the US).

Yes. My work on improving the chapel has at least partially landed.

Speaking of lossy JXL, one of the main problems I often hear from people (including Lee from here) at the moment is noticeable artefacts, ringing and jagged lines in art images (even at not very low bpp like 0.5-1 and higher)

And from conversations with many people, the main complaints were from those who are involved in art-like content and AVIF impresses them more (while with photo content, most are happy with JXL results) My opinion, and as many services do, is to store such images in full resolution in lossless, and previews and lower resolutions in lossy (lower resolutions are usually viewed less closely and small artefacts are not as important there or if AVIF wins there significantly, then use a mix of formats for now) But, this is a solution that not everyone agrees with, some do not want to use many different formats (or when they are not very supported), some can not keep lossless images (due to traffic and internet speed of most visitors), etc.

I'm testing latest build and... wow. When Chromium JPEG XL support will be ready, I'll create a support topic for Google Photos team and I'd like to suggest them some things, e.g. recommended Butteraugli distance to use that'll give the best quality/size ratio.

I'm testing `-j -p -d 2 -c 0 --noise=1 --dots=1 --epf=-1 --gaborish=1` with some photos grabbed from my dad's crappy temporary phone (common use case) and... wow.

This is one photo:

And that's another:

Challenge for y'all: tell me which one is original JPEG and which one is `-d 2` JPEG XL.

https://tenor.com/view/good-luck-liam-neeson-taken3-adventure-action-gif-4666091

Oh, and by the way: JPG - 3.45 MB JXL - 884 KB Let that sink in.

The low camera quality makes it very hard

8q4ocr7nasoi.png - Source ou4q3kjlds2p.png - JXl (it adds a bit of sharpening)

PARTY POOPER

Lol

But this is without zooming and quick flip comparison

8q4ocr7nasoi.png does look like aggressive shitty phone camera denoising

But I think that for an inexperienced eye (e.g. my dad, who took the photo and other Google Photos users) the difference is insignificant enough

You'll probably agree that both are quite pleasant to look at, compared to what you'll see in a while

Oh no <a:dogescared:749458017954562058>

OH YES

<a:CatNo:806636752810409986>

1.32 MB, Google Photos encoded JPG. Look in the shadows and burn your eyes afterwards.

Yep, but it also depends on the source photos quality (on higher quality photos it is easier to see the difference, for example with zooming I often distinguish -d 1)

Lots of people are shooting with mid-range photos

And even if they're using flagships, vast majority of casual photo takers probably still won't notice

I haven't heard neither my mom nor dad complain about Guetzli JPG encoding

But on such, it is not very noticeable to the naked eye (I can sometimes identify JXL by certain types of changes it makes, but that would be some cheating, also I use -d 1 more often as I notice its changes and much less use stronger compression except for tests)

Now they'll have smaller files AND better photo quality (that they won't notice anyway)

My first encodes done 3 years ago, just after joining Google Photos are quite decent

I don't know why, but the quality of current encodes is... decreasing? Maybe I'm crazy. Maybe Google lowered encode quality just for me.

But even on my phone and on my laptop in online viewer I can see those ugly 8x8 DCT blocks

Just look in the shadows in the Guetzli encode above

And -d 1 sometimes adds a little more sharpness, so I thought so when choosing and it seems that the encoded image even became "better" (but longer distances, on the contrary, lose detail and sharpness)

However, I identified a more blurry photo (just inferred incorrectly from this due to frequent use -d 1)

About Guetzli quality, perhaps over time there have been some "optimization" when this quality is enough for most, at first it is usually always with a margin (such as AV1 video bitrate on YouTube) and later tried to find the golden mean between quality and size

Or maybe Google decreased quality globally because they've had storage spaces problems and wanted to squeeze even more photos in limited disk space. That turned out to not be enough, hence the June 1st free tier apocalypse...

I don't think Google has a real problem with space, considering how many people are uploading videos to Youtube, including private videos or with a couple of views, some even use Youtube as a free VP9 encoder or to store their terabytes

Even the traffic with just previews on YouTube is more than all the other image services (btw, as far as I understand one of the main reasons to start developing WebP)

I'm pretty sure Google's interest is in reducing bandwidth primarily, not storage.

Hmm, I wonder why the sudden growth in 2020? 🤔

Can someone benchmark this?

Yet somehow they decided to ditch Google Photos free tier... maybe because 100% free storage for everyone isn't a viable business model even for a giant like Google? 😉

I get jxl bigger than png on this one

I've also got lots of screenshots with articles from a paywalled newspaper, can I add some to a test suite?

why ask me?

Not you in particular, it's more like open question

I don't want to spam here with my screenshots if you've already got plenty enough

Ok, s9 got it down to 775, and it took a while, like 5 minutes `3840 x 2160, 0.03 MP/s`

That's huge difference beetween s9 and s8

```C:\Users\zieme\Downloads\jpeg-xl-mingw64-e2b1e60f> .\cjxl -s 9 .\reading.png .\reading.jxl J P E G \/ | /\ |_ e n c o d e r [v0.3.2 | SIMD supported: SSE4,Scalar] Read 3840x2160 image, 47.2 MP/s Encoding [VarDCT, d1.000, tortoise], 2 threads. ..\lib/jxl/image.cc:102: JXL_CHECK: bytes_.get()``` OH HELL NO

NOT AGAIN

you didn't set lossless

Like if it mattered... ```C:\Users\zieme\Downloads\jpeg-xl-mingw64-e2b1e60f> .\cjxl -m -s 9 .\reading.png .\reading.jxl J P E G \/ | /\ |_ e n c o d e r [v0.3.2 | SIMD supported: SSE4,Scalar] Read 3840x2160 image, 44.5 MP/s Encoding [Modular, lossless, tortoise], 2 threads. terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc```

Idk, everything works for me

It takes a lot of RAM for these resolutions

It's either `..\lib/jxl/image.cc:102: JXL_CHECK: bytes_.get()` or `terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc`

6.0/7.9 GB (76%)

2140MB for me

You mean 2140 MB used by OS and apps?

The most RAM-hungry apps currently

2140 MB for cjxl

And no, don't ask me how many Chrome tabs I've got opened

Ah, OK

Because my OS and apps utilize 6 GB currently

And if crop a small part of the image, will the encoding work?

I'm trying to reduce memory use right now

I can't go beyond `-s 4` or `-s 5`, otherwise it throws one of those errors

If I'm successful, it is about 3 bytes less memory per pixel at some stage of the compression 😄

Especially if I explicitly enable `--patches=1`, then even `-s 3` is impossible

`[v0.3.2 | SIMD supported: SSE4,Scalar]` <:Thonk:805904896879493180> Then maybe there is something wrong with the SIMD?

probably we allocate 222 bytes per pixel or so, improvement comes in small small drops

I don't think so, I'm using your latest build from encode.su, if you haven't guessed that yet from my command listings

I've got the exact same `SIMD supported` on my own previous WSL2 compiles

I mean if the CPU does not support AVX and stuff, otherwise I had no problems except for RAM (but I also often encode a lot of images in parallel)

I like to think that there is no Google's interest, no single person is that represents Google with special interest.

Here's a CPU-Z screenshot, my CPU should support AVX

Motivation is more like a bunch of engineers decided to build it, and more senior engineers decided not to stop them.

<@456226577798135808> Not familiar with the JPEG-XL encoder codebase, but usually it's AVX2 that matters (as it adds 256-bit registers), not AVX.

Is there anything better than this `-s 9 -E 3 -I 1`?

My primary motivation was to build a present for humanity -- less so for making computers cheaper to run, and more so for reducing wait times that people have for the internet.

Then unfortunately it seems like my CPU doesn't support AVX2 🙁

Noble goal, thank you 😘

<@!532010383041363969> by the way have you seen my message on <#794206170445119489>?

You just missed it. Haswell has AVX2. Ivy Bridge is the generation just before it.

I'm actually thankful that this laptop can run programs in reasonable time at all, I've bought it used for 800 zł (~215 $)

Fortunately I bought a version with small (128 GB), but an SSD

`[v0.3.2 | SIMD supported: AVX2]` Btw, I compile a more optimized build for my CPU, are there any disadvantages when only AVX2 is used?

As in you compile with `-march=native` or so?

avx2 enables integer instructions, avx already has 256-wide float registers 😉

<@!179701849576833024> Ah, right. My experience is with applied cryptography... floating point stuff is pointless 😉

as for patches going out of memory... well, I can't say I cared so much about memory usage when I wrote that heuristic 😛

"only avx2" means avx2 and anything before it anyway, so no disadvantages that I can think of

Yep or sometimes `-march=haswell` (in order to have at least AVX2 support)

What are best settings I can try encode my image with?

lossy or lossless?

and what kind of content?

https://discord.com/channels/794206087879852103/803645746661425173/811747451069399100

I've tried modular lossy on this, not only quality dropped (of course), but also file size **increased**. It didn't pay off at all, just use lossless

Yes, `-s 9 -E 3 -I 1 -g 3 --patches=1` 😉 463 KB.

I've just run Scope's build on another, just powered-on, not bloated laptop. Unfortunately the CPU is Ivy Bridge, too, so no AVX, but at least the encoder didn't crash.

Aren't patches enabled by default at -s 7 though?

You know, better safe than sorry...

but that's lossy, right?

Nope, it's lossless

```C:\Users\Admin\Downloads\jpeg-xl-mingw64-e2b1e60f> .\cjxl -m -s 9 -E 3 -I 1 -g 3 --patches=1 .\reading.png .\reading.jxl J P E G \/ | /\ |_ e n c o d e r [v0.3.2 | SIMD supported: SSE4,Scalar] Read 3840x2160 image, 46.2 MP/s Encoding [Modular, lossless, tortoise], 2 threads. Compressed to 473186 bytes (0.456 bpp). 3840 x 2160, 0.03 MP/s [0.03, 0.03], 1 reps, 2 threads.```

hah, I forgot to actually pass the speed flag 😛 ignore me...

Are the presets different for modular vs VARDCT, or are the same encoders tools adapted for different compression techniques?

entirely different

Ah ok.

Does that mean you need to forcefully enable some flags for the encoder to use them?

Fun and useful parameters: `--patches=1 -m -Q -500 -s 9 -I 1 -E 3` let you see the patches in action (`-Q -500` is low enough to turn non-patched pixels into shit in order to reveal patched ones, but high enough not to turn the whole image into complete mess, because that's what `-Q -1000` did).

I think there is djxl command that disables patches, so you can decode with and without patches and compare results

I didn't find that in `djxl -h`, unfortunately

But with my command you can see the patches themselves

Here you've got one of those screenshots from a newspaper I was talking about, this is the first part of article about human trafficking in Russia (I don't have anything against Eastern nations, people can simply be assholes no matter the nation)

Here's JXL encoded with my parameters...

...and PNG result after decoding.

You can see now some weird behaviours of current patch heuristics at really low quality levels

They don't detect all letters

Basically all of detected letters are thinner and sharper because the patches lost all of letters' border smoothing (antialiasing)

And some of the letters are patched... partially? <:WhatThe:806133036059197491>

Here's an interactive comparison: https://slow.pics/c/2Tnboc6q

some things are shared though - like patches, splines, noise and restoration filters

Are the splines implemented in the current encoder?

nope

But the decoder can decode them, if you prepare the file in e.g. hex editor

good luck preparing them in an hex editor 😛 but yes

(it's likely easier to hardcode them in the encoder - we do have code to remove them from the image and write them in the bitstream, just not to find them...)

Are they removed in similar way as patches?

LowPoly set, difference with `-I 1`

https://i.redd.it/igu24zbqgud41.png ```S9_E3.jxl 2 833 233 S9_E3_I1.jxl 2 618 650``` Here the difference is more noticeable (in most other images less than 1%)

Hmmm... why not put spline parameters into a CSV file and make the encoder use that file for spline(s)?

Same answer to all other questions about things to do in the encoder 🤣

"patches welcome" ?

Interesting

Which command did you use?

`djxl -s 8`

You probably meant `cjxl -s 8`

No, it's downsampling on the decoder side

So it downsamples everything but the patches?

I thought that patches would be downsampled as well...

Patches seem to be stored and decoded as is

In this sample patches seem to work really well

I can read most of the text lol

Even default avatars (and status icons), yep

WAIT

OH MY GOD

I SEE IT LOL

AND STATUS ICONS TOO

Can you see those icons (available / idle)?

I truly underestimated current patch heuristics

But it doesn't mean that they shouldn't be improved at lower quality levels, because they probably can

yeah

which speed

which version of jpeg xl

0.3.2, latest build by Scope on encode.su

Various speeds, `-s 9` (or sometimes lower) on Modular, `-s 7` on VarDCT

why not -s 8 on VarDCT it is not slow

Last time I checked it was

Not as slow as `-s 9` VarDCT, but still

not as -s 9 lossless

-s 9 is also worst in quality then -s 8 on VarDCT

-s 9 should be fixed now 🙂

as in it has better quality than -s 8

probably not quite fast yet though

uh, I must retest

I make a image darker (for use as wallpaper) and I was curios if do it will make a smaller jxl

spoiler: a LOT smaller

dark

original

I've just tried to decode this file and guess what. While even `djxl -s 4` decoded this at full resolution and I found literally no differences at all in GIMP, `djxl -s 8` resulted in a completely black image. Does it mean that this `.jxl` file is... just patches and nothing else? <:kekw:808717074305122316>

Or maybe something broke in the decoder

I don't know

https://i.imgur.com/UGhOCH8.png ```WEBP 44 218 AVIF 47 452 BMF 40 452 PNG 120 487 S3.jxl 1 457 897 S9.jxl 96 995 S9_E3_I1.jxl 96 528```

- https://i.imgur.com/sxpUQlN.png ```WEBP 9 956 FLIF 7 682 BMF 9 676 PNG 12 364 S3.jxl 77 859 S9.jxl 16 980 S9_E3_I1.jxl 16 852```

- https://i.imgur.com/zHEkAUH.png ```WEBP 131 970 WEBP2 162 045 FLIF 154 871 BMF 137 108 PNG 150 327 S3.jxl 1 547 221 S9.jxl 213 727 S9_E3_I1.jxl 210 847```

- https://i.imgur.com/J86WTWN.png ```WEBP 1 666 WEBP2 2 536 FLIF 1 892 BMF 984 AVIF 46 966 EMMA 10 302 S3.jxl 3 464 S9.jxl 2 395 S9_E3_I1.jxl 2 398```

AVIF, EMMA <:Thonk:805904896879493180>

- https://i.imgur.com/yYj04fO.png ```WEBP 44 916 BMF 131 224 S3.jxl 3 369 891 <- S9.jxl 112 604 S9_E3_I1.jxl 108 344```

- https://i.imgur.com/Q2Uwkrh.png ```FLIF 1 673 BMF 656 <- S3.jxl 15 518 S9.jxl 2 514 S9_E3_I1.jxl 2 335```

<@!111445179587624960> what is BMF? I see it has the lowest size every time. We should drop jxl and use BMF 😄

it's a closed-source compressor that is only available as an `.exe`.

and it does not have the lowest size every time. It's quite good though. But as long as the author wants to keep it secret, its usefulness is very limited.

BMF is an old image compressor (first version in 1998, last version in 1999 and in 2010 recompiled with some tweaks) often tops in many image comparisons having reasonable compression speed, but EMMA (one of the new experimental compressors) wins in overall result except pixel art content. The problem is that all of them are not open-source and have symmetric decoding speeds.

Why is avif lossless so bad? I'm surprised jxl s7 beat webp2 overall

A better s3 will be the game on completly replace png

AVIF lossless will be better after adding YCgCo-R (9-bit) support <https://docs.google.com/spreadsheets/d/1DHJ0TajtxWDV6bfLwIZPHztxDuB3x3SS-HZ2Yi3sbRc/>

Yes, `-s 3` is good, especially if the efficiency on pixel art content is improved

<https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=0>

for pixel art and other kinds of 'weird' images, the gap between super-optimized PNG and quickly-encoded PNG is likely larger

I guess an -s3 for non-photographic content might be something like what you get today with `-s 8 -P (select? left? top?) -g 3 -I 0`

+ some option to do ycgco (I never remember *which* option xD)

`-C K, --colorspace=K [modular encoding] color transform: 0=RGB, 1=YCoCg, 2-37=RCT (default: try several, depending on speed)` -C 1 gives slightly worse results for pixel art

yes, but it should be significantly faster

likely `-s 8 -P (0|4) -g 3 -I 0 -C 1` is (or can be) as fast as the current `-s 3` if not faster and should be reasonably good for pixel art

```122 573 827 -s 8 -P 4 -g 3 -I 0 127 762 048 -s 8 -P 4 -g 3 -I 0 -C 1```

do you have some measurement of encoding speed in the two cases? (also how it compares with plain `-s 3`)

I can't measure exactly, but roughly according to my observations it corresponds to the `-s 3` speed

And `-P 0` is more effective on small pixel art images and `-P 4` on something like game screenshots

> `2-37=RCT` What is RCT?

Reversible Color Transform

How does it work?

it's just things like (a, b, c) -> (a-b, b, c) or things like that

you can select 3 channels (typically the first/only 3), do a permutation on them, and then subtract the average of two channels from the third and/or subtract one channel from another

For example (G, R-G, B-G) from FLIF? Or something like that?

basically slightly simpler versions of YCoCg that are also reversible

yes, subtracting green is an example

that would be -C 12 I think

but you could also e.g. do (G - (R+B)/2, R-B, B) or something like that

Which code stands for which RCT? Is there any way to know?

uh, no, only the code and the spec documents that atm

that's the spec

the cjxl option -C for some silly reason has 0 as no-op, 1 as YCoCg, and 2+N for the general case

(originally YCoCg and other RCTs were specified separately, now they're both called an RCT which makes more sense)

<@!794205442175402004> <@!179701849576833024> <https://docs.google.com/spreadsheets/d/1ugFqODnDnyeHbww9g4kKsqk--VKvosMQ2YYd1YN5KLA/edit#gid=1077758646> `-P 4` better on large and complex pixel art images like game screenshots (starting from 04gyqZ8.png - 1843)

For the rest `-P 0` (but, on a general pixel art set `-P 4` is also better) ```-s 8 -P 4 -C 1 -g 3 -I 0```

The only thing left to do is to somehow turn it on for `-s 3` by default on pixel art content <:Thonk:805904896879493180>

We could make some simple heuristic to select either current -s 3 or something that works better on pixel art.

Like just look at some random rows and count what percentage of pixels are equal to the pixel to its left

If less than, say, 20%, then it's probably photographic and current -s 3 (weighted predictor) is probably best

Otherwise probably -P 4 is good

Sound good

hi, how's JXL decompression speed ? are there benchmarks available, especially for lossless ?

for lossy, we have published https://infoscience.epfl.ch/record/277420, and additionally one can now run https://openbenchmarking.org/test/pts/jpegxl-decode

thanks, I don't know if it makes sense but do you have any guesstimate how it compares against png decoding (a magnitude order is ok) ?

Btw, in terms of speed `-s 3` is still faster by 1.5 - 3.5 times depending on the image (sometimes 5 times) than `-s 8 -P 4 -C 1 -g 3 -I 0` <:SadCat:805389277247701002>

And `-g 3` gives worse overall results on my pixel art set, so something like this is better: ```-s 8 -P 4 -C 1 -q 100 -I 0```

<https://docs.google.com/spreadsheets/d/1ugFqODnDnyeHbww9g4kKsqk--VKvosMQ2YYd1YN5KLA/edit#gid=1077758646>

Hmm, interesting, after some tests, `-s 4 -P 4` was faster and more efficient for pixel art, it is slower than `-s 3`, but on many images much faster than `-s 8 -P 4 -C 1 -I 0`, although on some images the size can also be much larger (but still not like `-s 3`) ``` [-s 3] Compressed to 691067 bytes (1.818 bpp). 2325 x 1308, 4.98 MP/s [4.98, 4.98], 1 reps, 0 threads. [-s 4 -P 4] Compressed to 596078 bytes (1.568 bpp). 2325 x 1308, 1.59 MP/s [1.59, 1.59], 1 reps, 0 threads. [-s 8 -P 4 -C 1 -I 0] Compressed to 948671 bytes (2.496 bpp). 2325 x 1308, 0.62 MP/s [0.62, 0.62], 1 reps, 0 threads. ``` ```[-s 3] Compressed to 196053 bytes (1.859 bpp). 1125 x 750, 4.73 MP/s [4.73, 4.73], 1 reps, 0 threads. [-s 4 -P 4] Compressed to 24439 bytes (0.232 bpp). 1125 x 750, 3.32 MP/s [3.32, 3.32], 1 reps, 0 threads [-s 8 -P 4 -C 1 -I 0] Compressed to 24160 bytes (0.229 bpp). 1125 x 750, 1.48 MP/s [1.48, 1.48], 1 reps, 0 threads.``` ```[-s 3] Compressed to 574821 bytes (0.994 bpp). 2325 x 1990, 3.57 MP/s [3.57, 3.57], 1 reps, 0 threads. [-s 4 -P 4] Compressed to 100171 bytes (0.173 bpp). 2325 x 1990, 1.81 MP/s [1.81, 1.81], 1 reps, 0 threads. [-s 8 -P 4 -C 1 -I 0] Compressed to 96273 bytes (0.166 bpp). 2325 x 1990, 1.07 MP/s [1.07, 1.07], 1 reps, 0 threads.```

Sometimes like this: ```[-s 3] Compressed to 967018 bytes (1.570 bpp). 2220 x 2220, 3.49 MP/s [3.49, 3.49], 1 reps, 0 threads. [-s 4 -P 4] Compressed to 166369 bytes (0.270 bpp). 2220 x 2220, 1.31 MP/s [1.31, 1.31], 1 reps, 0 threads. [-s 8 -P 4 -C 1 -I 0] Compressed to 98913 bytes (0.161 bpp). 2220 x 2220, 1.36 MP/s [1.36, 1.36], 1 reps, 0 threads.```

But overall, -s 4 was faster and more efficient <https://docs.google.com/spreadsheets/d/1ugFqODnDnyeHbww9g4kKsqk--VKvosMQ2YYd1YN5KLA/edit#gid=1077758646>

Perhaps there are some other options to make `-s 4` encode faster without much loss of efficiency, then it would be possible to replace `-s 3` on pixel art content, and `-s 8 -P 4 -C 1 -I 0` can be too slow (for that speed), except in some cases

we didn't quite optimize that combination as you can imagine by the number of flags you're passing 😛

that's just weird...

-g 3 can be bad for local palette

Or other local structure that can be captured in the context model / histograms

Yep, my idea was to take the maximum compression-enhancing features for pixel art from `-s 4` and disable the rest, so in terms of speed it was something between `-s 3` and `-s 4` (preferably closer to `-s 3`), since the default `-s 3` on such content is very "bloated"

Ok, now I know why it happened. My dumb ass didn't enable `-p`, so how was decoder supposed to downsample it?

I've just tried encoding this image with everything the same as my last settings, except that this time I added `-p` and (aside from making JXL file larger than original PNG) guess what happened after `djxl -s 8`? Letters turned into colorful mess. Then I zoomed into the original and found the reason. **ClearType.** It completely butchers patch detection.

Yes, subpixel rendering is a mess for compression

It also is display dependent if it even does the right thing

I can add a screenshot of my own with an even harder case of ClearType (even smaller letters), with additional images for added complexity. It did the same thing, completely blank screen on `djxl -s 8` without progressive, filesize larger than original and colorful mess on `djxl -s 8` with progressive.

You can add it to your patches' testing suite, it's a screenshot from a public website.

problem is that you can't do -s 4 without tree learning, and that's slower than lz77 with optimal implementations

of course we could try to create some reasonable tree

Fixed tree like in s3 WP could work

Use like 2-4 properties, put them in 5 buckets, use a fixed predictor like P4 or P5, special case it, and it should be faster than current s3 and decent for nonphoto (and not even that bad for photo if you use P5)

Could be worth doing, since nonphoto is the main use case where lossless can make sense on the web

🤔 Btw, I added Summary, but with Tab names (because something like Porn in the names does not look good in public presentations and Twitter posts) <:SadOrange:806131742636507177>

Will update tomorrow

<https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=174429822>

<:SadCat:805389277247701002>

Emoji still needs some improvement, though I guess it's mostly because they usually have alpha and we don't have an encode option yet to allow invisible pixels to change (which is default in WebP and FLIF), let alone to do premultiplied alpha (which is atm obligatory in wp2)

If we make a fast option like what I described, it would be faster than current s3, and better on nonphoto. We could call that s2. We could then change s3 to first do some quick heuristic to decide whether to do s2 or the current s3.

Yep, I will test with alpha options when they are added, but also `-s 3` is often more effective on Emoji than slow modes, so there are still tuning possibilities there

In the slow enough modes we should probably just also do s3 and see what is best

It would also be interesting to try some 10-bit, 12-bit and 16-bit images

If only to be able to write "n/a" in the WebP column, and also (for 12 and 16) in the wp2 and avif columns

I have not tested them because it is difficult to make such sets of public images, in addition, not all formats support it

Yes, it's harder to find test images for that

https://github.com/FLIF-hub/benchmarks/tree/master/test_images/Lukas-2D-16bit here are some

I think those are only really 10-bit, or maybe 11-bit

I'll look into it when I have time, different test sets probably exist with any data, but I don't really prefer them (because most likely they are often tested by various encoders and they will be better tuned for these sets)

thing is we shouldn't *need* to do that if our tree learning was good enough xD

Yes, that's also true. But can we make it good enough without more slowdown/memory requirements than it already has?

Trying some well-crafted fixed trees is cheap compared to actual tree learning...

(or maybe not fully fixed but parametrized)

`-s 4 -P 5` is also not bad

I benchmarked lossless JPEG to JXL conversion using 1,672 photos taken on my phone. Mostly pet photos, landscape shots, and random things I saw walking around. All 4608x3456 or 3456x4608 (16MP) with 8 bits per channel. I used libjpeg-xl v0.3.2, speed 7, and only one thread on both encode and decode (although up to 4 jobs were running at once) with a Ryzen 5 2600 at stock clocks (6c/12 3.4-3.9GHz). - Total space savings: **18.81%** (8.23 → 6.70 GiB) - Average encode speed: **808 ms/image or 19.70 MP/sec** - Average speed decoding to JPEG: **671 ms/image or 23.73 MP/sec** - Average speed decoding to pixels (aka PPM images on a ramdisk): **1.311 sec or 12.151 MP/sec** - Worst-case space savings: **12.90%** (it doesn't make files bigger!) Charts and the raw CSV should be below:

Comparison `-s 4 -P 5` on all sets (also better for Emoji than `-s 7` and `-s 9`)

I remember having ClearType briefly enabled on a Windows tablet, until I realized that Windows did not automatically take the tablet’s orientation into account