JPEG XL

these days, my opinion is that subpixel rendering is more trouble than it’s worth

full-pixel (grayscale) antialiasing is good enough, especially as displays get denser and denser

this increase in density also allows us to abandon hinting

for me, after using a high-dpi screen, there was no going back

Even on a normal dpi screen, I can't stand subpixel rendering. Actually, if your dpi is low enough, the subpixel discoloration can be visible at normal viewing distances...

filtering is supposed to help with that (https://nikramakrishnan.github.io/freetype-web-jekyll/docs/text-rendering-general.html#the-default-lcd-filter-for-subpixel-rendering-has-been-changed), but in turn, it sacrifices some of the resolution gain

so the question is what trade-off of color artifacts vs. resolution is acceptable to you

and it seems that for us two, the answer is “no color artifacts please”

actually, another factor for me was that subpixel rendering would end up in my screenshots

which makes those screenshots non-portable and less scalable

Subpixel rendering is not compatible with screenshot sharing

Once you want to share an image, it has to work on all kinds of screens in all orientations

Do modern operating systems turn it off on hidpi screens?

I don’t think Windows does

macOS dropped it at some point, which caused some backlash at the time

I get it for lower resolution screens, but if you have all those pixels why use it?

https://twitter.com/tapbot_paul/status/1004361530018852864

https://en.wikipedia.org/wiki/ClearType#ClearType_in_DirectWrite

right, but apps have to use newer APIs for that

there are plenty that don’t

it’s surprisingly difficult to get them to do grayscale antialiasing

I installed https://github.com/snowie2000/mactype for that

It's nice on low-DPI displays, but not necessary on high DPI and downright undesirable when sharing images with text.

For the sake of interest I also tested additionally `-s 8 -P 5 -C 1 -I 0` on all sets (-P 5 is slightly better on overall result than -P 4), that maybe it could be also somewhere better or the same as `-s 3`, but as before it is better only on Pixel Art And as already mentioned some faster optimized version `-s 4 -P 5` with fixed predictor, tree, etc. would also make sense for `-s 3`, and for example the current `-s 3` could be renamed to `-s 2` On the other hand current `-s 3` since it is well optimized and fast for photo and noticeably worse only on pixel art, then it is possible to just add detection and enable lz77 or some other fast compression variants on it

current -s 3 uses a photo-specific predictor that is not going ever to work well on nonphoto 😄 a fixed-predictor/tree version should potentially be a lot faster than the current -s 3

Here you've got another screenshots for patch heuristic testing. I no longer play those games, however good memories (and, of course, screenshots) remained...

The trick is: those patches are only *partially* pixel-perfect. At the edges there is some variance of grass, which has to be accounted for somehow, e.g. taking together all the samples that encoder considers not perfectly the same, but just close enough, take an average / median / mode out of all those matches and if the difference is small enough, the result will be encoded.

In those two cases that I've just uploaded the average needs to be encoded as a patch, because here the vast majority of the patch is pixel-perfect, only the grass on the edges changes, but those can be coded as residuals.

This GIF should explain it. Those are all same 40x40 px patches (after 4x NN upscaling for better visibility) in 100 ms intervals. They're pixel-perfect matches everywhere but the edges. Look closely, here there are different parts of the grass texture! The encoder should know that the part that doesn't change is a patch and all those pixels that are changing aren't and should be either averaged out (and the remainder encoded normally) or excluded from the patch with alpha channel.

Some (12) anime art lossy comparisons from low (0.15) to medium bpp (0.5-1.8), mainly because of noticeable artifacts on the lines in JXL, many site owners with such content consider AVIF (it loses some details, but for preview or thumbnail images is good enough) Same image sizes, latest builds of JXL (-s 8) and libavif (-c aom -s 1 -aq-mode=1), most likely Aurora1 will have even better results for AVIF: **<https://slow.pics/c/NvJvg7o9>**

- One example (also some noticeable color changes by JXL) <@!532010383041363969> Source:

AVIF

JXL

Could you give your settings for this comparison, or targets for file size/bpp?

I'd be interested to try it out myself.

`avifenc --min x --max x -a aq-mode=1 -s 1` `cjxl -s 8 --target_size=x`

Thank you.

Is it normal that I can't load the JXL copy? <:SadCat:805389277247701002>

Anyway, it looks like I was able to do a decent with modular. `cjxl -s 8 -m -Q 40`

Oh.

It looks like I can't decode anything with progressive on for some reason.

That's not normal behavior.

No it is not

This is my benchmark

AVIF - JXL

avif is so better then VarDCT on this, modular?

Modular is sometimes better, but it also has noticeable jagged lines, from which I once mentioned some techniques like LCEVC, where lines and edges are saved in full resolution and with less compression (lossless?) and overlaid later on a more compressed resized video (DCT compression is usually very noticeable on sharp lines)

Which quality setting did you use? I'm fiddling with `unknown_1hxwd.png` (the first one from slow.pics)

I don't remember exactly for each image, something between (--min 0 --max 20/--min 0 --max 47)

I'm talking about JXL

Just `-s 8 --target_size=x` Also WP2 as it is now tuned to save more details, can not keep as sharp lines as AVIF, perhaps in the future it will be useful to have something like presets for such content

Ok, thanks

to get nice looking low-fidelity encoding for such images in jxl, I think we'll need different encoder tricks. For images like this, AVIF's many directional prediction modes (+ chroma from luma) is very good, especially because on images like this, you don't notice that directions get rounded to the nearest available angle

I think near-lossless techniques (delta palette) could be good here, to avoid any dct artifacts while avoiding the typical issues you'd get with PNG8 (banding or requiring dithering)

for that we need something like pngquant to make a good palette as a starting point, and then use delta palette to fill the slow gradients

it's a matter of making an encoder that does that

Yep, I think further development of near-lossless technique ideas is good for such images And this kind of strong lossy compression is not just for my experiments, it is a real use, because usually various reduced resolutions from the large original image are very compressed, for faster loading and previewing of a huge number of images (where very much details are not needed, but maximum size reduction is important, until artifacts become noticeable and annoying), and for good quality there is a full size image (which can be in lossless, since quality becomes more important)

Btw, from the 4chan thread, one image, unknown CPU, number of threads and other things, but still...

Hello <@!111445179587624960> , For anime content you can try high quantizer like this. use --nclx 1/13/0 on png rgb file. // near jxl -d 0.5 // qp avifenc.exe --min 7 --max 8 --minalpha 0 --minalpha 0 --codec aom --nclx 1/13/0 -s 3 -j 12 // q avifenc.exe --min 7 --max 8 --minalpha 0 --minalpha 0 --codec aom --nclx 1/13/0 -s 3 -j 12 -a end-usage=q

My comparison above was mainly for heavy compression and low bpp

ok, and use default aq-mode and --nclx 1/13/0(on rgb file) probably can get better result

jxl rgb => xyb, avif rgb => ycbcr or ycocg

For low bpp `--nclx 1/13/0` gives worse results

what is nclx

Default:

`--nclx 1/13/0` (same size)

For low bpp, maybe also try 2x or 4x downsampling in cjxl...

Yep, sometimes it helps, but then blurring becomes quite noticeable

That's why I was thinking about LCEVC, where lines and edges are saved in full resolution, and everything else is downsampled

what is the settings of the vardct optimized scope

ziemek showed a GIMP photo

I don't know, for VarDCT I mostly just use quality and speed (rarely when I can noticeably improve anything)

how to use

--nclx 1/13/0

i want it at q 75

how to do

Scope used `-s 8 --target_size=XXXXX` (where `XXXXX` is the AVIF filesize, because doing it other way around, which is: to generate JXL and then match the filesize with AVIF, is much harder)

I used `-s 9 --target_size=XXXXX -c 0 --noise=0 --dots=0 --patches=0 --epf=3 --gaborish=1`

what is c 0

`--nclx 1/13/0` is for Avifenc ```--cicp,--nclx P/T/M : Set CICP values (nclx colr box) (3 raw numbers, use -r to set range flag) P = color primaries T = transfer characteristics M = matrix coefficients```

``` -c 0..2, --colortransform=0..2 0=XYB, 1=None, 2=YCbCr```

I simply wanted to make sure that XYB will be used

and what it means

why this / / /

<@!416586441058025472> next time just use `cjxl -h -v -v -v`, it'll show you all the options 😉 I can't understand some options or how would they impact compression, so I don't use them (leave them on default settings).

varDCT always uses XYB

we didn't even implement forward YCbCr

OK, guess I'll simply omit `-c 0` in my further VarDCT commands in order to simplify them

AVIF does better on these images. I consider it is mostly an encoder level thing, and I believe I can get within 5 % of AVIF quality at low quality when I'm complete with the low bpp optimizations. (within 20 % for an image that benefits a lot from directional prediction -- having a lot of directional obligue features without information across those features)

Also AVIF encoders are developing, me and Luca are helping them to take the best methods of JPEG XL to improve on the photography compression

In JPEG XL our code for choosing integral transforms is weaker (and faster) than similar code in AVIF -- this relates a lot to the low quality images

the formats are much closer to each other in image quality performance than what we can observe today

I consider that the end state is going to be something like JPEG XL is 5-20 % more efficient in high end photography, AVIF in the low end or smooth graphics -- JPEG XL will decode faster on software for HDR and YUV444 and have progression

AVIF can model the XYB colorspace at least partially if they add a 700 byte ICC profile in it

JPEG XL also otherwise has a smaller overhead for small images (if one aims at < 1000 bytes size, it doesn't really work competitively with AVIF particularly if there is alpha channel in the image)

What is your experience of XYB vs other colorspaces?

You mean in general? I haven't tried other ones than XYB, to be honest (IIRC). Again, I just wanted to be 100% sure that XYB will be used.

I'm working on exactly this, trying to tame those obligue lines. I have made some improvement from 0.3 to 0.3.2, but not yet enough.

<@!532010383041363969> Also about the noticeably different colors (on the faces), I checked on a higher quality and they are fine, in this image it happens only with heavy compression https://discord.com/channels/794206087879852103/803645746661425173/814002782776328242

<https://slow.pics/c/fGLgux2b>

oh, so much more yellow

Yep, also banding AVIF - JXL

<@!532010383041363969> here's my comparison: https://slow.pics/c/A7HJkgFe You're doing good job on improving low-bpp quality. There's more ringing, but lines look more faithful to the original and more pleasant to look at (in Scope's JXL the lines are jagged, only lossy Modular gives more jagging).

this is a lot better but avif is f*** good

how much bpp?

> but avif is f*** good Unfortunately yes. It can be even better though, I'm on AV1 Discord and talking with BlueSwordM about encoding it with rav1e.

Also I have noted that rav1e is better in still image, less smoting

That's what BlueSwordM said to me 😉

I added very basic avif support to benchmark_xl and in my brief testing, aom seemed better than rav1e according to butteraugli

but I must admit that I haven’t really looked at the images

Rav1e is sometimes visually better on art images (better AQ tuning?)

<@604964375924834314> what encoder speed did you look at?

I didn't use the slowest rav1e speed here(since there's not much difference in terms of speed yet between s1 and s4, as the presets in cavif-rs haven't been properly tuned yet) but it points what each encoder's strength is. The image comparison is rather interesting as you can see. It seems to try a different approach than aom as expected. It manages to preserve more detail as expected, but it doesn't have as good filtering(that is being worked on, and will improve with time), so it looks worse in this scenario However, the devil is in the details. rav1e does better than aom in my experience at higher bpp, which is particularly interesting in the case of the video comparison that I showed you in the AV1 discord server. https://slow.pics/c/nagJiHIk

When I tested, rav1e was often worse than webp at its fastest setting, but at slower speeds it is quite good

SVT had the best speed/quality trade-off for our use cases at Cloudinary

Until we licensed Aurora that is

mainly the default speed iirc, so 8

But SVT is limited to 4:2:0 so for 4:4:4 we did fallback to rav1e

Default rav1e speed is crap

Isn't default rav1e speed 6 though?

rav1e’s might be, but libavif’s isn’t apparently

Depends on who sets the default, I guess

I also tested rav1e mostly at slow speeds

Slow speeds are not feasible for on the fly encoding though

We cannot have latencies that large, first requests will time out and stuff

We used svt speed 7 iirc

And even that was too slow

With Aurora it's about twice as fast as that, for a slightly better quality/density curve

Yep, but I tested with an eye to future optimizations and the overall quality capabilities of the encoder

Sure, that makes perfect sense

For testing

Not for production 😅

Anyway, I'm not too worried for rav1e vs libaom.

rav1e should be receiving a nice large speed boost both inter and intra soon™️

As wb has said, keep JXL for images, and use AV1 for video. 😛

And for previews and thumbnails, for now (If there is extra CPU power available <:AV1:805851461774475316>)

for thumbnails, I think WebP is atm the best choice. For some reason, for small images, AVIF doesn't really shine. It might be the combination of relatively large header overhead and not being able to use bigger blocks.

300 pixels wide is roughly the point where AVIF starts to beat WebP, smaller than that often WebP is better

jxl can be good for 300px image in the future?

Yes

Yes, I didn't mean very small images, but also: Source:

AVIF

JXL

WebP

For such images and heavy compression, AVIF is also still good For more complex or photo images, JXL is already good for small resolutions (unless adding more options for Alpha)

<https://slow.pics/c/tFGNKwvB>

Yes, avif is quite great at illustrations in such a drawing style with clear, hard lines. This is what many directional prediction modes is great for.

But for typical low-fidelity photographic thumbnails like e.g. in a product gallery, WebP and AVIF are close to one another.

original: Mika Pikazo, www.pixiv.net/users/1039353 https://www.pixiv.net/artworks/78511602

is this speculation or knowledge -- could we try it without any directional stuff in AVIF?

speculation

It would be interesting to see what is actually giving AVIF an edge in these images (no pun intended)

Could be directional prediction, could be directional deringing filter, could be palette mode blocks, could be chroma from luma, ...

<@321486891079696385> <@111445179587624960> do you know how to run avif encoders with particular coding tools disabled?

<@532010383041363969> are we exploiting clamping behavior to reduce chroma entropy? For images like these, it could be worthwhile to e.g. pretend that all the hair area is purple in chroma, instead of letting chroma go to zero for the white and for the black parts. If luma is zero, the pixels will be black anyway, and if luma is high, they will be white anyway, whatever the chroma components say.

No, I'm mostly comparing encoders in the way most people will use them

Yes, I think directional deringing also affects https://youtu.be/VHm5Ql33JYw?list=LL&t=602

I remember seeing a tool that visualizes av1 block type selection. It would be interesting to compare choices made by av1 encoders with those made by cjxl, especially for images where avif is doing better

Does anyone know how to get those visualizations?

https://github.com/xiph/aomanalyzer

Yes, I've seen some browser-based analyzers like the one described here <https://medium.com/hackernoon/av1-bitstream-analyzer-d25f1c27072b>, but, the most convenient and detailed analyzers are paid And also some tools can be enabled/disabled just through aomenc options (but I have not tried to disable anything more)

Let's get more data on what is causing this significant difference between AVIF and JPEG XL.

it can be ac strategy, integral transforms, filtering, prediction, palette modeling, yuv420, .xyb vs yuv, cfl dfferences, and many others

Just yesterday I fixed a bug in ac strategy in jpeg xl, who knows how many more are lurking in there 🙂

there can be 77 bugs or inaccuracies each worth 0.23 %

I consider we need an ablation study on both ends

turn off features until the two codecs converge into the same result

the codecs are not inherently different, both are mostly dct based

I have learned that often the combination of things is stronger than just pure reasoning

I believe all main codecs (webp2, avif, heif) other than jpeg xl perform well on this anime data

I believe we will be able to fix it soon after we understand fully what is happening

<@!532010383041363969> <@!794205442175402004> Another comparison, but when JXL quality is between -d 3.5 and -d 4.5 (in the past it was up to -d 8) <https://slow.pics/c/Ri6dHVlu>

did you try the delta palette option of jpeg xl?

it seems to me that avif is getting gains by small local palettes and from triangular/other weird block splits

weird block splits forming the block from two colors, usually white and another color

we don't have local palettes in jpeg xl

but we have something that is more powerful in the high-mid category and might work here, too

(the delta palettes)

they work with the modular mode and the lz77

the context modeling that system has is roughy equally powerful to local palettes, but allows less hard degradation than forcing a locality to 8 colors

Delta palette in modular mode? I've tried, but these sizes are usually much larger than the ones I'm testing. Without very heavy compression, all better and mostly noticeable ringing artifacts and jagged lines, but in this image the differences are not just artifacts <https://slow.pics/c/F1KDUnQe>

The benefit with AVIF encoders might be that they are using chroma from luma in intra only encoding and others filters.

I think it's mostly from a combination of directional prediction, directional deringing and palette mode blocks (like Jon said)

one problem in delta palette is that it doesn't add custom colors yet -- so if there is a color in the image that is not in the stock palette, it is dithered

dithering is more bits than a constant color

there is a chroma from luma approach in jpeg xl, too

is there a way to try this same image in avif without directional prediction and directional deringing

I have not tried at the moment, maybe need some changes in the source code (as not everything can be disabled through the options) And as I said, the further development of delta palette is very interesting and I think will be needed for art images (I am personally more interested in it), but there is also a need for even higher or more predictable compression

Perhaps CDEF has some effect as well (it can be disabled)

the delta palette in jpeg xl is pretty amazing creature, but I tuned it only at one fixed quality

at around 2.3 bpp photography

most likely for this kind of images we'd need other approaches

custom colors, less dithering, more lz77, locality for context modeling ++++

how many bpp do you get with delta palette for these images and what is your target bpp?

(sorry for the many questions)

<:Thonk:805904896879493180> aomenc ```--enable-paeth-intra=<arg> Enable Paeth intra prediction mode (0: false, 1: true (default)) --enable-cfl-intra=<arg> Enable chroma from luma intra prediction mode (0: false, 1: true (default)) --enable-palette=<arg> Enable palette prediction mode (0: false, 1: true (default)) --enable-cdef=<arg> Enable the constrained directional enhancement filter (0: false, 1: true (default))```

I have tested delta palette on past images (I will have to check again on these)

Perhaps some additional options are needed?

Or that's enough: `--lossy-palette --palette=0` + speed?

That's enough, maybe should add -P 0 too

`--enable-restoration=0` would also be good for AVIF aomenc-av1.

Lossy-palette: ```1hxwd.png.lps9.jxl 212 447 1hxwd.png.lps9P0.jxl 209 883 1kak1.png.lps9.jxl 217 250 1kak1.png.lps9P0.jxl 215 841 1lr9k.png.lps9.jxl 169 481 1lr9k.png.lps9P0.jxl 166 577 2rzpm.png.lps9.jxl 807 738 2rzpm.png.lps9P0.jxl 785 886 UH1ppCM.png.lps9.jxl 248 116 UH1ppCM.png.lps9P0.jxl 245 235``` Lossy: ```1hxwd.avif 59 356 1kak1.avif 80 596 1lr9k.avif 46 104 2rzpm.avif 321 617 UH1ppCM.avif 88 249 1hxwd.jxl 59 491 1kak1.jxl 80 691 1lr9k.jxl 46 169 2rzpm.jxl 322 077 UH1ppCM.jxl 88 217``` It's not that much of a difference (given that these are near-lossless sizes), but a lot more compression is often needed and this was a test with higher quality (AVIF can look good with much denser compression)

It would be useful to make an encoder for delta-palette that has adjustable target quality and that detects large surfaces of constant color and puts them in the palette (or just any color that occurs frequently enough)

Maybe Kornel would be interested in that. He made pngquant a lot better than it was, for example.

Yep, it would be nice to have a delta-palette mode with even more compression, where other lossy techniques are additionally used

The delta palette encoder can choose how much deviation to allow and where to allow it. It could theoretically produce anything between (probably very inefficient) lossless and very lossy (e.g. if it just very often says "predicted value is good enough, use delta 0,0,0")

Currently it only uses the default palette (the implicit absolute colors that are at indices higher than the palette size, and delta colors that are at negative indices), and iirc it just picks whatever implicit value gives a color nearest to the correct color, with a simple 1,2,1 weighing of R,G,B.

Hmm, maybe I should try encoding content with mostly lines and black and white colors, like manga 🤔

Oh, another comparison? Ok, here's the first image, just like before with `-s 9 --noise=0 --dots=0 --patches=0 --epf=3 --gaborish=1` and targeted size.

no, I have a complex rgb metric there -- actually I'm quite proud of it, could be worth to port to many other projects using rgb diffs

it even models primitively the rgb compression in lms directions

getting it right is iirc about 5 % better butteraugli scores

here some low bpp improvements:

vardct -d 11

before

after

trying to clean that up and get it submitted ... but I'm proud of this 😄

it is also deleting code rather than adding 😛

Wow, that is an impressive difference.

Not only is the PNG filesize smaller(indicating less artifacting), but I can also tell how much cleaner it looks at 1st glance.

Ah I hadn't seen that yet (https://gitlab.com/wg1/jpeg-xl/-/blob/master/lib/jxl/modular/transform/palette.h#L124) — older versions did something simpler there.

I really consider it one of my 'masterpieces' because it is rgb diff tuned to my xyb creation, and it makes sense, but it is definitely not obvious and I doubt that anyone else has ever come up with anything like it

if (c == 2 && ((a[2] + b[2]) < 1.22 * ave3)) { weight -= 0.5; }

this is brilliant for palette use

😛

unfortunately it looks like a confused schoolboy had coded it

<@!532010383041363969> <@!794205442175402004> Comparison on black and white manga, with medium-low and lower bpp [image size] <https://slow.pics/c/RaX7RJA5>

I'll post my version soon

<@!111445179587624960> are you using the latest commit?

it is clear that we need to improve on these

I also doubt that Scope is using 0.3.0

or earlier

because I feel that in 0.3.2 I have fixed some of those things

(can be that I'm overly hopeful)

Shouldn't that be `if (c == 2 && ((a[2] + b[2]) * 1.22f < ave3)) {` there? I.e. blue is even less important if there's less B than average of R,G?

I'm doing something wrong for having details in gray backgrounds -- it shouldn't be a big diff to having white background

I always use the latest builds of all encoders when comparing (if there are no critical bugs)

Thank you for confirming my deepest fears Scope!

I used simplex search to find the 1.22f, I didn't want to think

it is just the level where that rule kicks in because of L and M receptors are already saturated not to be receptive for blue

This is also avifenc encoding without tune and other options, only speed and quality

when there are crazy constants in my code and they are not 0.7777777f they are often found by simplex search against a somewhat relevant optimization corpus

when things fail, it means that I need to add stuff into the optimization corpus

looks like I have a lot of work

I've seen your low-bpp improvements on Air Force chapel and the lines got blurrier, but more accurate. Scope's encodes behave like the old version, they're sharper, but a bit jagged. Scope confirmed using the latest compile, so it must be something else. I'm using `-s 9`, but it didn't differ *that* much compared to `-s 8`. Main differences are `--epf=3` and `gaborish=1` in my command line (just in case gaborish isn't default). I'm also disabling noise, dots and patches (IMHO they're not helpful with that kind of imagery and they'd probably slow down encoding a bit, correct me if I'm wrong).

that is what I have been aiming for -- I consider the effort 33 % complete

my first iteration slowed down the encoding 25 % at default settings -- then we launched 0.3.2 -- then I slowed it down another 30 % with a promise to recover and recovered -- now I have new possible slowdowns

but I have plans on how to recover the speed

currently we try dct64x64 first and start splitting from there

I believe that if I reverse that to try the 8x8 transforms first and merge, I can make things a lot faster

but I'm scared of big quality changes, so I proceed step-by-step to keep things working while providing some improvements

I am mostly using the codec without -s setting in all the optimization work I do

I anticipate that that will be the normal way to use it

I'm secretly hoping that people would also use it without -d and we would get internet filled with -d 1 images with near-perfect quality

My main idea was to test the default settings on the encoders, also `--epf=3` and `gaborish=1` although somewhat help lines quality, but it gets worse in other areas

those filters improve quality and comfort, but they slow down decoding, too

Yep, I also noticed that

one of the worse weaknesses of our solution right now is the decoding speed on old arm cpus and all this filtering is contributing

I'm hoping to change the dct quantization in clever ways to get some fraction (20-30 %) of filtering benefits without actually using filtering

but I'm not yet working on that

I don't know what exactly makes Scope's lines sharper and more jagged, so I'm going to do some additional test encodes in order to determine what influences those lines the most: - `-s 9` only (with gaborish and EPF off), - gaborish off and EPF 3, - gaborish on and EPF 0, - and finally gaborish on and EPF 3 (my current command).

another clever idea that I had that we used in brunsli is that 2d dct has a weird property...

if you compute sums and alternating sums of dct coefficients across columns and rows, you end up having an extrapolated 1d dct of the edges of the 2d block

we use that for context modeling in brunsli and for smoothing in knusperli

we can also use that for quantization in jpeg xl to have less block-to-block disagreement

if less border artefacts overall, there is less need for smoothing

computing sums and alternating sums is a lot faster and simpler than doing pretty much anything else with the dct

usually a lot of head scratching and butterfly computations are involved, but this is just delightfully simple and fast stuff

ziemek.z and Scope -- if you are able to recompile, try with a lower and a higher kDcQuant value

perhaps we are putting all bits in the dc value and not enough is left for ac

Thank you!

we could turn off dithering at first -- that is often a -40 % in file size in other methods (didn't try it in this use)

Okay, I'll try it, does this value need to be changed in one place? Also surprisingly on content where there are only lines, AVIF reproduces them very accurately (they are not thinner, disappear, bend in other ways) and that is why people with this kind of content are so interested in AVIF (from storage to various services) This is also why I try to show and test on the weak points of Jpeg XL first

enc_adaptive_quantization.cc

increase it by 20 % and decrease it by 20 %

I don't remember what direction saves bits

probably increasing

I believe AVIF goes into 'screen capture' mode for those lines -- but don't know for sure

all integral transforms are panicking when they see simple oblique lines are create a lot of mess

However, JXL is better on the background (AVIF has noticeable squares and banding)

On the second image and low bpp

people don't look at the background -- only butteraugli cares about it 😄

when you put the kDcQuant high enough you will get banding with JXL, too

--progressive_dc will come to rescue at some stage

but even that is not magic

having major dc artefacts is usually a much worse failure mode than having some amount of ac artefacts

Also, yes, -d 1 by default will hopefully save the web from a lot of overcompressed images (and adjusting quality by metric in general)

many nice technology efforts were used to give worse quality to users -- like the 1st gen digital tv

I certainly hope that that will not happen with JPEG XL

-d 1 compresses to about current average bit rates, even slightly below

if some part of the quality improvement will be used for bit rate and some part for density, we may end up with -d 1.5 or so as the new 'internet quality'

With AVIF however, I notice that people compress images very heavily (even when it is not needed), precisely because the artifacts are very well masked (and the loss of detail is not so noticeable)

that is a bit scary for me

if it looks ok, but is possibly semantically changed, it can have interesting effects

like people can today use their old vacation photos from the beach to track the growth of a mole on their skin

if we occasionally leave it out but otherwise create a nice looking photo, people can become worried

there could be a crack on the engine block, but compression removes the crack

etc.

Another example, this comparison: https://slow.pics/c/CGOCjUbz

Some people even re-compress highly compressed Jpegs to remove ringing artifacts

cool

Look at the heart in this comparison and see what happens with the marked line in every codec.

In AVIF it gets cut off partially, in WebP2 it vanishes almost completely (!!!), and JXL is the ONLY codec that preserves that line, although blurry because of compression.

Yet another proof that JPEG XL cares about being faithful to the original 😃

reds are very difficult for color modeling

if AVIF is done in the XYB colorspace it likely gets that right

Can it be done? It'd be awesome to see! Interesting at least.

it is nearly possible

ICC profile is pretty expressive

Luca and Sami have created an XYB ICC profile

or nearly the same thing

we don't know if it works any better, but in my thinking it should solve that problem and the problem of white clouds vs. blue sky

usually simplistic color modeling (YUV/RGB) thinks that white clouds can be merged with the bright blue sky, or at least it is not a big thing

it is because blue details usually just doesn't matter at all

but when blue is poking out from r and g, then it becomes a stronger visual element

that kind of non-linearity is not possible to model in yuv

L and M receptors can receive blue light, they are just usually saturated with R and G

Sorry to interrupt, but Jon created X-B plots with constant Y. If they're supposed to be the same perceivable brightness, why is there so much blue in darker Y's? https://cdn.discordapp.com/attachments/794206170445119489/811627202097315840/xyb.png

XYB is made for high frequency colors

if you have one pixel row of a color delta and the next pixel row of the opposing delta

repeat 4 times

i.e., ac colors

XYB doesn't work well for smooth colors

or -- it is ok for smooth colors but not the best

CIE1930 solved smooth colors very well

You know about Oklab, another perceptual color space

yes, it is based on CIE

I anticipate that it is better for smooth colors

Here's a hue plot with constant luma and chroma:

https://bottosson.github.io/img/oklab/hue_oklab.png

also, it differs from butteraugli xyb by not having a bias

oklab gives beautiful smooth colors

It's as constant luma as it can be

yes, perfect

Same plot as above, but in HSV:

https://bottosson.github.io/img/oklab/hue_hsv.png

disgusting

You can probably see that blue in the middle is darker

oklab has many same ideas with xyb

xyb has a linear ramp for dark colors -- much much better for compression

xyb is for HF colors, oklab for LF colors -- if image has information in texture, xyb is better

And Oklab actually "sees" this darker blue, here's HSV's luma seen by Oklab plotted here:

https://bottosson.github.io/img/oklab/hue_hsv_lightness.png

Not only compression, I'm also interested in image processing, denoising in particular

https://observablehq.com/@raphlinus/perceptually-smooth-multi-color-linear-gradients

here you can compare xyb vs oklab in javascript

there are two different xybs

one in jpeg xl

and one in butteraugli

...

one in butteraugli is a biased log and weird spectra for xyb

one in jpeg xl is a biased cube root with easier to understand spectra for xyb

same basic ideas

we don't want to compute three logs/exps for each pixel in jpeg xl 🙂

six muls is much more attractive per pixel than three exps

For image compression XYB will probably be better than Oklab, but I've got a question about image processing.

werent cam16-ucs & jzazbz the most perceptually uniform colour spaces atm? _trying to half-remember some papers she read on a wikipedia binge_

I would not make a lot of conclusions based on Jon's XYB plots vs the oklab plot

they are different things

bonnibel -- depends on perception of what

highest frequency vision is bichromatic, low frequency vision is trichromatic

no S receptors in the fovea

if your color discs are 10 degrees or 2 degrees, they are solidly in the trichromatic space

if they are 0.03 degrees, they approach the bichromatic space

pretty much every 'truth' about human perception is a coarse approximation

for both vision and hearing

Chroma denoising in various programs, even commercial ones, is probably done in Lab, ITU YCbCr or some other not really perceptual color spaces (I know that because I put an image into GIMP, subsampled chroma and put back components, results were similar to excessive chroma denoising). You probably know how excessive chroma denoising destroys color information.

they just recently (like 20 years ago) found out that some middle cells in retina can also receive light and modulate their behaviour based on light reception

<@!532010383041363969> so here's a big question: which color space would be better for better chroma denoising, Oklab or XYB?

XYB and denoising B is safer than Oklab and denoising U

denoising X is usually not safe

X will have 4x more noise than Y, so something can be done

the eye is also about 4x less sensitive for X noise then Y noise

but the detectable noise level depends on background color

depends why you want to denoise, too

For example I've been fiddling recently with Topaz DeNoise AI. While luma denoising is state-of-the-art, chroma denoising is as mediocre as everywhere else and loses details.

That denoiser is truly addictive...

it is highly non-linear and the X/B direction rotates in RGB space with the background color

I'd recommend trying that out in XYB

the denoising

oklab denoising wouldn't work for dark areas

because no biasing/no linear ramp (like in sRGB)

I can't really convince Topaz Labs' R&D Team to implement Oklab or XYB in DeNoise AI in order for some random from the interwebs to try out lol

also the S receptor response adds to the CIE color space luma, but doesn't add to the XYB's luma

haha

Ok, that's a bad thing, especially because dark areas are quite common in photos. Why does it happen in Oklab, worse accuracy in darks?

oklab compression function is pure cubic root

cube is scale invariant

So worse accuracy in darks?

this is beautiful for maths

but it has too much accuracy close to zero

when you try smoothing there with too much accuracy, no smoothing will actually happen/be observable

Maybe properly trained ML denoising model (because that's what Topaz is using) would account for that? ML models can learn some properties no human could've ever noticed.

Nah, just let aomenc denoise.

It does quite well by itself. <:kekw:808717074305122316>

<:MonkaChrist:654081051513061388>

So if the model notices dark areas in the photo, I'm wondering if it could be trained to know that it should denoise those areas stronger in Oklab because otherwise it wouldn't be visible at all after converting it back to sRGB...

yes, in oklab it should smooth stronger there

but having to do such tweaks defeats the purpose of a colorspace

when all things are right, you just do the simple thing

Though it's probably one of the easiest properties that ML model has to learn on its own...

But you're right about colorspaces, they should Just Work™️

Honestly I'd just re-train the models with exact same learning data, but in both XYB and Oklab and compare the results.

<@!532010383041363969> oh, and another crazy idea before I go. I'm also interested in audio and obviously know about Fletcher-Munson curves (equal-loudness curves). I'm wondering if it's possible to make equal-color 3D LUT by interviewing people, showing them random colors and asking them if they differ in brightness and if they do, then which one is brighter. Because that's essentially how Fletcher and Munson made their curves. That'd probably make perfect luma separation possible...

funny that you like audio -- I'm working on that, too

do you know http://www.machinehearing.org/

I have heard very good things about this book

https://tenor.com/view/culture-man-of-culture-cultured-gif-10903367

another interesting thing in hearing is the CARFAC model

Haven't heard of that one, unfortunately

the author is guiding the project that I setup in that domain

I was mainly sitting on HydrogenAudio and testing Opus Codec, been there since opusenc 1.2

currently we are looking into partial loudness models

I hope to eventually bring some new ideas into audio compression from that work

Now since the audio compression tech matured, I moved my interest to images

now it is still in deep research

nothing in tech is mature 😄

it will take another 500 years

We are at times like 50 years after Gutenberg invented press -- there is still room for improvement 🙂

Ok, in another words: general use, "classic" audio compression technology is mature

this is very interesting, but let's chat more another time, I need to go into bed...

Because there's still some research on narrow use-case compression (e.g. codec2) or AI-based compression (but this one will revolutionise every field, not only sound, images too)

I also think that's enough **for now** 😉

Opus 1.4? 😛 Anyway, sorry for bothering you. Good night. 🛏️

I'm still waiting on rnnoise integration into Opus' Silk for even higher quality(less random noise=better compression) VLB speech.

> Opus 1.4 God I wish... Opus 1.3.1 is already good, but can it be made even better? Opus development unfortunately stagnated, it's been half a year from 1.3 to 1.3.1, now we've got barely any news (if any at all) since Apr 12, 2019 (1.3.1 release date).

Yes. It can be even better I bet, especially at lower bitrates for speech.

Amateur. I'm denoising my mic before the sound reaches any app 😉

Same here. I've got Cadmus running, which runs the audio through rnnoise, detecting whether or not I'm actually speaking.

For me it's Equalizer APO + Werman's RNNoise DLL, because I'm on Windows.

Are you on Linux? ...ok, I've just searched for Cadmus and it's for Linux.

Low BPP D20 `kDcQuant = 0.89` I20 `kDcQuant = 1.34` <https://slow.pics/c/tApsL00L>

Visually different, but it didn't help much

<:This:805404376658739230>

For me unfortunately it's PITA to recompile this thing, that stuff takes lots of RAM and now I'm low on memory, *again*. Fortunately you took care of compiling this thing 🙂

Yep, but, it did not help much, perhaps in some areas `kDcQuant = 1.34` is better, but in others its worse

Same thing for me, it's neither better nor worse, it's just different.

Those plots were made in a lazy way: I just let X and B vary while keeping Y constant per subimage. That means that at the edges, the plots have "impossible" color combinations that would never happen when you start from RGB and convert it to XYB. For example, in the darkest one, the blue at the bottom is where Y is very low but B is very high — but that does not actually happen: there is no RGB combination that will map to that XYB combination. The inverse XYB transform maps it to some RGB values, and that's what you see there. But a more accurate way of making such plots would only show the part of XYB that is possible, and then you would see that the shape is not a square, and at the darks and at the brights, the area of X,B values is much smaller than at the mid-values of Y.

The current XYB space might be slightly different, I made this visualization a while ago

That's XYB on the left, YCbCr in the middle, YCoCg on the right

(there are some holes in the XYB slices because populating them with all 8-bit sRGB colors does not reach all pixels of the plot, but you get the idea)

this looks smart and nice https://www.youtube.com/watch?v=eNOGqNCbcV8

I like how they consider the original spectrum of captured light instead of trying to juggle it when it is already too late

that is very similar to the thinking I had when creating XYB

trying to give it a 1st principles systemization approach instead of loads of glue, cargo cult and inherited rules of thumb carried over from the analog computer era 😛

basic physics (at my level of knowledge) indicate that photos don't compress -- they just superposition, gamma compression can only happen once they are no longer photons

the process that converts them into chemistry can be modeled as a differential equation, but that process absorbs wide spectra of light

the compession process (mostly in the receptor itself I believe) cannot know which wavelength the photon had before being converted into chemistry

because of it we need a 3x3 matrix that models the conversion of linear light into the received light in the LMS receptors

and hey, while at it, let's make it 4x4 homogeneous transform so that we can add the spontaneous triggering of that process (which also biases the dynamics)

after that matrix we gamma compress, like the opsin dynamics process in LMS cells

at that stage we have what is close to LMS, and we do X = M-L, Y = M + L and B=S (or B=S-Y, to decorrelate a bit more)

after that we find Y->B and Y->X decorrelation factor (spatially varying images), similar to chroma-from-luma in other systems

doing this right allows for two interesting things:

we can store the B channel with less spatial accuracy

we can do higher (dead-zone) quantiation for the X channel, and use the bits for the Y channel -- the perception of XY is highly elliptical and our formalism knows the direction of that ellipse for a given background color, other systems can do it wrong -- especially for deep reds

my thinking is that it is premature to go into AI before we understand the human perception of sound

the HiFi enthusiasts talk about transparency and spatiality of sound

the psychoacoustic modeling people cannot trace three audio sources in space while humans can trace 17

AVIF (cdef on)

AVIF (cdef off)

CDEF smoothes a little, but does not affect the quality much (so it's something else)

Thank you Scope!!

This matches with my experience about CDEF, that it is usually only a small improvement

I think we do something awfully wrong in JPEG XL in integral transform selection

(we tried a version of CDEF in JPEG XL and it was not impressive to us)

we made the gaborish directional at a time (and tried it with an approximate reverse transform at encoding time)

it was not much better or worse but much more complicated

----

ok, that is for CDEF -- I wonder if there is a way to disable directional predictors

Yep, I have not found how to disable it through the options

On a lower bpp AVIF (default)

--use-dct-only `Use DCT tx onlyq`

The best case is that is a bug of the encoder

I found two minor bugs in the encoder earlier this week but they were something like 0.008 % and 0.33 % improvements

it can be a thousand minor problems kind of problem too

the dct looks similar to JPEG XL?

I believe the quantization matrices of JPEG XL are more sloaping and in AVIF flatter

if AVIF's DCT looks better than JPEG XL's basic reconstruction then it could be the quantization matrices

No, it's still better, but only with this option I saw some artifacts, AVIF has very powerful techniques to save lines even with very heavy compression

For example, the image size is 15,500 bytes (AVIF)

JXL (33 558)

To me jxl AC choice seem pretty bad on this case, i don't know if only a good filtering from avif but:

to me this nose line is so bad

JXL has too small integral transforms at low bitrates

when it see black and white, it drops down to 8x8 dcts

we know that they don't work at low bit rates