JPEG XL

yeah

you're on arch, right?

Yes

the thing is that static builds ain't gonna work if your OS doesn't have the static libs

and arch doesn't do that IIRC

Yeah you're right

https://github.com/libjxl/libjxl/commit/96e1f432595669ab9912934ec4d8b4d24d3f8eab Why no ASCII art tho <:PepeHands:808829977608323112>

Seems like I have to update my one-liner compiling script with `git revert`...

It's probably to make room for a nicer logo with custom console pixel manipulation.

Differently, is there a tool that can simply strip metadata from WebPs?

Exiv2 or exiftool?

exiftool doesn't write WebP, I'll look at Exiv2

https://developers.google.com/speed/webp/docs/webpmux

<:kekw:808717074305122316>

Good luck doing that in a portable way! 😅

I wanted to do ANSI colors originally, turns out that doesn't work in Windows

ffmpeg somehow manages to color its cmd output on Windows

it does actually 😛

Well it depends on the version I guess

Old enough: it works. New enough: it works. In the middle: it doesn't. At least that's what People On The Internet say, I am not going to touch Windows to try it out

hahaha

see how jxl looks good

Good points, could you open a GitHub issue for them? 😛

The gitlab repo is not going away, it is still the official repo like it always was. Development was never done there though, it was done in a different, private repo. That is what has changed and is now public, exactly to make it easier for people to not only report issues but also contribute fixes and improvements.

Migrating the gitlab issues to github would be nice but I doubt that can be easily done. So I think we'll just have to solve all the gitlab issues 😅

I think we're slowly going through the gitlab issues and either closing, migrating or asking to migrate them

then we'll close the gitlab issues

as for confidential issues: this exists https://github.com/libjxl/libjxl/security/advisories

we'll update the URLs soon 🙂 (well, not the one of the official website, that needs to wait for an ISO meeting)

but that's a good point 😛

ah, that's disappointing

although, even if you could create confidential issues - with development in the open, you'd have the problem of the pull request being public, no?

ah, fair enough

I guess we should write to send security issues to some email address

We looked at progressive image loading in the latest HTTP203. Thanks to folks here for helping me get a JXL decoding demo in there https://www.youtube.com/watch?v=-7k3H2GxE5E&list=PLNYkxOF6rcIAKIQFsNbV0JDws_G_bnNo9&index=1

(the JXL bit is at 20:50)

you can't compete with our army of Internet watchers: https://discord.com/channels/794206087879852103/822105409312653333/847145002105700403 😆

thanks for the video!

Level design of video games is kinda art.

So does this qualify as <#824000991891554375>? 😉

Haha why not

Here's the actual game play: https://youtu.be/cQAuH25hAao The jxl logo is better visible at the end, after crushing almost all the stones.

<@!179701849576833024> hah! Looks like you folks found it pretty soon after it came out. Nice!

WE ARE ~~ON~~ SPEED

<a:FurretSpeed:832307660035850310>

left jxl d1 s7 and right source png. I see banding artifacts. Is it normal?

source png is a jpeg resized to 25% with imagemagick.

Try decoding to a 16-bit png. `djxl --bits_per_sample 16`

also resizing may mess up dithering if it was in source

wow that banding is extreme though

can you share the source png?

this fixed it. now looks fine.

yes but I'm still curious how a default 8-bit png can look that ugly

maybe ProPhoto or some other icc profile like that?

icc:description: sRGB IEC61966-2.1

this banding is strange and much more than just quantization to 8-bit

can you share the source png so I can try to see what is going on?

that's some really bad banding...

probably noise is too much I think but they are demosaicing artifacts.

as I suspected, that ICC profile is not actually sRGB

or something weird is going on with it at least

I think sRGB IEC61966-2.1 is not compatible.

It is embedded from camera app. (google camera)

I think I should stick to display p3 or dci p3 when taking photos. (On my phone)

Converting sRGB IEC61966-2.1 to sRGB_v4_ICC_preference.icc fixed the banding.

I downloaded icc from color.org. 61966 icc works fine on other formats ( jpeg, PNG, webp)

it ends up getting encoded using sRGB primaries with a linear transfer curve

../lib/extras/codec_png.cc:87: Unknown type in 'Raw format type' text chunk: icc: 596 bytes

But I have this problem on source jpeg too. Jpg from camera app

yes somehow something is going wrong causing things to get encoded using linear sRGB

which is a problem when reconstructing to 8-bit

Right now I find Google camera embeds icc profile. Other camera apps don't. People with pixel phones will be affected after android jxl adoption I think.

should be reasonable, we can simply close if we can reproduce with 0.3.7 and not head

as long as it’s not too difficult to reproduce

``` -rw-r--r-- 1 47431412 28 mai 14:33 0.3.7-s3.jxl -rw-r--r-- 1 7920251 28 mai 14:33 0.3.7-s4.jxl -rw-r--r-- 1 7464122 28 mai 14:33 0.3.7-s5.jxl -rw-r--r-- 1 7449433 28 mai 14:33 0.3.7-s6.jxl -rw-r--r-- 1 7154828 28 mai 14:33 0.3.7-s7.jxl ```

are those the sizes you get?

oh, indeed I forgot to check the size of the original file 😅

probably -g 3 could help a bit

recompressing GIF losslessly is kind of tricky to do in a consistently-better way, since some GIF encoders are quite clever in how they do the color reduction and associated dithering in a way that compresses well in GIF but might be hard for cjxl to do a good job on

yeah no -s 3 is going to be horrible on GIFs no matter what you do, it uses the predictor that works well on photos but for these palette indexed images it will be horrible

hm strangely adding -q 100 in cjxl changes what it does, I wonder what's up with that

yes, it is

with gif input it does lossless by default

but adding -q 100 still makes it encode it in a different way

``` 5897751 slime_7cover.gif 36884914 slime_7cover.gif.jxls1 29426864 slime_7cover.gif.jxls1q100 31391478 slime_7cover.gif.jxls2 23240224 slime_7cover.gif.jxls2q100 47431412 slime_7cover.gif.jxls3 26518874 slime_7cover.gif.jxls3q100 7156799 slime_7cover.gif.jxls7 7184973 slime_7cover.gif.jxls7q100 5915585 slime_7cover.gif.jxls9g3I1 5934902 slime_7cover.gif.jxls9g3patches0 5911795 slime_7cover.gif.jxls9g3patches0I1 5442127 slime_7cover.gif.jxls9g3patches0I1P1q100 ```

for non-photo stuff like this (which includes all GIFs because they are by definition palettized), -s 2 works better than -s 3

(still poorly, but better)

Now I wonder why the -q 100 makes a difference, <@!179701849576833024> any idea?

5046479 bytes for just `-s 9 -q 100`

which should be the same thing as `-s 9` on GIF input but it somehow isn't...

-q 100 is mathematically pixel exact lossless, not GIF-bitstream-reconstructible lossless (only for JPEG we have such a thing)

how to verify that it is lossless: one way is to djxl to frames and compare each frame with what you get by doing a coalescing decode of the gif (`convert bla.gif -coalesce frame.png` iirc)

what 😮

Yes but gif is a lossy format againt source file. If you give us the original video file jxl will produce better quality and file size :p You need to encode gif in lossy mode. (with and without --patches)

Ok so with this gif all dct options is shit 😄 and lossy modular too 😄

even cjxl.exe .\slime_7cover.gif ./gif.jxl -s 8 -m -q 30 6.5mB 😄

cjxl.exe .\slime_7cover.gif ./gif.jxl -d 14 --progressive_dc=0 5.8mB 😄

cjxl.exe .\slime_7cover.gif ./gif.jxl -d 14 --progressive_dc=0 --dots=0 --patches=0 5.1 mB ! yes !

It's an awfull result 😄

gif is so optimised fot this kind of picture, so we do not convert it

compresses p well here with modular plus some extra flags

apng conversion is even worse 😄

Try using lossy Modular (`-Q`)

doesn't even need lossy to compress to quite lower size than original gif

just `-m -q 100 -s 9 -E 3 -I 1`

basically the usual "stronkest lossless" params

also git version

for this type of images (palette, sharp lines, text), lossy compression is not very useful and will often result in bigger files than lossless

it depends on your use case

you don't really know how much it took to compress the original if you haven't compressed it yourself

also it probably lost some details during compression process so it's not quite fair so say that

apng with imagemagick takes 7.8mB 😄

but i i convert the gif in multiple png it's 5.5mB but impossible to watch it 😄

``` % wc -c slime_7cover.gif* 5897751 slime_7cover.gif 5014182 slime_7cover.gif.jxl ```

it's funny to see inside a GIF 😄

But we understand why it's so hard to compress it.

tbh iirc cjxl currently doesn't attempt that sort of blending

it just renders every frame independently and compresses each separately

so there are a lot of optimizations possible there

which aren't there not because they're not possible to do, but because there wasn't much dev attention spent on that

I think that's individual gif frames, which are normally intended to be blended with previous frames, but in this case, applied to gray background for visualization purposes. I'd sorta like to know how to do such visualization aswell

yeah, trying to compress those delta frames with lossy is not going to do anything useful

but i think it doesn't compress delta frames at the moment at all, no?

for gif input it does

or it should at least

```5897751 slime_7cover.gif 5897505 slime_7cover_gifsicle.gif```

then I guess my info is sorta outdated :P

still, optimized-for-gif files are likely hard to recompress with other stuff

it's easy `magick.exe .\slime_7cover.gif .\slime_7cover.png` It extract all gif frames 😄 then i record my screen

Now do it with the mp4.

3.15mB in AV1 video, but it's pretty high crf 35

just `--no-conserve-memory --no-ignore-errors`

My Gifsicle 1.91 doesn't seem to have an `-s` (in any case, I didn't use one). `--no-ignore-errors`: "Exit with status 1 when encountering a very erroneous GIF. Default is to muddle on."

Uses about 10 MB memory on this image.

The latest release of ImageMagick supports setting the effort with `-define jxl:effort=<number>`

Might be the same thing that causes gif input to do something different with or without -q 100. Something funny is going on in the encode parameter passing I guess

Yes please

Interesting

good question, maybe different header?

comparing files themselves instead of pixels they hold is bad idea

yep, there's many different ways to get the same ppm...

Any whitespace can separate the header fields in ppm, and you can add comments

could be a change in bit depth

although the error looks a tad too big for that

This looks like a bug

How do they compare to original?

And what do you get if you first djxl them to png/ppm?

I mean what does compar -metric pae say when comparing cjxl's jxl to the original image

And IM's jxl

there's someone doing IM jxl support here in discord iirc...

Yes, <@693227044061839360>

Could be it is somehow doing things in XYB or something

Encode api has not been well tested, and cjxl doesn't use it...

how good noto sans lisu is?

best font ever

Noto sans lisu very relevant on topic

Does Android chrome canary support jxl?

not unless you build it yourself

Android chrome doesn't like extra package size for only an experimental thing, I guess

works on firefox nighty android though

we'll ask what we should do to enable it on canary builds on android 😛

well, not just canary

<@!794205442175402004> I'm not putting this question in <#805176455658733570> because it'll be kinda related to JPEG XL. How does lossy FLIF work?

quite different from lossy FUIF, actually

lossy FLIF is basically zeroing least significant bits of predictor residuals in the weird interlacing that FLIF does - it has the neat property of idempotence which is great for generation loss (as long as you don't crop/move pixels, that is)

Is it possible to do lossy FLIF without interlacing?

lossy FUIF (and lossy modular jxl) is quantizing Squeeze residuals (more quantization for higher-frequency residuals), which is a somewhat stronger way to do lossy, and also more progressive (partial bitstreams with Squeeze are actual downscaled versions of the image, while partial bitstreams with FLIF's interlacing are basically NN-subsampled versions of the image, with all the aliasing artifacts that has)

lossy FLIF without interlacing - not as good; you can basically do the same things you can do with PNG then: reduce colors to take advantage of palette, or quantize the predictor residuals to get better entropy coding, but you can't do a large amount of that because the artifacts get too strong

(with the interlacing you can avoid the artifacts better)

OK, so here's the actual question: We've seen in <#824000991891554375> that Modular predictors can generate pretty natural-looking, convincing images. How viable would it be to create lossy Modular *without* Squeeze, just with carefully picked lossless predictors and quantizing their residuals?

(Optionally the encoder could use lossy palette for even better compression, but it's not the main requirement.)

sounds hard for a practical encoder

I mean, my first idea would be to make some kind of genetic algorithm or something AI-based, but it would likely be insanely slow

it's a bit like printing a photo by carefully stuffing a lot of colored paintball capsules in a tennis cannon and then carefully aiming it at the sky hoping that it will rain down in a pattern that will magically look like the photo

I actually think there are a few reasonable options

like what we do for DC

use a fixed-Gradient predictor, then drop the lowest 2 bits from the residual, except for small values

or also, use a fixed-gradient predictor, and in leaves that correspond to low errors have a multiplier of 1, in leaves with a high error have a multiplier of 2/4

(similar, but different)

<@!794205442175402004> want to try playing with that?

yes, something like that could give good near-lossless results and also be fast

I meant something that does <#824000991891554375> style cellular automata to produce different but perceptually similar textures of things like clouds, smoke, grass, etc, that would be insanely hard and slow

yeah, no

but I do wonder if a good mix of fixed tree + predictor choice + quantization levels wouldn't give pretty good results

that would be an interesting phd project, to explore that and see what can be done (it's highly nontrivial)

Btw, can the XYB color space give any advantages for lossless/near-lossless, like -q 100 but with XYB or it doesn't matter?

for near lossless yes

for lossless likely no

for lossless it's not going to help, you need way too many extra bits of precision in XYB to do things losslessly at the RGB level

(it's not like YCbCr which basically just trims one bit from R and B so if you add two bits you're probably close to OK; the transfer curve is different in XYB so to make it mathematically lossless everywhere requires a lot of extra precision)

for near lossless it's likely a good idea

Thanks for your answers, <@!794205442175402004> and <@!179701849576833024>. I've got another question, this time related entirely to JPEG XL. IIRC in VarDCT mode DC is compressed with Modular and progressive DC is made by applying Squeeze to that DC frame, right?

yep

So is it possible to do lossy Modular on the DC frame? If it is, how do you think it'd affect visual quality and compression performance?

we do that

with --progressive_dc

and by default at low enough qualities

the selection of lossy modular quality might not be the most well tuned thing ever though xD

(read: I eyeballed it)

Are we talking about the same thing? I know that below `-q 7` Modular is used exclusively, but are you using lossy Modular in tandem with VarDCT on its DC frames?

yep

and above -d 3 or so, progressive dc is enabled by default

Nice to know

maybe 4.5

anyway above some point 😄

One last question (for now): lossy alpha. Is it allowed by JPEG XL?

https://discord.com/channels/794206087879852103/803645746661425173/817734220167774239

Damn, you've got good memory...

eh, I chose that constant 😛 I am pretty sure both me and Jyrki are eyeballing it by memory in both messages xD

https://github.com/libjxl/libjxl/issues/76

Because this is one of the things that improved quality of art content on low bpp

though us both mentioning 4.5 makes it somewhat more likely

anyway, alpha is encoded with modular, so anything modular can do, it can do it to alpha channels

I wasn't talking about *lossless alpha* lossily influencing invisible pixels, but about *lossily compressing alpha itself* no matter if you change invisible pixels.

see my previous msg 😄

I was replying to <@!461421345302118401>. Don't worry, I've seen your message: https://discord.com/channels/794206087879852103/794206087879852106/849325168119250995

We are already doing slightly lossy alpha in lossy mode, iirc

At least I remember I did add that at some point, pretty mild but slightly lossy alpha. Not sure if cjxl still does it by default

Lossy squeezed alpha shouldn't suffer from ringing at least (the main concern when doing lossy alpha: you don't want ripples of visible pixels near hard edges in the alpha, which is what DCT is likely to produce)

It might get a bit blocky though

Or rather pixelated

If you crank up the loss in alpha that is

The mild settings it uses now should be fine and help a lot for typical alpha masks used on photo+alpha, where it's mostly a binary mask with feathering.

It can probably be tuned though, maybe doing alpha a bit more lossily than what it is doing now. Need a good representative corpus of images that have alpha and where lossy makes sense though, which is something that is a bit hard to find. Most images with alpha you find in the wild are those where lossy doesn't really make that much sense, things like logos, icons, ui elements, game graphics. Actual photographic images with alpha are not that common atm in the wild because until recently, the only universally supported way to deliver images with alpha was PNG. People rather use JPEGs with a background color matching the website style than to serve huge PNGs...

I also have some question want ask, could I ask some question?

> previously in guetzli we had d1.0 = quality 94 or so --- if today quality 90, we may have slipped a bit in quality About this comment, why jxl d 1.0 choose quality 90? lower quality(90) will affect Adaptive quantization and butteraugli assessment? https://discord.com/channels/794206087879852103/794206170445119489/832524993966899210

I think guetzli may aim a bit too much at maxnorm BA optimization, I dunno though

In jxl we usually looked at bpp*pnorm to judge if a change is good or not

non-photographic images have much entropy, probably have chance adaptive quantization will do some radical lossy compress(choose lower quality) on much entropy area? (on libjpeg some non-photographic images need q99 444 to keep quality)

Well if things go wrong, it can either be the metric not accurately measuring human perception, or the encoder not accurately optimizing for the metric.

I using butteraugli and butteraugli jxl, but sometime they have different assessment on same compressed image.

Probably cjxl target distance have some limit on maximum distance? (like target distance -d 1.0 -s 8, maximum distance -d 1.5, result will on d 0.9 ~ d 1.55)

https://ai.googleblog.com/2021/06/a-browsable-petascale-reconstruction-of.html

I can't understand this... 😢

https://1.bp.blogspot.com/-TpKb0Ycw72Q/YLaHKBDDERI/AAAAAAAAHrM/Om5ioZmrnVYgVKaKRu4d3LkdSn-GrNphwCLcBGAsYHQ/s1999/image5.png

it's comparing compression ratio to the fidelity of... neural connections preserved i think

denoising making compression easier isn't new -- both jxl and avif has some noise model to basically take it out during compression and add someting like it back in during decompression. but i guess brain pics can use some specialized magical denoiser

I understand, thank you 🙂

Cool, avif and jpeg xl are the only choice in recent studies

HEIC combines all the disadvantages of AVIF with some of its advantages, while being a patent-encumbered mess that nobody except Apple can actually deploy.

Solution for H.265 images: drop HEIC and use Bellard's BPG instead 🙂 https://bellard.org/bpg/

That is a better container, yes.

Still patent-encumbered mess though.

At least it's one "patent mess" less...

Now you only have to deal with HEVC's patents instead of HEIC's *and* HEVC's...

Afaik only Nokia claims patents on HEIF. I don't know how many they claim to be HEIF related but I cannot imagine it being more than a handful.

HEVC has _thousands_ of patents, held by many different companies, some in patent pools, others not

All of this applies to BPG too: https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#Patent_licensing

I've got a great idea for a new container for AVIF <:megapog:816773962884972565>

.j(av1)xl

.javxl ?

avxl

https://fr.wikipedia.org/wiki/Eau_de_Javel

.javl

Bleaches your images nice and clean, all subtle textures gone! 😂

jax1v

yep

I'm still waiting for the new CNET article to get published

technically yes

but that will only happen if camera vendors and Adobe want it to happen, is my guess

that would be a useful convention, yes

the technical capabilities of the codec/file format are one thing, getting applications to implement it is something completely different

some banding for you 😛 (but only if it's not 100% rescale...)

That's what my phone does to that when making an icon

<:banding:804346788982030337>

<:banding:804346788982030337>

I don't see banding on Google photos on all scales. It shows banding only when loading the image.

fun thing is, it's just a gradient

each band is two nearby colors in 8-bit sRGB I believe

<@!794205442175402004> <https://www.twitch.tv/videos/1036703639?t=19h46m7s>

interesting - gonna watch that

https://github.com/libjxl/libjxl/pull/106

🤔 https://twitter.com/9550pro/status/1397812947049533445

and > Intel Alder Lake's Hybrid big.LITTLE design means that each chip has a number of Golden Cove traditional cores and a number of Gracemont Atom cores as well.

yep, heterogeneous cores is becoming ubiquitous

some phones have 3 different cpu types in them now - e.g. 1 really fast one, 3 fast ones, and 4 slow ones

which is cool but it requires somewhat trickier scheduling than when you can assume that all threads are the same

Is the github one the mirror or gitlab one ?

Github is where stuff happens

Gitlab is the mirror

you people are stalkers! 😛

https://twitter.com/SteveStuWill/status/1400859899702038530?s=19

What does it do? Improves performance?

We live in a ~~society~~ **simulation**

https://tenor.com/view/joaquin-phoenix-joker-smile-sad-smirk-gif-16148806

Better load balancing for combinations where some cores are faster and others slower (now it is mostly mobile processors, but the next generations of desktop processors are also planned to be similar)

also you can decide # threads afterwards, say based on image dimensions

I think this comment can explanation why current jxl vardct have some smooth issue for my use case. > From Jyrki Alakuijala > Two rather speculative considerations on this: > > Smoothing the input is a double edged sword for JPEG XL. If there are smooth areas in an image, the current encoder of JPEG XL considers that there is no visual masking there and can end up putting more bits there than without smoothing. Even a small smoothed area (such as three near-by 4x4 pixel areas that are smooth) within the same integral transform can reduce quantization opportunities significantly in the current encoder. Likely when and if we would like to introduce smoothing as a way to achieve higher compression ratios, we should reduce the effect of visual masking in the respective areas. (Sup. Fig. 1 supports this hypothesis -- JPEG XL is not benefiting from the soothing the same way AVIF is in compression ratio, and the most likely culprit is the modeling of visual masking.) > > AVIF's filtering is more compatible with producing smooth geometry (such as photos of plastic or metallic objects and outline drawings), while JPEG XL's filtering is more compatible with preserving low contrast textures (photography of real life objects, textures like clouds, skin, marble, wood). In video you don't necessarily need to get the low contrast textures right in the keyframe -- if the scenery is static, consequent P or B frames can fill in the low contrast texturing rather quickly and maintain a more stable use of bandwidth for video. > > https://encode.su/threads/3397-JPEG-XL-vs-AVIF?p=69868&viewfull=1#post69868

https://twitter.com/ID_AA_Carmack/status/1400930510671601666

Have a vardct question, could someone teach me please? visual masking how it works on jxl vardct?

#nobodyknows

(kidding)

... but I don't know really

probably only <@!532010383041363969> truly does

I try to understand this Jyrki Alakuijala comment, but visual masking is out of my skill knowledge...😢 https://discord.com/channels/794206087879852103/794206087879852106/850470364609249341

well, I do know the general idea - visual masking tries to estimate if an area is more "active" (such as like a bush may be, as opposed to the sky) - artefacts in more active areas are less visible, so we can quantize more

But how to assessment active area importance?

eh good question - it's done in https://github.com/libjxl/libjxl/blob/main/lib/jxl/enc_adaptive_quantization.cc and it uses a few heuristics invoving the Laplacian of the image and other things like local deltas in a block

just guessing, but I think the general idea is: few edges -> smooth area -> less quantization because you're more likely to spot issues like banding etc. Lots of edges -> busy area -> can get away with more quantization

very very roughly 😄

point is, it's encoder heuristics, and encoder heuristics can always be improved if they don't give the desired result

what cannot be changed is what the bitstream can express, but I don't think jxl has an issue there

So vardct will distribution lossy error on complex(busy) area, let error hard to see? but I still don't understand why drawing image will get some issue...

I think the real answer there is that we never did that much work on non-photo content 😛

I just expected vardct (-d 1.0, 0.5) will like webp near-lossless (40,60) quality, but use dct(smaller size).

Thank you for teach me about visual masking 🙂

Patches for text (and even small icons) works pretty well, but that's about the only real use of nonphoto-specific heuristics we have atm in the vardct encoder

I suspect avif uses palette blocks for those, but maybe also in other cases of hard edges where dct isn't that great

For nonphoto, likely an encoder that uses patches for the hard-with-dct stuff and dct anywhere else would be good

avif has CDEF, very powerful (deblocking and not) filters, and can also do directional prediction instead of DCT

all of those very likely help a lot

I think there's quite some room for cross-pollination of encoder techniques between av1 and jxl

Directional prediction is good for straight lines but I don't think it helps much for curved ones

Same with CDEF

Splines and paletty patches could help a lot, detecting them is the hard part

Also, discussion on HN, including mention of Jpeg XL and BPG (again): https://news.ycombinator.com/item?id=27399416

It's ironic that people want higher resolutions, more dynamic range, wider gamut, and then you have your nice 10-bit wide gamut high-res HDR screen and to save some memory, you end up using 8-bit YCbCr 4:2:0, effectively only using your green subpixels at full resolution (for red and green you could as well have only 1/4th of the subpixels), and effectively reducing your 1 billion 10-bit RGB colors to only 4 million 8-bit YCbCr colors, or even less if the 'tv range' variant of YCbCr is used like in WebP.

(honest question: why the heck is tv range still a thing??)

My theory is that it's only still a thing because it's quite good for PSNR: reducing the range of input and output can only help for compression, and by redefining the uncompressed input to be tv range YCbCr 420, it's even a lossless operation!

Maybe I am too cynical

Why modern video format av1 still use PSNR for default setting? (vvc use XPSNR)

What is XPSNR? Not familiar with that name

https://ieeexplore.ieee.org/document/9054089

I dunno, I guess because it's easy to measure and easy to optimize for? That plus tradition and inertia.

Mostly I think because there is no uncontroversial metric that just correlates perfectly with human opinion across the fidelity spectrum, so since that doesn't exist, many throw their hands in the air and say "well we don't know what metric to optimize for, so let's just optimize for a simple one"

Maybe we need a new image quality assessment method? mix Butteraugli + Butteraugli p-norm + dssim + ssimulacra + XL

Yes, there's interesting work being done and to be done on making better metrics. AI-based metrics can also be an interesting approach...

I have my own random ideas for a metric that takes some inspiration from Butteraugli and NLPD

but, as always, time...

XYB-based (instead of current L\*a\*b\*-based) SSIMULACRA should be quite good

But I'm waiting for AI-based metrics

That'd basically mean free psy-tuning

As said before by some AV1 devs when I asked if I could submit a document detailing why XYB should be the color space used in AV2 and what advantages it would pose: "HW ASIC manufacturers often pose some problems in the ratification of some features in the standard, such as color handling and chroma subsampling due to higher bandwidth requirements. Other times, another reason is that nobody actually pitched the idea and proved the large benefit of doing something novel."

there's no particular reason why xyb, or some variant, shouldn't work well as a ycbcr replacement - in particular subsampling B should be almost free, psychovisually speaking

Well, that's why I've been writing a small XYB requirement paper for AV2 so that it gets included in the main spec and we can leave YCbCr behind after all this time.

I expect you're going to have one heck of a time with it

Hey, as you've shown before, the benefit from going with XYB psycho-visually speaking is rather large as a single coding tool, so convincing them with a well written paper and experiments with JPEG YCbCr>XYB that you've shown earlier this year should not be too difficult.

IIRC there is a more fundamental problem: the decision process is entirely metric-driven, and the metrics are either computed in ycbcr space (which couldn't possibly help) or are VMAF, which IIRC hates XYB with a passion

I'm not saying you *shouldn't* do it

but it might take some extra work

I see. So this is an uphill battle 😛 I mean, one of my arguments is that JPEG-XL uses it, so that must mean it's good xD I can also use butteraugli as a metric which will help.

in fact, I recommend you write to me (veluca@google.com) and possibly Jyrki (jyrki@google.com) to set this up 😛 at least I would like to try

(I was asked by a couple of people to try to add butteraugli as a metric in AV2, that could be useful too)

or maybe that one if they want something easier to explain.. (and if I can manage to prepare something that works in time)

(I've had a few thoughts on how to do XYB in other codecs too)

Some rav1e/daala people are working on integrating butteraugli as an AWCY metric(Are We Compressed Yet, or a scalable cloud testing program which is what we use to test encoder improvements: https://medium.com/vimeo-engineering-blog/scalable-codec-testing-with-are-we-compressed-yet-c3a64003f67b), so that is in fact something that should come this year.

reeeeally? why did I not hear of that 😄 do you know what exact version?

<@!321486891079696385> can you use Dolby's ICtCp in AV1? It's kinda in the same spirit as XYB (perceptually-driven) so it probably should perform in the same class as XYB... And if you *can* use it, how?

I know you can, probably through the CICP box

but I don't know much more

otoh you're not going to get many benefits without some serious reworking of quantization tables (IIRC AV1 uses flat ones)

Perceptual vector quantization then?

(I'm not familiar with the term)

It's a daala thing mainly https://jmvalin.ca/daala/pvq_demo/

I actually suggested it as a Google Summer of Code Project in rav1e, and someone actually took it up to implement it during the summer https://summerofcode.withgoogle.com/projects/#6634888326807552

pretty cool

the activity masking thingy is pretty similar to what JXL does (although computed differently)

How do they compare?

no clue 😄

the DAALA thingy is DCT-based

from what I can see

we had a piece of masking estimation that was DCT-based, but got rid of it

Why?

it didn't help

one reason for that could be that DCT is not particularly good with step functions

CDEF was turned on/off for the anime images where it was anticipated by some to be the reason for JXL/AVIF differences. Nothing happened there and it was a 1 % thing there in objective metrics. Directional prediction is a bit in the same category, but can be more effective -- just not on all images. Corpus level effect is likely 1.5-2 % and on '1 % best case' images for directional prediction you can see 10-15 %.

It could be one of the other two filters though

I suspect, but don't know, that Dolby's ICtCp is still CIE 1931 based -- XYB does things differently to not to be based on psychovisual experiments of slabs of color, but it also relates to the eyes ability to receive high frequency information, which is very different for S receptors than M and L

someone could look at the before compression colorspace rotation and how it relates to those of XYB

XYB is a great colorspace for image compression, but not very scientific -- no publication, and I only used my own eyes to come up with it, ... We did verify it with many eyes, though

if my suspicions is correct about ICtCp vs. XYB, it should give a 3-5 % compression benefit for XYB over ICtCp for normal images

ICtCP has logarithmic behaviour for high values making it likely more efficient in HDR than JPEG XL's XYB

... butteraugli's XYB would likely work better there

with JPEG XL we largely compensate for it in adjusting adaptive quantization respectively, but if one adds XYB to another codec without thinking about the cube vs. exp (or cubic root/log) difference, then it is not going to be that amazing for HDR

In AV1 colors are someone else's problem, the codec transports channels. There are eight(?) supported short codes for colorspaces, one of them is ICtCp. I tried to convince them to add XYB back in the day, but they were not enthusiastic at that time.

AI doesn't know the intrinsic compromises in human vision. When such a system is trained, the training material should cover these compromises -- no effort that I know is trying to tackle this area.

Visual masking works in many different ways in different algorithms (for different speeds and for different uses). The reasons for diversity/complexity are half speed related, half code archeology. Luca made an attempt to replace that mess with 'new heuristics', but I/we haven't followed up on it yet. One of the most important ways is in initial quantization. ...

There we compute a 4x4 subsampled view of visual activity (measured by a laplacian or similar)

each locality looks at a 3x3 neighbourhood of such 4x4 samples, i.e, 12 x 12 area is considered

we take the three 4x4 areas with the lowest visual activity (in that 3x3/12x12 neighbourhood), weighted-average the visual activities in them, and that works as a visual activity for that area

in Butteraugli we do roughly the same but with Gaussians and pixel accuracy instead of 4x4 and subsampling

enc_ac_strategy.cc needs something similar but computes it on its own in a slightly different way (which is just stupid, but that is how it is)

often heuristics like this include more compromises (i.e., modeling other things than expected) than one would think, and clean reuse may be more difficult to tune than having specialized code

check out this pixel upscaling filter from russell kirsch (inventor of the pixel)

heres the white paper

it's not an upscaling filter, it's a proposal to use a non-square pixel representation

which is an interesting idea but I don't really see how you would apply it in practice

for display technology, it seems unlikely that something like that is practical

for capturing technology: the same

for compression, this is a bit like directional prediction built into the representation, but I think it complicates things more than that it helps

for uncompressed representations: yes, this might be useful; maybe for use in a texture format?

yes this is most similar to texture filters like hqx xbr etc

those are pixel art upscalers

this is not an upscaler afaiu, that image on the bottom was not generated from the image above

both were created from a higher res original, just one is using square pixels and the other is using the same number of variable-sized pixels

or a slightly lower number, but the same number of bits needed taking into account the 3 bits per 2 pixels needed to signal the pixel shape

JPG XL is the future

like if you agree

pixels are generally not considered to have a shape in the imaging community

they are point samples

the image can be reconstructed with squares, and it effectively corresponds to nearest-neighbour upsampling

but that’s not really what they “are”

in a sense, what he seems to be proposing is kind of a vector graphics approach with a specific set of primitives

but for a fair comparison with more “traditional” bitmap storage, I think he should have used better upsampling than nearest-neighbour

(and made sure that capture aliasing was properly avoided, that’s not really clear to me either)

Is there a lot of difference between google's butteraugli vs libjxl ? https://github.com/google/butteraugli/blob/master/butteraugli/butteraugli.cc

Not a _lot_, I think, but <@532010383041363969> will know better

good question 🙂

> Butteraugli vs Butteraugli(jpeg xl) > > From Jyrki Alakuijala > butteraugli's xyb is likely more accurate, > because of asymptotic log behaviour for high intensity values (instead of raising to a power), > > jpeg xl's xyb modeling is going to be substantially faster to computer, > because gamma is exactly three there. > > Shouldn't matter in the end. All butteraugli's have been calibrated to the same reference corpus with pretty good results. > The reference corpus contains 2500 image pairs in the area of 0.6--1.3 max butteraugli values. > Earlier butterauglis may be slightly more accurate, later butterauglis more practical and compatible with compression. > Later butterauglis can be slightly better at detecting faint larger scale degradations as > they do recursive 2x multiscaling of top of the Laplacian pyramid, not just a single run of > four levels of Laplacian pyramid. > > I, as the author of butteraugli, use the latest butteraugli in jpeg xl for my use.

Also, it's not just a question <https://summerofcode.withgoogle.com/projects/#6634888326807552>

cool 🙂 Rust Butteraugli

eh, that's one case where I don't really think it's a good expense of effort xD

Perceptual metrics is not immediately the first thing that springs to mind where security/robustness is hugely important

Could be nice for the slow settings of a hypothetical Rust jxl encoder though, and to integrate in rav1e

meh, you can use FFI and the C++ butteraugli 😛

From what I've heard Rust is better in terms of security/robustness. Does it come with a performance hit compared to pure C with assembly (like in x264 or libjpeg-turbo)?

I am the one working on it 🤣

We are mainly looking at maintainability and how well will it fair against Cpp

I think the claim is that you don't really pay a performance price - it's compile time robustness, not runtime checks or something

But it might be trickier to write some things in a way that Rust likes

In C you can basically do any dirty trick, whether it means you shoot yourself in the foot and write fragile code or you make something very fast in a robust way, C just lets you do whatever (it's basically just some syntactic sugar over portable assembly)

I'm too much used to C and "dirty tricks", do you think Rust will work for me? Because I don't think so 🙁 (despite the fact Rust seems like a fun tool).

I've read <@!826537092669767691>'s article: https://kornel.ski/rust-c-speed

> "Clever" memory use is frowned upon in Rust. In C, anything goes. For example, in C I'd be tempted to reuse a buffer allocated for one purpose for another purpose later That one hit me. On the one side there are MS Teams devs that programmed this shitty app and called it a day. On the other side I still program like it's '80s and I still carefully think whether to add another variable or not, sometimes re-using them **exactly** like Kornel described.

you do get *some* performance penalty

otoh, probably not much

butteraugli is not SIMDfied, so it shouldn't be too bad

There is also the angle of "maintainability", let's say you come back after months or say you working with other people, it might work now but there is not "guarantee" as to if it will from 2-3 months from now

Why is that ?

Somehow mozjpeg and x264 get maintained despite literally having *assembly* code in there

I think we just never had time to SIMDfy it

(btw, if you'd be interested in more rust stuff, I have my nice pet jxl reimplementation in Rust that could use some love by somebody that actually knows rust... :P)

When I said other people I meant others who are not maintainers and have not worked on it a long time, basically 3rd party people who just wanna modify it the slightest and can shoot themselves in the leg, A good C dev won't but a normal one might

I would like to take a look at it

By the way: is it simple to SIMDify C (not C++) code and is it possible without assembly (which I don't know *yet*)? I've read that Rust supports SIMD out of the box, which is awesome...

intrinsics exist

is that easy? I dunno...

https://github.com/libjxl/jxl-rs it never got even a review though

and it doesn't do *that* much yet

https://doc.rust-lang.org/edition-guide/rust-2018/simd-for-faster-computing.html

SIMD in Rust looks like this: ```rust pub fn foo(a: &[u8], b: &[u8], c: &mut [u8]) { for ((a, b), c) in a.iter().zip(b).zip(c) { *c = *a + *b; } }```

IIRC that's not necessarily SIMDfied

Making binding would be the best way if you want it as of as soon as possible, as long as cpp code does not break neither should bindings

there's already a couple of people that made bindings 😄

it's more of a mix of learning rust for myself and having a real Rust impl

(of a subset, possibly)

SIMD in Rust is basically when LLVM decides to vectorize code

Most of it gets taken care by llvm until it does not

Don't take butteraugli, I am doing that <:PeepoDiamondSword:805394101340078092>

stdsimd is being done 😛

wasn't going to

I was thinking of JPEG to JXL transcoding to start with

oh, nice

I remember them closing stdsimd https://github.com/rust-lang/rfcs/pull/2366#issuecomment-511724227

There is standalone crate now though that is being maintained https://github.com/rust-lang/stdsimd

My another issue with Rust: > Rust's lack of implicit type conversion and indexing only with `usize` nudges users to use just this type, even where smaller types would suffice. Seems like Rust is not for me, I was so hyped about it 🙁

Yep cannot index with u8, u16 or anything, but you can do `<any u8> as usize` just for at the moment use, I think cpp also converts them to usize, the only problem is you have to explicitly tell rust that make this as usize, will help if you accidently try to do something like this with f64 or i16

```rust ... let x: u8 = 3; let q = arr[x]; // will error out during compilation let q = arr[x as usize]; // will work ... ```

But will such cast actually use less memory and/or CPU?

It will be same as cpp in all regards you just have to state it out explicitly in rust, ~~in terms of speed vs memory it is most likely whether the number `x` is a compile time constant or not and what optimisation are being applied for speed vs storage~~

these casts are free.

Check with rust.godbolt.org

In the end everything gets inlined and crunched by LLVM

For simd there are intrinsics 1:1 copied from Intel's C intrinsics

And hook into the same thing in LLVM

yeah, those explicit casts can be a little annoying in the beginning but after a while you realize they're lifesavers

what can forced array index cast save from? use of wrong var to index stuff?

for array index, maybe not so much, but for other things, absolutely useful

unless you can tell me if this is true or false in C/C++: `((uint8_t)8*(uint8_t)8) - (uint8_t)70 > 0`

promotion to int applies, no?

oh you meant automatic integer cast rules. but does array indexing even use that?

the '-' operator itself causes promotion, I believe

i think * could do too (to int ithink)

so it would probably end up being false

oh, yes

they get more complex when it goes above 32bit ints too.. it's kinda mess

yup

for array indexing, I believe it simply comes from the "no automatic conversions" logic

also - what should `a[-1]` be? 😛

but ithink array indexing operators could define stuff for all unsigned kinds without much issue

probably - but rust has no function overloading, so...

no automatic casts is nice property, i like it in golang too

but i think i can address slices and arrays with whatever types there

If `a` has a pointer or array type, `a[b]` is identical to `*(a + b)`.

both rust and go bound check if they can't prove it's within bounds

*in C*, not in Rust...

Oh, sure. I have no clue about Rust.

Apologies, I missed context.

I'd say `a[-1]` is just usual bounds check failure at runtime (because compile time proof fails).. regardless whether you interpret that as signed int, or reinterpret as unsigned

this statement is too restrictive btw, `1["abc"] == 'b'`

(not meant to imply C-style int type here, rather isise/usize equivalents in rust)

```c int main() { char x = *("abc" + 1); printf("%c\n", x); } ``` works

(prints `b`)

well yea obv

it's more worrying that `1["abc"]` does xD