JPEG XL

What codec is google planning on using?

JPEG XL is partly a google project, but there's also webpv2 and they were fast to put AVIF in chrome

Google has no unified front.

I think they are betting on all horses in this race

I do wonder what the plan is with wp2 though. It sounds a lot like a lightweight version of av1-intra plus a next version of lossless webp. Which would be quite great, if avif and jxl didn't already exist.

In terms of added value compared to jxl and avif, the only thing I can think of is "it has webp in its name so maybe that will trick people into adopting it"

It feels like WebP2 will be to AVIF what JPEG XR was to JPEG 2000.

Kind of useless and obsolete on arrival?

In short, yes.

In testing, putting a WebP2 next to an AVIF with same/similar settings is usually difficult to tell a difference. Yet the creator told me on Twitter that it is not based on a video codec? If not, they definitely took a lot of the same techniques since the output looks very much the same as AVIF but with an even slower encoder as of now. I do agree that if AVIF wasn't developed it would probably have taken it's place, but given it does exist, I don't see much adoption on the developer side given it looks & smells like AVIF but with fewer features.

The main motivation for WDP/JXR was to make something that can do high bit depth and alpha like j2k, but computationally cheaper and simpler to implement, because j2k was widely seen as too slow and too complex

If I understand skal's intentions correctly, he wants WebP2 to be a simplified avif, more suitable for software decode, without losing too much compression efficiency.

That was exactly how jxr was positioning itself: somewhere in between jpeg and jpeg 2000 in both compression results and complexity/speed.

Even in the graphics next webinar, the AVIF creators were seeking to drop a number of less-likely-to-be-used & redundant features. I am just not sure how in the long run it will be different enough from AVIF that people would opt for WebP2 over AVIF.

Defining a low complexity profile in avif seems like a much better approach than making a fresh codec

What I see happening is AVIF being used for video thumbnails (like on YouTube, Vimeo, etc), JPEG XL being used for the vast majority of all other images, and JPG used for lower-quality (at least until the lower quality portion of XL is made better).

JPEG as fallback will be there for a while, but jxl is already a lot better than jpeg for low-quality

I haven't actually tested it since before 0.1, but when I did I noticed between the 40-55 quality there seemed to be a switch where JPG would produce a smaller file size and look about the same if not slightly better in most cases than the JXL. Maybe something has changed? I haven't tested that in a while.

Yes, things have changed, low bitrate is looking a bit better every new version now

What's happening with MPEG?

You mean after Leonardo ragequit?

yeah

I don't go to MPEG meetings, only JPEG ones, so I don't really know

I got some LinkedIn spam from Leonardo at some point, where he seemed to make the point that patents are a Good Thing, or something

His resignation creates a lot of confusion about MPEG

I guess in Leonardo's world, MPEG == Leonardo

Anyway

Looks like Apple broke something in WebP

https://ericportis.com/etc/broken-webps/

Yeah, well they don't get any of the experience with chromium's or firefox's implementation right?

Hey that's my meme)

I have no clue how they messed it up

It's very random

Many WebP images work

Some break

No way to predict what will work and what will break

VP8-only webps can break

Lossless-only can break

VP8+alpha can break

And obviously no way to check to code to try to debug

Is the bug with webkit? cause then it could be debugged on open source

Safari/webkit team blames the underlying system

Says they cannot fix

I have no way to contact the CoreImage team

Afaik

I used the webkit-based gnome web and got pretty much the same thing as firefox

I don't have a mac to test any further

https://chromium.googlesource.com/codecs/libwebp2/+/cf5abfa3c304cf9f18254ca374c5d55693394ae3

This just raises more questions, it uses a different file extension, and it's goals seem to be catching up with avif...

Is in early development, same avif problems (details)

You can also test it on squoosh

Yeah, I know it's still early days for webpv2, I'm just questioning the effort when your goals are to get something that is equivalent to something that's already in production and free

squoosh version hasn't half-block distortion

new experiment introduced 24th january

Simply avif is a bad generic porpus image code and a lot of people are realizing it

Is a strong improvement?

no

just better quality at q61 q74

maybe

but anyway is the same

is the same

It Is so slow encoding e so focus on low ppb, I have low aspettative on it

Is is slower than AVIF?

That would be an achievement

even effort 5 don't achieve anything

On decoding I think yes

And jxl have a lot more vardct dimensions

"compresses 2x faster than AVIF, but takes 3x more time to decompress. The goal is to reach decompression speed parity."

"more efficient lossy compression (~30% better than WebP, as close to AVIF as possible)"

So the goal is to be as good as avif in density, with the same decode speed

🤔

but not be avif

Isn't there a rather simple way to achieve that goal?

Call it navif

Ditch the container overhead of heif

Done

is heif catching on anywhere outside apple? or has avif killed it?

Avif hasn't killed heif

Avif IS heif

heic

the one based on hevc rather than av1

We do have some customers using heic for iOS apps and stuff

But it will never be a web standard, with the hevc patent mess

Unless we can wait until 2036, when the patents should have expired

but by that point VVC will have been a distant memory

Heic have some supercool patent tool or always dct?

heic is just an i-frame from HEVC which is patented

What is the stronger idea on hevc?

I don't really know tbh

"The majority of active patent contributions towards the development of the HEVC format came from five organizations: Samsung Electronics (4,249 patents), General Electric (1,127 patents),[10] M&K Holdings[11] (907 patents), NTT (878 patents), and JVC Kenwood (628 patents).[12] Other patent holders include Fujitsu, Apple, Canon, Columbia University, KAIST, Kwangwoon University, MIT, Sungkyunkwan University, Funai, Hikvision, KBS, KT and NEC"

https://www.mpegla.com/wp-content/uploads/hevc-att1.pdf

That's a list of all the hevc patents just in the MPEG-LA pool alone

That's just the patent numbers, in 3 columns

That document is almost as large as the jxl spec, lol

I honestly don't know how there can be thousands of patentable things in a single codec

I think the development of hevc was 10% writing code and inventing coding tools and 90% inventing and writing patents

MPEG is a special place

Someone once described it to me as 1/3 codec engineers, 2/3 patent trolls

case in point their website is still controlled by someone who split from them 9ish months ago

I stay away from mpeg as far as I can

MPEG LA /=/ MPEG

But some people in jpeg also go to mpeg, and both are part of SC 29 of ISO/IEC JTC1

MPEG LA is a vampire/parasite that finds all the patents that apply to the codec the engineers were forced to add

Then extorts people trying to comply with the international standard

but they don't even do that properly

The whole ISO process is kind of broken imo

No clear pool of people you have to pay

Yes a complete mess

The thing with the ISO process is that working groups are not allowed to talk about IP

You're supposed to judge contributions by technical merit only, without considering patents

cool

Well it makes sense in a way

I'd like to be able to legally use it over being 0.1% more efficent

But it's mainly very stupid

Smart think with other's money

It makes sense to get the best standard possible, but it also forces vendor lock-in

ISO still operates under the assumption that a standard will have a lifespan of > 100 years, so the first 20 years being patent-encumbered is not a big deal in the overall picture.

But when standards become obsolete after ~15 years, as is the case with video codecs...

I was going to say 15 years sounds long even, but then I looked and learned that H.264 was standardized in 2003, and is still very much alive.

JPEG and h264 are exceptionally long-living examples

VP8 was made and became obsolete during the lifetime of h264

The codecs that were meant to replace it died

https://en.wikipedia.org/wiki/Internet_Video_Coding

https://en.wikipedia.org/wiki/VC-1

VP8 was slower to encode for at best equivalent quality, and didn't get much hardware acceleration

In the early 2010s, google was also looking to have a rapid development model on their video codecs, getting a new one every 18 months or something insane like that

I think that ended when aomedia got so frustrated with MPEG they formed a new standards organization

I think VVC will failure... https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=144949

When you think HEVC was a mess, they go and make a Micheal Bay style sequel

Long live AV1

25 hours ago

Did anyone make a lossless converter between 🅱️ 🅿️ 🇬 and HEIC? The payload is <:H265_HEVC:805856045347242016> for both so that should just be a container thing, right?

(at least if you don't use fancy heif features, like Apple does with its independent hevc bitstreams in 512x512 tiles)

By the way, which is better quality-wise, efficiency-wise and complexity-wise, H.264 or VC-1?

h.264

How much is VC-1 worse? And are there *any* of those 3 mentioned aspects that VC-1 may be better?

Slower vs say the x264 encoder, less efficient, and is less supported overall, especially in terms of hardware decoding.

VC-1 is less efficient AND slower? No wonder it was obsoleted

well since VC-1's less efficient i imagine it hasn't seen as much optimization effort as H.264

Especially when some weebs realized that the encoders are crappy and decided to tune the fuck out of them just to make their encodes of *Perfect Cherry Blossom* gameplays good-looking

And that's how (roughly) x264 was born IIRC

You can also use h.264 in HEIF as AVCI for no particular reason

VC-1 has some things that are better than H.264

but I don't think any of the encoders got the love or attention that H.264 encoders got

x264 tune veryslow is amazingly powerfull 😄 and "more amazing" in 10bits 😍 Better than x265 in very high bitrates for many people.

<:H264_AVC:805854162079842314>

mozX264

X264tzli

X264 XL, bring it full circle

make a new video codec that uses jxl for intracoding <a:tapshead:482628289941340200>

well, there's already MJPEG 2000, but wouldn't that be too agonizingly slow and inefficent for anything but digital cinema?

I guess jxl could be used to replace j2k there too, but maybe we should let them have their niches

J2K is still quite active in JPEG, with their HTJ2K now and all

I was about to mention those things

JXL can have better quality than J2K in DCP

If they're planning to create next version of DCP standard, they definitely should include JXL

Better compression efficiency- and quality-wise

Grain synthesis (similar to AV1's)

And I'd also be curious to see a video codec with JXL's intra and e.g. AV1's inter

not sure if grain synthesis in jxl is useful for multi-frame, it might look weird

but for digital cinema you basically don't want to use any of the video codec trickery with grain synthesis and inter prediction and all that

because you can afford much higher bitrates

and the screen is huge

so you really don't want artifacts

motion estimation is good for video on the web, but for action scenes at the very high fidelity you want in a cinema, I don't think it helps much at all

https://www.reddit.com/r/AV1/comments/l8k2ed/is_youtube_phasing_out_vp9_in_favor_of_av1/

steven robertson Youtube engineer

read his answer

https://twitter.com/boutell/status/1357707533612441604

https://twitter.com/boutell/status/1357707536770678790

444 h264 still doesn't play in many environments

Probably never will, at this point

That's one of the reasons why still image codecs based on video codecs can have a problem with 444

It's a 4:2:0 video though, but guess it might not worked, because of the 4:4:4 profile

Video codecs seem to assume that end-users only ever need 420, and 422/444 are only for professionals

So you get WebP that only does 420

HEIC that on MacOS doesn't do 444

wow

so technically speaking HEIC is incapable of producing images of the same quality as jpeg... 😬

at least on macos

Yes it is quite crappy actually

Just double the resolution :D

They do independent 512x512 tiles, causing tile boundary artifacts because there is no cross-tile deblocking or anything

That's one reason I want jxl to succeed, it is better *on all fronts* than legacy codecs, not just selectively better for some use cases. sure, those are maybe the most common usecases, but there is no inherent reason to compromise in modern codecs.

If you do "export as lossless HEIC" in Preview, you get something that has a Butteraugli distance of 2-3 and peak pixel errors of 30%

Apple has a weird definition of 'lossless' it seems

oof

Nostalgia: this was 5 years ago: https://encode.su/threads/2321-FLIF-Free-Lossless-Image-Format?p=46705&viewfull=1#post46705

<@532010383041363969> asked "Do you believe that some interesting algorithmic middle ground exists between FLIF and WebP lossless?"

I think for lossless, we found not just a middle ground, but something way better than either of the three original ingredients (FLIF/FUIF, lossless WebP, lossless Pik)

Yeah, the reason I didn't use flif was slow decodes

https://tenor.com/view/the-princess-bride-inigo-montoya-meaning-are-you-sure-of-that-gif-7536490

Loss less, not no loss

So it only gets allows what every other codec would call 'slight loss'?

Less in that context means minus the loss, or free of loss. Not the way people normally use that word, admittedly.

https://twitter.com/jyzg/status/1358018700977328129

https://twitter.com/jyzg/status/1358023796247175169

Btw, the compression speed of LEA in single-threaded mode is about the same as `JXL -s 5`, and EMMA is faster than `JXL -s 9`, but the main problem is the decompression speed, it is sometimes even slightly slower than compression (therefore, they cannot be used for example as a Web standard or as a format for viewing photos/images, but they may be suitable for storage) Also EMMA has more efficient compression than BMF on everything except Pixel Art content

They are also based on this Context Mixing Compressor <https://encode.su/threads/2459-EMMA-Context-Mixing-Compressor>

Encode speed can always be improved later. Decode speed typically not really.

What's EMMA'S concept?

How does it improve upon current techniques?

Is it just an evolution of PAQ?

libjpeg-turbo significantly improves decode speed in comparison to usual libjpeg (using assembly and SIMD instructions, e.g. https://github.com/libjpeg-turbo/libjpeg-turbo/tree/master/simd/i386)

Yes. Modern codecs typically have already done most of the "-turbo" thing in their decoder implementation. Original libjpeg didn't because SIMD wasn't really much of a thing yet when it was written...

I found this helpful to know what happened to MPEG: https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=141678

I'm making progress on low bpp vardct jpeg xl 😄

work-in-progress

baseline

0.1 bpp

it is playing partial catch-up with avif and webp2 which are very good with this particular image

You can read similar 'JPEG XL' is closed stories in 30 years when I and Jon start quarrelling. When we start publishing dirt on each other, don't believe a word from either side 😄

Wait, the first one is optimized and the second one is before optimizations?

I'm still working on reducing the ringing -- it will need some more advances in masking

correct

How does forced modular mode stack up? IIRC, it does better at extreme low bpp(0,03-0,1bpp).

I'm not testing it in my work, just trying to improve vardct -- to reach lower bpp (and to improve in a wide range)

I can finally see the stairs and the narrower texture on the back of the church that got squashed before

And less color banding

this is distance 23 :-----D

How low for `-q`?

No idea. I don't use -q

But still, HOLY SHIT

the optimizations are still running and I have a few more ideas for algorithmic improvements

I see more ringing near the upper spikes of the church, but for me that's a good deal for optimizing the rest of the image

and looks like I can delete more code than add for this improvement

yes, I'll take that ringing away, I promise

it is already mostly gone for distances lower than 23

Still better than without optimizations 😃

(I'm testin 1, 3, 5, 7, 11 and 23 in this patch)

this photo is the main weakness that I know of -- vs. AVIF and WebP2

on most photos JPEG XL is far better already

I believe I can make it within 20 % in density, i..e, no longer embarrassing

with some luck better

Let's move this discussion to <#794206170445119489>, okay? Idk seriously why it appeared on <#805176455658733570> 😜

I'm doing this mostly to make jxl competitive with other codecs

Ah, ok

but I don't mind

I'm just learning to use this chat program

I used IRC the first day it was on

A man of culture

but haven't used these for 25 years or so

my guess is that I was the fifth user on IRC

also, I wrote the first bot 😄

I didn't realize that chat bots would become a business one day

Ahead of time, I guess

Me too

I was just a young fearless hacker with more energy than competence 😄

I was social outsider in school and getting notes and homework was kinda problematic for me bc it required social interaction, so I wrote quick e-Notebook website in pure HTML

With pre-made ODT template for OpenOffice (yeah, I didn't have MS Office back then) I manually rewrote every day's notebook into computer and exported that thing into PDF and uploaded onto the site

It was just a simple directory

Yeah... it sunk bc of lack of any interest from others

That was in 2012-2014... I guess? I really don't remember exactly

But now 2020 came and the age of e-learning began

So I kinda know what being ahead of time is like 😉

I lived in a small village and didn't belong -- then my family moved to a university town (in 1988) and everything was suddenly normal

yes, looks like I gave up with chat programs 30 years ago, not 25 😄

Not only compression, but also IRC chatbots?

Damn, I wish I knew as much as you all do...

I'm really struggling at maths, we collectively hate them as a whole

It's hard and I barely understand anything

I can only play with already existing things (like various codecs), it's a shame that I'm too stupid to create something mine (or at least participate in a bigger project bc JPEG XL definitely isn't a work of one man)

Also (~86 500 bytes each image) <https://slow.pics/c/CGOCjUbz>

Source

AVIF

JXL

Look at the ribbon on the heart-shaped box, JPEG XL is the only codec that fully preserves one of the lines

AVIF cuts it prematurely and WebP 2 completely omits it

Same with some other lines with same color on both sides

Yep, WebP v2 is also worse than AVIF in this image (although with the new builds on photographic content it is often better)

I'm delighted that WebP team is also looking into photographic quality -- not just psnr etc.

red colors are very difficult to model

the same for blue sky with white clouds

both are underestimated with primitive color models

the edge of the round cup is crappy with jxl

(I'll promise to fix it soon :-D)

updated the flif website: https://flif.info/

I remember that website without the first and last paragraph...

I still remember playing with FLIF lossless and lossy

And Xiph's Daala encoder

Back then I had to use Ubuntu VirtualBox and compile all the stuff there, because WSL wasn't even in our dreams

Ah, those were the times...

After FLIF I didn't have time, so I missed the birth and death of standalone FUIF, I've also heard of Pik, but didn't play with it, only watched visual comparisons by others

And now here I am, jumped straight into JPEG XL after merging Pik and FUIF

Seems great to finally be back after a break

Yes, good timing 🙂

typo: FLIF **development development** has stopped since FLIF is superceded by FUIF and then again by JPEG XL

oops, thanks! fixed

no problem

I've been following FLIF since the beginning, it had interesting features, open source, but slow decoding from the disadvantages. I was always interested in dense compression and there were formats which are strong even now, for example BMF was already in 1998/1999 (and in 2010 only some tweaks for slow mode were added) but it also has slow decoding and in addition closed source code, also various encoders from Alexander Rhatushnyak (who also helped in developing Jpeg XL). I was also interested in PIK (where I think lossy compression and visual tuning was the right direction to develop, unlike video-based formats) and I was happy that FLIF/FUIF and PIK merged and got best features of both, as well as improvements in lossless decoding speed and now it is fast enough for practical use in very dense compression. Also one of the important things is openness and communication between developers and the community, which is rare these days and format development is very closed (from what I remember before it was communication from Jyrki and Pascal about WebP development or sometimes from Thomas Richter about Jpeg formats)

For example, the development of AVIF was and still is quite closed from the community and only some third-party developers like Joe Drago, Kornel, Derek communicate.

Btw, in my opinion, more news and more discussions were about FLIF when it appeared than about Jpeg XL now, perhaps people have a worse attitude when the format is behind the big companies and organizations like Jpeg, and FLIF felt more open and independent.

Yes, the JPEG "brand" has positive and negative effects

Same with Google and Cloudinary, I guess, probably mostly negative though – nobody really likes corporations

superceded -> superseded

<@&803357352664891472> y'all probably know about AV1's CDEF. Was it possible to include it in JPEG XL and if not, was it because of technical limitations (it would break the format) or practical (e.g. being AVIF-slow)? Was CDEF even considered/discussed?

AFAIU CDEF operates better with low bitrates, which we didn't really consider a priority - so we have our own filter that works better for detail preservation

I assume directional filtering works well in tandem with directional prediction, which we also do not have.

Intuitively, for low-bpp high-appeal encoding, directional stuff is nice to get sharp edges and a smooth 'vectorized' look. It is not very useful for higher bpp, high-fidelity encoding. At high-fidelity, context modeling / entropy coding matters more.

Video codecs have to keep context modeling / entropy coding simple enough (hw-friendly), and they can afford to, because at the low bpps they are targeting, most of the stuff they are signaling has high entropy anyway, so entropy coding will not help much anyway.

At least that's my gut feeling.

> At high-fidelity, context modeling / entropy coding matters more. ...and seems like you've MASTERED it 😃

What's MPEG's numbering scheme? Right now they are working on MPEG-1 part 3 and MPEG 5 EVC at the same time.

As far as I understand, MPEG-1 part 3 is audio (that's why MP3 is called MP3). Otherwise, MPEG-1 is quite outdated.

And MPEG-3 was skipped for exactly that reason I think (confusion with MPEG-1 layer 3)

MPEG-2 is h.262, which was used for DVD

It would appear that I am wrong: https://en.wikipedia.org/wiki/MPEG-3

MPEG-3 was kind of skipped

And h.263 is not MPEG. h.264 is MPEG-4 part 10.

MPEG-4 contains lots of parts, a.o. h264 and the .mp4 file format

it's quite bizarre how many weird parts they have in MPEG-4: https://en.wikipedia.org/wiki/MPEG-4

Yes, and MPEG-4 also contains part 2, which is ASP, more widely known as DivX

VVC is mpeg-I part 3

h.266 = VVC = MPEG-I part 3. Indeed. I, not 1.

Not confusing at all.

h265 is in MPEG-H part 2

(and HEIF is MPEG-H part 12)

and yes, h266 is MPEG-I part 3

in short

There is only one rule: there are no rules

How much ambiguity, confusion and bearucracy can we get?

MPEG and ISO: yes

ITU-T H.266 MPEG-I part 3 VVC ISO/IEC 23090-3

these are all the same thing

JPEG XL is ISO/IEC 18181 btw

That's easy to remember at least

> ISO/IEC 18181 We've got quite lucky number, wow

missed that DIS submission date by a few days, not that it matters

We tried three different ways of directional filtering and transforms. None of them produced significant improvements in quality d1-d8. At d12-d16 we started to see benefits, but that was not in the quality range we were so interested in the end.

I studied AVIF performance for turning on and off the directional filtering on a larger corpus and saw the best gains to be 20 % at a relatively low quality

Our team was not able to make it work in a way that was satisfactory, so we left that out

we tried directional dcts (three different forms), complex perspective-like tiled geometry transforms, directional filtering (transforming gaborish into directional lines with a guidance field)

none of it worked as I'd like it to work -- over a large quality range, be fast to encode for, be easy to decide the heuristics to use for -- the directional stuff just didn't meet the quality bar we have

we didn't copy over the implementation from AVIF, but tried everything independently

also, we didn't try exactly the same approach as in AVIF

also, we didn't look into comfort at any stage but measured success through fidelity rather than comfort

it is highly likely that there is a way to do directional coding, intuitively should be possible, but we were not able to find good solutions for it

Two bright senior software engineers tried looking for these solutions, and one high-performing junior dev. I rotated the task so that we would find new perspectives to it. Still, it didn't make it.

sorryyyyyy

Here is some data for av1

x265 PLACEBO compared to aomenc cpu-used 6 (red color - need more % ,relatively , to get same quality)

https://cdn.discordapp.com/attachments/738825198534656100/809527697705271316/2021_02_11_22_52_54.webp

cpu-used 6 -> cpu-used 0

https://cdn.discordapp.com/attachments/738825198534656100/809528902875480134/2021_02_11_22_58_35.webp

what are the values? it is the first time I see such reports

it's from AWCY (are we compressed yet) it's a benchmarking tool for video codecs. Values are relative %, positive percentages under time category mean that codec we compare to take more time and values under metric category are BD-rate difference (how much more bit rate needed to get same quality on average, negative means that we need less bit rate (more efficient, green) and positive means that we need to spend more bit rate (less efficient, red)) In second example by switching from cpu-used 6 to cpu-used 0 (from fastest to slowest aomenc setting), our encoding time increased by 7361% for same set of videos, ~73 times slower and coding efficiency increased by ~25%

Here is x265 placebo vs aomenc cpu-used 6 https://beta.arewecompressedyet.com/?job=aom-master%402021-02-10T20%3A15%3A36.010Z&job=x265_preset_placebo_reduced_probe_set%402021-02-11T18%3A40%3A03.490Z

Here is aomenc cpu-used 6 vs aomenc cpu-used 0 https://beta.arewecompressedyet.com/?job=aom-master%402021-02-10T20%3A15%3A36.010Z&job=aom-master-cpu-used%3D0%402021-02-10T20%3A53%3A10.688Z

cool

can you turn on/off directional filtering?

and directional prediction?

Yes, in some parts. `--enable-cdef=0 --enable-restoration=0` are the most important ones in aomenc.

Yes, I can make any set of options/encoder cli

perhaps you can demonstrate how much we lost in jpeg xl by our inability to make directional stuff work 😄

for cpu-used 6?

sounds good to me

One second will start benchmark and notify you when it's done 👍

run all four combos, please

off+off, on+off, off+on, on+on

on+on is default, so I started 3 benchmarks

oioioi, I'm so excited 😛

is there a specific quality setting it goes to

(what we saw in jxl was worse performance at mid to high quality, and better at low)

https://distill.pub/selforg/2021/textures/

Now I have texture envy 🙂

Alex's tweet on this https://twitter.com/zzznah/status/1359971771030601731

<@!532010383041363969> Both off https://beta.arewecompressedyet.com/?job=aom-master%402021-02-10T20%3A15%3A36.010Z&job=aomenc_cdef_off_loop_off%402021-02-11T21%3A56%3A30.571Z Only cdef off https://beta.arewecompressedyet.com/?job=aom-master%402021-02-10T20%3A15%3A36.010Z&job=aomenc_cdef_off_loop_on%402021-02-11T21%3A57%3A16.869Z Only loop off https://beta.arewecompressedyet.com/?job=aom-master%402021-02-10T20%3A15%3A36.010Z&job=aomenc_cdef_on_loop_off%402021-02-11T21%3A58%3A07.393Z

CDEF is clearly pulling its weight all alone quite well actually.

<@!532010383041363969> what you think about results?

cdef looks like a small net positive without impacting encoding or decoding time

it can be that there are two things that are different in this eval and in the eval we did

1. our material had likely more high frequency components -- cdef can interpolate more effectively if the features are not super-sharp

2. these metrics use L2 aggregation, having some higher errors here and there doesn't matter so much

perhaps also 3. this comparison is 0.01 to 0.1 bpp, and our comparisons were around 0.3 to 1.5 bpp

Modular jxl can also do quite interesting classes of cellular automata

probably even more than flif

https://cloudinary.com/blog/compressing_cellular_automata

it would be a fun artistic exercise to see what kind of cellular automata can get their rules encoded in an MA tree. If the full rules are in the MA tree, the entropy coding gets a perfect context model: all histograms are singletons, so zero entropy left.

There are many, many very beautiful synthetic images expressible as jxl files under 1 kb.

new golfing game: designing codecs that make as many different CAs compress as small as possible.

the class of CAs that compress well in jxl is quite large

just exploring some variations on a simple CA that is based on only the value of `North - NorthWest`

someone should make a program that lets you define a CA by defining a jxl MA tree, visually

such CAs will compress to just a few hundred bytes

might be useful for some kinds of game textures, for example

another fun game: finding MA trees that result in the most visually complex images and then making people unfamiliar be like "whoa" when they see such a complex image compressed so small 😄

the above 222 byte jxl file will already be quite challenging for other codecs to not do 100x worse, even lossy

aye!

what options did you use to cjxl for that one?

`-q 100 -g 3 -s 9 -I 1`

I wonder why this one doesn't compress better, 1525 bytes is probably not optimal

a `-quality 50` jpeg is 1.7 MB so maybe I shouldn't complain though

ohhh so I got a worse size on foo.jxl because I used png as the intermediary format and cjxl stored the ICC profile >.<

using ppm instead got me 217 bytes

bless the existence of jxlinfo

anyway, this is just the result of CA based on a single MA tree property, and probably not the coolest ones but just the most interesting ones I could find after looking at a dozen random ones. The space of things that can be expressed is quite huge, and I wonder what kind of stuff is possible with it

Hm, you wouldn't happen to have handy any papers or anything else about how this MA stuff works?

MA trees can express at least any CA where the cell value depends on some combination of the neighboring cell values of W, NW, N, NE, WW and NN. Probably more.

cell values can have whatever range, and of course you can use whatever color palette to show them

Yeah it seems pretty cool. I mean like, can I read about how they work technically somewhere, or is the stuff in JXL so fresh I'll have to read the code to find out? 😃

uh

there's the old flif paper: https://flif.info/papers/FLIF_ICIP16.pdf

modular jxl has the MA trees like flif – not exactly the same but not hugely different

it's no longer MANIAC like flif, it's now MAANS

alright 👍 thanks

probably the code is your best bet if you want to know more 😄 once the lib is production-ready, we can spend some time documenting the fancier things...

Uh no <:kekw:808717074305122316> CDEF is one of the most demanding features of AV1 in terms of decoding performance.

When properly SIMD optimized, disabling it makes the decoding around 5% faster, which is negligible in the real world.

Without any form of optimizations(including O3), disabling it makes the decoding around 2x faster.

5% is not that much of a slowdown, at least for images

True, but it has to be properly SIMD optimized to massively speed up decoding performance.

<a:speedy:714294670985003018>

as is the case for most image processing algorithms - try to do DCT without SIMD...

True.

Well... they tried, see: `libjpeg` (before the inception of `libjpeg-turbo`).

for 8x8 it's still more or less reasonable, but for things like 32x32...

well, that's not true actually

Issue with AV1 decoding is that filters atm decoded sequentially, and in single thread

I'd expect entropy decoding to be the worst offender, if things translate between codecs

I wouldn't say that "no, don't use CDEF because it's computationally expensive"

Since when did I say that? In fact, I always use CDEF. Its benefits are very nice, especially with some content(aliasing in games come to mind... 😄 ) Also, single-threaded performance is the one that matters the most when profiling stuff that is hard to decode without SIMD by design.

How to interpret these plots? Encode and decode time: is the vertical axis made so higher is better? frames per second or something?

And do I see it correctly that CDEF is good according to PSNR but bad according to VMAF?

(is the horizontal axis bpp?)

In "both off" direction of comparison if from default to both loop and cdef disabled. Positive numbers under encoding/decoding means that we need more more time(red - more, green - less time). Positive numbers under metrics mean that we need spend N% more bit-rate on average to get same metric result, negative means that we need less = more efficient. So from this comparison, disabling CDEF and loop restoration, hit majority of metrics by 1-2% and give VMAF 2% and 6% increase

so disabling CDEF/loop restoration makes encoding and decoding *faster*? I don't understand

opposite, encoding 2% slower, decoding 5% slower, 1-2% decrease to majority of metrics, 2% and 6% increase for VMAF

<@!794205442175402004> here is more clear example of comparing fastest aomenc preset to slowest https://beta.arewecompressedyet.com/?job=aom-master%402021-02-10T20%3A15%3A36.010Z&job=aom-master-cpu-used%3D0%402021-02-10T20%3A53%3A10.688Z

encoding time increase 7361%, ~25% improvement on all metrics

sorry I meant slower

I don't understand why disabling stuff makes things slower

*this benchmark tool run everything in single thread

(to make accurate measurements with as less hardware "noise" as possible)

I will ask one of the devs and report back

ah thanks for the clearer example. so encoding time is actual time in seconds or something

A to B relative %

I don't trust any perceptual metric, but I certainly don't trust any of the metrics listed there

I don't even trust myself 😄

Seriously, they have decent battery of metrics, from PSNR to CIEDE2000 to VMAF

you can have a lot of bad metrics

quantity does not compensate for quality

CIEDE2000 is the only of all those methods that operates in a somewhat perceptually relevant space

yep, can be the case, what to use when?

that's a very good question, and I think we need more research to find out

my gut feeling for still images: - dssim or ssimulacra at low-medium fidelity - hdr-vdp or butteraugli at high fidelity

for ultra-low fidelity maybe vmaf might somewhat work, but you probably need an AI-based metric for that – at that point you're measuring appeal and that is a very semantic and tricky thing

I fall for appeal too often 😄

Y-PSNR is somewhat ok, so is Y-only MS-SSIM, VMAF doesn't work for JPEG XL's distortions at all -- better practical metrics with C++ implementations are: dssim, ssimulacra and butteraugli, (also NLPD I have seen give good results against test corpora, don't know if it works well with JPEG XL distortions)

I think a commonality in dssim, ssimulacra and butteraugli is that they do the gamma correction in approximately correct direction whereas most metrics do it in wrong direction or just don't care about color at all

dssim and ssimulacra work in lab, which is an ok colorspace

butteraugli uses two different colorspaces XYB for high frequency, and another ad hoc colorspace for low frequency colors

how to normalize diffs in low frequency color and high frequency color is a big thing

if you normalize it for good performance in flip tests, the metric will suck at side-by-side testing (or the other way around)

butteraugli is more normalized for flips, i.e., more importance for maintaining constant LF intensity and colors

I suspect (from observed discussions) that dssim is more calibrated with something like TID2013, which is a (AFAIK) side-by-side thing

in the usual image compression I just rather overprovision the dc if possible -- it is less information overall in typical images

... in side by side comparison the luma has more adaption than red-green

... in flips the other way around, very small LF luma differences create perceptible flicker

supposedly the neural metrics are also getting better in the low quality area (lpips)

I suspect that they are mostly for side-by-side comparison (in normalization between LF/HF)

practically all of the ~2500 butteraugli calibration images are from butteraugli scores of 0.6 - 1.5, with about 500 images between ~0.9 and ~1.1, i.e., it is only calibrated around d1, and the rest is linear extrapolation

our initial plan with pik was to deliver a system that could give decent compression in the range of d0.5 to d2 with hard guarantees of butteraugli scores, i.e., visually lossless or close

when we joined the jpeg xl effort we needed to expand the scope to low quality and complicate the codec a lot (for example with larger transforms and filtering)

JPEG XL's requirements included operation down to 0.06 bpp, which is something like distance 77 -- just not achievable what pik was designed for

later the committee understood that just the act of writing low bpp requirements in the codec's requirements spec doesn't necessarily make the codec more useful

(0.06 bpp for a full 4k image in 60 kB, whereas today the median website sends 999 kB of images)

https://chromium.googlesource.com/codecs/libwebp2/+/3610526508602f568ce97e8e8587603aee58b0aa

av1 vs x264 web rtc at 250Kbps

Haha laptop battery goes brrrrr

webp2 q89 good

How image-dependent are q-settings in webp2?

In jxl, -q actually means quality, in the sense that -q N gives perceptually relatively consistent results.

135 kb i got 145 kb

from the original jpg

not good with the image i used

In cjpeg and cwebp, the -q means quantization, and it gives consistent results in terms of quantization tables, but in terms of perceptual results your mileage can vary a lot

. jxl -d 1 is `VMAF score: 96.473198` jxl -d 3 is `VMAF score: 93.050041` 🤔

VMAF is garbage

On this

Higher distance giving lower quality is normal though

Or lower vmaf score

<:SadCat:805389277247701002>

on screenshots you can do distortions to the color to get more savings like in 0.3.1