JPEG XL

If your original is already lossy, better not destroy it further by recompressing in a different lossy way

so webp2 q89 is garbage

even for screenshots

should not be used

i prefer jxl because the bistream is finalized

VMAF does not correlate well with human perception, at least not with modern codecs. It sometimes says jxl is worse than anything else (including old jpeg) while if you look at the images the opposite is true.

what the usage of VMAF was? asking because it's explicitly said that if you have less than 1080p image you need to scale it up to fix 1080p, as default model is trained by people watching 1080p screen from 3 heights of the screen. (also vmaf type and model)

Scaling up?

That sounds like a bad idea

Main idea of VMAF as a metric that they try to simulate actual human being seeing the video. Default v0.6.1 model is trained on DMOS scores (Differential Mean Opinion Score) from people watching video on 1080p tv, at certain distance, at certain conditions. There is 4k, default, and phone models publicly available

People watching video is not a good model for still images

You tend to look at still images for more than 40 ms

That's for sure, I don't make point for it))

btw you might find interesting

https://netflixtechblog.com/toward-a-practical-perceptual-video-quality-metric-653f208b9652

Just my opinion, I think Butteraugli_jxl 3-norm is more suitable on video, maxButteraugli is suitable on high quailty still images.

It depends on the use case and what fidelity target you have in mind. Being "visually lossless" means very different thing if you do zoomed in fliptest, regular size fliptest, side-by-side comparison, or even context-switching comparisons like when you let people watch video clips

For some news article with a stock photo just to avoid dry text, probably low-fidelity high-appeal is good enough

Btw, for animations and animated emoji also well suited vector JSON-based frameworks like Lottie (although vector formats also have their limitations and drawbacks) https://lottiefiles.com/lottie

Yes, svg and co are nice if you can use it

https://sneyers.info/jxl/ has no raster images on it, I just notice

Unfortunately I've just noticed that the test animation (car) in Lottie Web Player lags when I zoom in, even though I'm on ex-flagship Samsung Galaxy Note 9

Yes, this is one of the disadvantages of vector formats - unpredictable performance

<https://encode.su/threads/3143-New-Telegram-animated-image-format?p=60825&viewfull=1#post60825> Not Brotli <:PepeHands:808829977608323112>

Brotli is nice

Also, Steam uses APNG for animated stickers <:Thonk:805904896879493180> https://store.steampowered.com/points/shop/c/stickers

Which is also not supported in discord https://cdn.akamai.steamstatic.com/steamcommunity/public/images/items/637310/979bb2133de5a89a3f7a0524f8a1d27163301f6d.png

That's the problem with APNG (and JPEG XT for that matter). Adding new functionality to something that already exists, with a 'graceful degradation', means most implementations will just settle for the graceful degradation.

You see an image, so it 'works'. Doesn't move? Too bad.

Like a JPEG XT with HDR: only see the tone mapped SDR image? Too bad.

The transparency is gone? Too bad. At least you still see an image.

Most users don't even realize they're not getting the intended experience. They don't get a broken image icon. Just a 'graceful degradation'.

So the implementers don't get a lot of bug reports and angry users.

So, what you are saying is that if we notice unintended bugs, we should report them quickly? Blasphemy. <:kekw:808717074305122316>

<:FeelsReadingMan:808827102278451241> <https://www.reddit.com/r/programming/comments/livw57/svg_the_good_the_bad_and_the_ugly/> https://news.ycombinator.com/item?id=26114863

I would love to at some point take a sensible subset of svg, define a special purpose compressed representation for it, and define a jxl file format box for it so you can combine raster and vector (or of course just one of the two) in a well-compressed way.

Call it SXL - Scalable XL

All the vector formats somehow have the problem of feature bloat and unnecessarily Turing complete stuff

PostScript could already make printers go in infinite loops

PDF extended that

SVG also basically requires a full browser to render

A simple vector representation with limited functionality but enough to do editable text and all the vector drawing needed to do fonts (bezier curves, polygons, etc), plus maybe gradients and blur, and that's it. No javascript, no hatchings or whatever weird fillings, no macro bomb stuff

WAIT SVG CAN RUN JAVASCRIPT

Yes, and you also need to implement CSS to make an SVG renderer

At Cloudinary we have to render SVG using a sandboxed headless browser, basically

I thought PDF only used a non-Turing-complete subset of PostScript, is that not the case?

<@!794205442175402004> https://aomedia-review.googlesource.com/c/aom/+/130141 🤔 ```Add tune=butteraugli Kodak Image Dataset with baseline tune=psnr avg_psnr ovr_pnsr ssim butteraugli 4.295 4.295 -1.739 -16.660 TODO: 1) replace the Butteraugli interface with the official release 2) use a test dataset with high resolution images 3) there are 7 out of 150+ test images report BD-rate loss 4) fine tune the parameters of the RD model```

Should be noted that PDF can also contain arbitrary JavaScript.

As in: as formatted text in it? :D

Nope, as code to be executed

Who comes up with that?

Anyone here use webp2 on arch linux? I get this error:

`cwp2: error while loading shared libraries: libimageio.so: cannot open shared object file: No such file or directory`

just install libimageio

I expect some `sudo pacman`, sorry I'm a ubuntu user

`sudo pacman -S openimageio` would likely do it

Linux packaging needs some kind of standardization across distributions

that'd be flatpak basically

or snap

yeah, it doesn't look like traditional distribution packages will come to a common ground anytime soon

snap still had some performance issues the last time I tried Ubuntu

But I've seen there was some work to fix that

and flatpak, I tried one but it didn't work so left it at that

AppImage was cool as an ideea

offered the best from flatpack and windows installers

when I was using Ubuntu years ago everything that I needed I was finding in rpm format. Now that I'm using fedora everything is in deb format 🤣

I have installed that. still it shows the same error

it doesn't have libimageio.so

now it is better

also flatpak now is solid

deb is more diffuse in my esperience, only Oracle go only rpm

ughhh Oracle...I'll never forgive them for destroying Sun

and torture java

Hmm, does the jxl encoder also use multi-pass for some things? https://youtu.be/MBVBfLdh984?list=LL&t=1239

define multi-pass

https://cloudinary.com/blog/how_to_adopt_avif_for_images_with_cloudinary

I don't think that speed comparison is very fair though.

By that logic, JPEG-XL with Effort=3 is much slower than mozjpg in Squoosh.

This is cool! I wonder if any resizing provider allows specifying your own `q_auto:custom` (by eg. encoding all formats via `f_auto` to the given metric like DSSIM) and choosing the slimmest file supported by the client? A lot of compute, that, but I guess giving eg. `q_auto:good` already entails multiple encodings of the same file to hit a certain metric?

>don't really need lossless for the web maybe not for photographic content, but I beg to differ for all other stuff 😆

Uh no, cjxl -s 3 is a lot faster than default mozjpeg.

I was talking about using Squoosh as a comparison. 😛

Squoosh seems to have a different scale.

Ah, yes, but even native libavif encoding with the fastest foss encoder (SVT-AV1) is pretty slow, unless you set it to fastest speed but then it is worse than webp

For nonphoto you also don't need lossless, you just need visually lossless, which is not exactly the same thing anymore

E.g. png8 made by pngquant or a lossless webp version of that is lossy, but often good enough for the web (depending on the image of course)

That's what f_auto,q_auto does: for each client, it picks the best supported format for that image

So e.g. currently on a recent Chrome, that will often be avif, but for some logos with alpha it can e.g. be best to do lossless webp.

While on Safari you will often get jpeg 2000, but sometimes old jpeg or png are better. WebP is kind of semi-broken atm in Safari, but once it reliably works again we will again serve webp to safari versions where it works

Once jxl support lands in browsers, I suspect usually jxl will be best, and all the other codecs will just be for fallback

> it picks the best supported format for that image Best = smallest at the same quality metric? What I would love is to set this metric goal myself . For two reasons: a) I suspect `q_auto:low` will be much too good for me b) I would want to vary that by dpr (lower for higher density).

We don't do a huge amount of metric tweaking, that would give too much latency on first requests. So things are based on heuristics, some trial encoding, and lots of assumptions and offline calibration

We are probably going to add a target below q_auto:low, with avif that becomes useful

Varying by dpr: that kind of already happens naturally because of the metric we use (ssimulacra)

You automatically tend to get lower encode settings for higher-res versions of the image, I mean

Default q_auto also varies based on the `Save-data` client hint (in chrome you have an option to save bandwidth, I think it is now called Lite mode): it gives q_auto:good by default, and q_auto:eco to those who want to save data

> So things are based on heuristics, some trial encoding, and lots of assumptions and offline calibration Thank you, this is very useful. One more question, not to derail even more: suppose, in theory, a client of yours would be able to run something on all the images, and provide you with an output of this thingie, prior to requesting renders. Does a thingie like that exist, that would, ideally, shift some of the compute burden from you and allow you to make better decisions, faster? I have no idea what I’m talking about, but a lot happens to images before they end up being requested (so the _latency_ could shift my way, I mean). Interesting regarding higher res imagery shifting the metric: I would only want it to shift with an explicit `dpr` para, because that way I can distinguish _higher res coz displayed huge_ from _higher rez coz displayed very small but densely_ (I know the “physical” dimensions of the images, as much as one can on the web, that is).

For now we don't do anything different for a dpr_2, w_500 (so 1000px wide) image than for a dpr_1, w_1000 image, but we might at some point change that. We would probably not really change the quality target as such, but rather the relative bit allocation (more bits to low frequencies for higher dpr).

External compute is not impossible (we do it for some addons, for example) but it's generally tricky to actually offload much burden - sending images around for processing also comes at a cost

> dpr_2, w_500 vs dpr_1, w_1000 Yeah: so we differ target quality **dramatically** between those two.

It's a bit risky to make assumptions on physical dimensions or viewing distance based on dpr though

We need better web platform level infrastructure for these things, imo

Which gets complicated by privacy concerns (fingerprinting potential)

> External compute … sending images around for processing So in me little brain, I was thinking about running some compute on the source image that I would later send to resizer together with an output of that compute (I don’t know: that much high freq detail, these many colours, etc.), and that would somehow help you immensely when making decisions around `auto`s. No extra images would fly around between us: I’m not thinking about requesting every image from a resizer at all possible qualities and formats and making the decision myself 😜 . Possibly impossible, just a thought from an ignoramus. If that would be easy/possible, you would be doing it your-side, I suppose. True, obviously, about risks around “physical” dimensions assumptions. Less for staticly sized images vs. “fluid” ones (hero etc.), but yeah., risky still. We decided to do that, because tests on HiDPI devices proved we don’t have to send 4 times the data over the wire inconveniencing users more than trying to help them. And we’ve stuck with that.

I'd love to see a word about decoding times in articles saying "go ahead, use AVIF!"

What we see happening is that customers do different things depending on the context of the image, e.g. they might e.g. use q_auto:eco for the small thumbnails in product galleries , but q_auto:best for the large images on the product page when people are considering to actually buy the thing and fidelity is important.

Yes, it is now even possible to send a 2x image and use a new exif trick to tell browsers that its intrinsic dimensions are actually 3x. For the very high dpr devices that makes sense - no point sending 4K images to a smartphone, people just don't see the difference if you send something lower res instead

I think avif decode time with dav1d is OK, except of course that avif decode is not progressive and not even incremental.

<:PepeHands:808829977608323112> https://discord.com/channels/794206087879852103/806898911091753051/810300639065407499

Is there free tool to repair jpeg? And jxl? For example i have a jpeg broken at the middle of the picture

Most of the broken images I see, are just truncated (e.g. partially downloaded)

Not much you can do in terms of repairing

Maybe in the future we'll use just lossless 🤔

Maybe. I doubt it though. Bitdepths will go up, and the fidelity we want from lossy compression will go up, but for anything that comes from noisy sensors, going full lossless is kind of wasteful since most of the entropy is in the noise that you don't see anyway (and with higher bit depths, there will be more of that).

You're right. Maybe visually lossless with be the better option for images with noise.

Or maybe a filter could be applied before saving to remove the noise

How much bit depth jxl supports ?

as resolutions go up, maybe there is a place for algorithms that encode a lower-resolution version losslessly, and high frequency corrections on top of that lossy

The current implementation of jxl supports up to 24-bit integer bit depth losslessly, or 32-bit float. We could extend it to 64-bit float and ~60-bit integer if needed (but I don't see many use cases that can actually meaningfully use that kind of precision).

<@799692065771749416>: the VarDCT approach is to encode the 1:8 image in whatever way you want, which then gets translated into "DC" coefficients (actual DC in case of 8x8 blocks, DC + some low frequency coefficients in case of bigger blocks).

you could do the 1:8 image losslessly (well, up to color transforms I suppose, you don't want to encode things in RGB)

between RCTs and chroma-from-luma, even RGB is likely not *that* bad of an idea

Yes, it's doable - but the encoder is currently optimized for XYB and it would take some effort to make it do something sensible for RGB

I have some friends that do photography on film. I know they prefer to scan the pictures in TIFF format because it has 48 bit depth.

no it has not

you are confusing 3 x 16 bit with 48-bit

it is confusing, people often multiply the bit depth by the number of channels, and e.g. talk about PNG8, PNG24, PNG32, PNG48 and PNG64

I prefer not to do that because it is very confusing: 32-bit can then mean either e.g. 32-bit float grayscale, or 16-bit grayscale with alpha, or 8-bit RGBA.

so when I say jxl supports up to 24-bit integer bit depth, that is **per channel**, not in total. In total that could be up to "98376 bit", since you can have up to 4099 channels in jxl

Oh, OK. thanks for the info

so, jxl supports 4722366482869645213696 colors?

So you are using a tool to scan the structure of jpeg? What is that name?

or more if you use floats, or if you use more than 3 channels to represent colors (e.g. you can use spot colors to reach otherwise out-of-gamut colors)

nah I usually just go by the errors libjpeg gives

Are you storing the entire electromagnetic spectrum?

I suppose you could, though XYB is designed for visible light

but I guess you could go into very out-of-gamut colors that are no longer visible light? not sure actually how this works

Idk how XYB works in this regard, but in lab, most out of gamut is simply non-physical.

I guess it's the same in XYB

Or in RGB for that matter

What are the constraints on XYB values?

Are they all positive?

No, X is basically cube root of L-M so it gets negative

Well, anything that corresponds to a negative L, M, or S value is inherently non-physical, I think.

Y and B are basically cube root of L+M and S so should be positive

I guess it depends on how you translate LMS to physical light

I could imagine something that turns negative L into ultraviolet and negative S into infrared

light outside of the visible spectrum would simply be projected to (0, 0, 0)

since the color matching functions are 0 outside of the visible spectrum

in LMS and XYZ, I mean

Yes, in that direction. But in the other direction, what do you map something with "impossible" LMS values to? Like L=big number, M=zero, S=some negative number

Actually, you don't need to go that extreme. Just any mix of some zero L,M,S, and nonzero others is non-physical.

ah, right, but maybe this can be achieved by exploiting chromatic adaptation by habituating the receptors

see for example: https://upload.wikimedia.org/wikipedia/commons/5/56/Chimerical-color-demo.svg

Well I suppose you could make a very precise display that targets individual cones cells and stimulates them in arbitrary ways

so we could have an animation that changes color to trigger this effect :p

in context: https://en.wikipedia.org/wiki/Impossible_color#Chimerical_colors

Those can be represented in LMS/XYB but just aren't caused by any combination of light frequencies, right?

that’s my understanding of it

it’s also why the CIE chromaticity diagram is horseshoe-shaped in the first place

(forgot the “horse” part)

It is quite fascinating how there is this line of purples of non-spectral colors that turns color into a circle

https://en.m.wikipedia.org/wiki/Line_of_purples

Is that the decoder used by Chrome and Firefox? and what does OK mean?

OK means it is fast enough in practice, at least for 8-bit yuv420 avif decoding.

4chan ||https://boards.4channel.org/g/thread/80290429||

lmao

> 14 KB, 758x644 https://i.4cdn.org/g/1613829104548.jpg <:banding:804346788982030337><:banding:804346788982030337><:banding:804346788982030337>

https://i.4cdn.org/g/1613851758062.png

correct answer "never"

well assuming new codecs are allowed

<@!111445179587624960> can you drop me post?

Try that image with `-d 0 -g 3 -I 0 -s 9 -P 0`

Also: my compliments to the chef for testing on such a representative corpus of a whopping N=1 images

4chan is such an uncivilized, puberal place. It's kind of funny that even there, people are discussing image codecs

https://boards.4channel.org/g/thread/80290429#p80300225 haha this made me chuckle

"alwayspng.jpg.png 3.99 MB PNG"

i thought the images looked way worse downscaled for some reason

turns out 4chan has low res jpeg previews

https://i.4cdn.org/g/1613860519557s.jpg

go on 4chan and call them names 👀

<@!794205442175402004> what you think about that blogpost? https://cloudinary.com/blog/how_to_adopt_avif_for_images_with_cloudinary

Main takeaway is that you can now try Visionular's AV1 encoder, Aurora, which is supposed to be the best one (but also a proprietary one and not easy to license either)

At the moment, avif is the best available option in chrome. Hopefully soon jxl will be the best option, but for now (since jxl is not yet supported) avif is the best option for chrome.

I feel a little bit of irritation in that one 👀, am I wrong?

Well every step forward is nice, I am not complaining. Maybe with hindsight we'll later say that it would have been better to skip WebP and/or AVIF and immediately go for jxl, but we'll have to see what the future will bring...

JPEG ULTRA waiting room

Sometimes it's a pity that wawelets methods were not as effective and promising as everyone thought before https://i.redd.it/h3fgeti8rqh61.png

4chan ||https://boards.4channel.org/g/thread/80307415||

Patches <:Thonk:805904896879493180>

<:Thonk:805904896879493180>

🤔 https://twitter.com/cramforce/status/1363182846731382790

🤦

I think I have hit my stupid-people-on-the-internet quota for this week now, thanks

"This post is advice for tuning your automatic image optimization process where if you're anything like me you'll configure a global value once and then never look back. "

are there really people encoding JPEG with a global setting of q50 or q60 and never looking back? my eyes hurt just thinking about what that means

also, you could at least mention what JPEG encoder that is – there is quite some gap between libjpeg-turbo and mozjpeg

> also, you could at least mention what JPEG encoder that is – there is quite some gap between libjpeg-turbo and mozjpeg Yep, that! Took me a while to find it in https://www.industrialempathy.com/posts/avif-webp-quality-settings/#caveats. Still, I don’t know, but I suppose libvips can use any encoder you want…

To be honest, though, there isn’t a lot of info on the interwebz on how to do multi-encoder quality normalisation for an unknown and varied image corpus… And Cloudinary’s `q_auto`, imgIX’s `auto=compress`, fastly’s `optimize` etc. are just black boxes, too, with not enough dials to play with. Easy to add a dancing dog to an image, and create 500b WebGL placeholders made of chocolate, but normalising quality… errrm… _that’s a bit hard, let’s not talk about it, I’m gonna sell my own box_. At least his post mentions stuff like dssim and he mentions low sample size. I must admit my own sample size was of similar size. But instead of dssim I used my eyes 😱 .

<@!794205442175402004>

just make a free cloudinary account

or if you're too lazy for that, do `wget res.cloudinary.com/jon/w_500,f_avif,q_70/http://blabla/orig/image.png`

where you can drop the `w_` resizing if the original is good, and you can change `q_` to something else

(note we don't implement lossless avif because it's not good anyway, so like in jpeg, `q_100` is also lossy)

Seems like JPEG2000 was the JXL of it's day

Futureproofed to hell

Essentially removing the arbirtrary restrictions of JPEG

But unfortunately no open implementation and too slow for machines of the day. JPEG XL retains the speed, but you can trade it off for better encoding.

J2K failed to replace JPEG for several reasons imo. - not sufficiently better compression (especially with early encoders it was barely better than jpeg) - too slow for the hardware of its days - too complicated to implement - no good foss encoders available (even today) - royalty-free status not clear in the beginning - still worse than png for non-photo

Can someone explain mozjpeg's trellis quantization to me?

AVIF is 10-bit max? https://cloudinary.com/blog/time_for_next_gen_codecs_to_dethrone_jpeg?utm_source=reddit&amp;utm_medium=social&amp;utm_campaign=time-for-next-gen-codecs-to-dethrone-jpeg

12-bit AVIFs work fine for me (even if there's not much point to them unless you're doing HDR) 🤷

I also wonder who is scoring things in those tables...

and maybe add a "supports inter-frame compression" in addition to the "supports animation"

It's a scoring of image formats, I don't see why being good at video would be a plus.

AVIF is limited to 10-bit, yes.

It's not clear to me to what extent inter frame is supposed to be supported in AVIF

From what I've read, avif sequence is a completely normal full on AV1 bitstream.

I think so to, but it wasn't always that way

AVIF was supposed to be implementable without requiring a full AV1 decoder at some point. I guess they abandoned that idea though.

If you feel like sharing my blogpost on various reddits or hackernews, please go ahead. When I do it myself it usually doesn't work well because people don't like self-promotion...

I got banned from r/webdev/ for sharing my blogpost on progressive decoding there

<https://www.reddit.com/r/AV1/comments/lpwpga/time_to_dethrone_jpeg_a_comparison_of_nextgen/>

I posted it on Tildes, and left a comment I hope will give correct background info to users there, let me know if I got anything wrong: https://tildes.net/~tech/vau/time_for_next_gen_codecs_to_dethrone_jpeg_comparison_with_newer_image_formats_by_co_creator_of_jpeg

"who one of the creators " there's an "is" missing

oops

Background info looks good to me, nice summary 😊

Thanks :)

can webp2 do 4:4:4 ?

Yes, unless skal changes his mind

Gonna try on /r/programming too: https://www.reddit.com/r/programming/comments/lpzxnc/time_to_dethrone_jpeg_a_comparison_of_nextgen/

updooted

Same here

+1

well, 12-bit may be out-of-spec for AVIF, but it works in practice <:Thonk:805904896879493180>

"12-bit" just wish someone made ternary computers again 😔

12 bits = 7.571157 trits

12 bits = 8.317766167 nats

Not nearly as bad as 4chan but I'm still getting cancer from that thread.

oh god

someone says BPG has the "best license"??

You're right.

You also have this lovely part from the 1st guy. 😛

`The fact that you are trying to claim dominance on the whole market instead of finding a niche (like, idk, focus on computer generated images, or viceversa) speaks of the contrast between your ambicious scope and your accomplishment so far. `

But... we **Don't** want to be J2k

> You're not going to dethrone JPEG or MPEG before Intel CPUs have instructions dedicated to doing your decoding. :-)

With video formats... ok hardware is important

But are any image formats that hardware-dependent?

darn we need intel to make some dedicated jxl instructions before i can view it on my phone

only for stuff like hardware encoding in cameras

nothing on general purpose cpus

Smartphones can encode in software

I'm curious if there are any camera JPEG chips that can be repurposed for simple 8x8 VarDCT JXL backwards-compatible encoding...

To what extent do hardware video coders reuse hardware between formats? like x264 x265 vp9 and av1

I think someone in the reddit thread is downvoting everyone with positive opinions of new image formats 🤔

Honestly, who cares? <:kekw:808717074305122316>

As long as you are reasonable, intelligent and thoughtful in your responses, downvotes do not matter.

> Spreadsheets are a niche of images.

oh lord someone is saying that new formats should have to prove themselves for 10 to 15 years before being considered for the web

> someone is saying that new formats should have to prove themselves for 10 to 15 years before being considered for the web

How? How do you prove your format without the web?

otherwise it gets stuck in a niche based on what the early adopters were

They probably think JPEG came out in 1980. 😆

So your only option is probably being lucky enough to have your image format designed before the web emerged.

Sooooooo.......

Back to GIF, BMP, and JPEG

any takers?

I'm sorry but png didn't wait until it was put on the web, therefore it is unproven

TIFF.

J2k might skirt in since it's sucess has been outside the web

Oh boy

https://c.tenor.com/-5jtyxuFnmwAAAAM/shit-jurassic-part.gif

It's not that bad actually

I already got the highlights of worst takes here before I read the thread there

Just a lot of skeptics that want PNG and JPEG to be around forever

And a few HEIC shills that think it's the best because apple uses it

That lake example is not contrived, it's what I get if I open the image in Apple Preview and do Save As HEIC with medium quality settings. I do not understand why anyone would consider such artifacts acceptable...

They don't notice it

The true tragedy of learning about image compression

Acceptable becomes horrifyingly terrible

I agree with someone there that the comparison images are not giving justice to codec differences (even after a lot of zooming; WebP blocky sky is hardly pronounced). A link to https://eclipseo.github.io/image-comparison-web/#buenos-aires&MOZJPEG=t&JXL=t&subset1 is more telling (although not all old codecs there, sadly).

https://sneyers.info/jxl/comparison/ is that better?

when I look at the water, I can see the differences quite clearly

but the limited width of the blogpost is a bit of a problem indeed

> but the limited width of the blogpost is a bit of a problem indeed True dat. > is that better? So on my macbook pro “retina” in a bright room, JP2k and WebP are acceptable and almost indistinguishable. from better codecs. I guess what I would see more useful is not a comparison at “sensible” bpp (the realistic one, showing only slight differences). I would crank it down by 33%, so that only JPXL and AVIF look acceptable. Or – even further, so even the newest codecs are on the verge of breaking up ;-). As soon as one understands that this is the same amount of bytes and see the dramatic difference in image quality – that’s the message.

maybe my standards are too high, but I consider even the JXL not really acceptable in the current example 🙂

At least that’s what I’m looking for when I want to get excited about JXL & AVIF. I always use Tiny on those comparison sites like eclipseo. To be perfectly honest, my production environment may also be closer to Tiny that it is to Small, too (yes, we are a bit radical, but it’s only a small subset of images that are kinda unacceptable and I deem the compromise worth it. That’s why I’m so excited about new formats).

https://tenor.com/view/faltas-de-ortograf%C3%ADa-eyes-eye-wash-gif-13468132

is bandwidth that much of a cost for your use case?

eclipseo tiny is really yucky if you ask me

It’s not about the cost of bandwidth. More about the democratic accessibility, if I may be allowed to sound a bit aloof :-D.

ok, but then I would ruin images when `save-data` is used, but not in the general case

also, progressive helps more for that imo than low bitrates

at least if network speed is the main bottleneck for accessibility

if the network is nice and fast but just expensive per MB, then progressive doesn't help and bitrate is all that matters

Hush, hush, but switch WebP off in the browser, make sure you are on dpr>1.3 (we reduce quality further) and look at the sky https://www.theguardian.com/news/gallery/2021/feb/19/floral-eye-shadow-and-newborn-foals-fridays-best-photos

This pic:

<:banding:804346788982030337>

Yep.

Lack of quality normalisation between encoders (AFAIU, it cannot really be done perfectly and staticly, because they differ depending on the image makeup in different parts of the quality spectrum) and lack of flexibility from resizing service’s automagic blackboxes is a big problem for us. Not sure DSSIM can take physical image density into account. But I do when ~~pulling me quality numbers out of a hat~~ using a panel of trained scientists to set quality. I am very much interested in claims that quality scale for JXL is based on some perceptual metric (would love to read more about that!). And I would love to see the best automagick blackbox dissected and reverse engineered, so that we can look into doing it on our side (most of the time, we have plenty of spare time before the image must be ready and even if it’s seconds, we could replace it later). I see Malte Ubi’s article useful for all its flaws. I get that CDNs/resizers won’t ever be offering a real per-render perfect quality setup or they would go out of business. But how can I get close to it without creating multiple renders for every image is something I’m most interested in. Also, having occasional poor quality helps for when photographers are upset we are sharing high resolutions versions of their pics. Sometimes it’s still Napster vs Metallica 😉.

Yes, I have had discussions with photographers about generation loss resiliency, where some of them actually have the opinion that generation loss is a feature, not a bug, because it acts as a copyright protection mechanism when a photo gets shared a lot

The cjxl encoder sets quality based on targets that are expressed in "just-noticeable difference" units

distance 0 is mathematically lossless

distance 1 is just-noticeable difference (in a careful flip-test)

distance 2 is probably no noticeable difference in a side by side comparison

distance 3 is probably acceptable if you don't see the original (like on the web)

distance 4 is probably acceptable if you can live with some artifacts in return for smaller/faster images

I personally wouldn't go higher than that, but you can

Nah, we all know this is the best quality you can possible want and get ;)

For Cloudinary, I implemented the q_auto feature which selects a good encode setting based on the image. For JPEG that can mean anything between q60 4:2:0 (or lower, if you don't do default q_auto) and q92 4:4:4 (or higher, with q_auto:best)

Operationally that is nontrivial, and even if you have such a system, it is somewhat error prone

With jxl, I can just set the distance to 2 and call it a day

Or to 3 for q_auto:eco and 4 for q_auto:low

> it is somewhat error prone Never-ever would I expect perfection! As you could see above, we have our share of errors already ;-).

JXL is going to make one of Cloudinary's killer features, q_auto, completely redundant

Eventually

<https://www.reddit.com/r/webdev/comments/lqi1q6/time_to_dethrone_jpeg_a_comparison_of_nextgen/>

https://tenor.com/view/sweat-sweating-nervous-wet-sad-gif-5796001

lol

JPEG: awful banding/blocking but still some detail [0.5/5] J2K: quite smeary/blurry, loss of details [1/5] WebP: sky banding, selectively blurred water, desaturation of small details [1.5/5] HEIC: selectively blurred water, slight chroma/luma shift [3/5] AVIF: stars disappear, extremely blurry water, desaturation of small details [2/5] JXL: smaller amounts of ringing or blur [3.5/5]

<@!794205442175402004> Btw, do you want to add your blog link to https://sneyers.info/jxl ?

Yes, still have to do that

The luminance issue you're seeing is most likely a chrome png rendering bug

https://res.cloudinary.com/jon/image/fetch/f_png,q_100,cs_tinysrgb/https://sneyers.info/jxl/comparison/full-jxl.png

Does that look better?

https://sneyers.info/jxl/comparison/full-jxl.png

These two images should look the same

But in mobile chrome the second one looks desaturated/brighter for some reason

I don't know why mobile chrome is rendering that png differently from desktop chrome

No, they look the same for me (Firefox).

<@768090355546587137> do you know? Is mobile chrome somehow applying both the gAMA and the ICC profile?

Firefox?

Oh firefox likely renders the other pngs incorrectly

What settings did you use to encode the JXL?

I don't remember, just picked a -d to match the size

-d 4.5 or so iirc

So we are at a stage where no browser can render PNG correctly, a format from the last century. ^^

https://res.cloudinary.com/cloudinary-marketing/image/upload/cs_tinysrgb/v1613766717/Web_Assets/blog/original.png

Does that the same in firefox?

looks same

Same as https://res.cloudinary.com/cloudinary-marketing/image/upload/v1613766717/Web_Assets/blog/original.png I mean

I think so?

Firefox by default does not do color management on images without explicit icc profile

Which is fine if you happen to have an sRGB screen

On my phone, the comparison looks ok in firefox

But not in chrome

ok, so the full-jxl.png looks fine when downloaded

but firefox darkens the image

Final rating for low bbp: JXL > HEIC >> AVIF > WebP > J2K > JPEG

i prefer AVIF over HEIC, trees on HEIC are completly flat

Yeah, we should keep in mind that this comparison is only based on one image and other photos might perform differently across the codecs.

It depends a bit on where I look, heic and avif are roughly equivalent to me

> Is mobile chrome somehow applying both the gAMA and the ICC profile? Can’t remember which browser is wrong where anymore… This is useful: https://kornel.ski/en/color for testing (doesn’t have DCF EXIF for JPEGs, though).

Firefox is wrong on that test, chrome is right

But somehow on the png output of djxl, mobile chrome renders it differently than desktop chrome

Even with `gfx.color_management.mode` set to `1` from a default, Firefox isn’t perfect. But is much, much better. Incidentally, outdated colour management just got AVIF delayed in Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1682995#c32. I’m kinda happy, maybe they will feel the pressure to finally fixed it (default got changed in one release recently, but they chickened out after reports of some smaller bugs. I have mine set to `1` since I can remember).

First image has (`pngcheck`): ``` chunk iCCP at offset 0x00025, length 297 profile name = icc, compression method = 0 (deflate) compressed profile = 292 bytes ``` Second one has extra `gAMA`: ``` chunk iCCP at offset 0x00025, length 453 profile name = 1, compression method = 0 (deflate) compressed profile = 450 bytes chunk gAMA at offset 0x001f6, length 4: 0.45455 ```

The ICC profiles also look different. First one is Facebook’s simplified sRGB equivalent. The second is described thus in Photoshop (which also displays differences between pics):

Yes, I know they are different, but they should both be basically sRGB

It must be the gAMA chunk that is upsetting mobile chrome

3 more likes...

https://c.tenor.com/LG4GkCe_p-gAAAAM/puss-in-boots-cat.gif

I would expect `gAMA` to be discarded if there is an ICC profile to use. But Photoshop displays those pics differently even if I assign standard sRGB to both. Suggesting the pixel values differ:

There could be some rounding differences, there shouldn't be huge pixel differences though

🤔

https://twitter.com/jonsneyers/status/1363889625702543367?s=19

One more like to get 4:4:4

I don't have a twitter

It's at 444 now

Now stop liking it please 😅

https://tenor.com/view/no-bugs-bunny-nope-gif-14359850

You don't need Twitter account if you want to only read the post

<@794205442175402004> Max 4099 channels? Who comes up with such a number?

I was talking about liking it

4096 + 3 color channels 😛

How do you signal having only say 2 channels?

Can signal the main image to be grayscale

And then one extra channel for e.g. alpha

The 3 main channels are special. The VarDCT part is hardcoded for 3 channels, and many things are shared between those 3 channels (they use the same block type selection, adaptive quant weights, epf filter strength, etc).

Any other channels are encoded with Modular, which can deal with an arbitrary number of channels

but you can choose to use modular for the main channels too?

The limit of 4096 extra channels is just because that's the highest number the header can signal. We could allow more but we don't know applications that could have a use for more than a few dozen

Yes, modular can also do the main channels

modular is the maniac successor?

or alternative

Yes, it's based on maans

Also contains a cool new predictor made by Alexander Rhatushnyak

It's a self-correcting weighted predictor that feeds past errors back in its subpredictors

Very good for photographic images

Can use it with a simple context model based on only the quantized max error it has to maintain anyway

That's what we do at lossless speed 3 at the moment

But it is also just available as a predictor, and this error can be used as a property in a MA tree

(for speed 3 we just use a fixed tree that is special cased to avoid branching)

Also new compared to maniac is that the predictor is not signalled per channel, but it is signalled per MA leaf node

which means you can use a predictor depending on context, which is quite cool

ha

Besides the predictor, leaf nodes can also have a multiplier and an offset

The multipliers allow you to do something similar to quantization, except it's not the values themselves that get quantized, but the predictor residuals, which is not the same thing: first quantizing and then predicting gives color banding, while doing it the other way around gives smoother gradients (with the right predictors)

> The VarDCT part is hardcoded for 3 channels So you can't use VarDCT for "true" (single-channel) grayscale?

You can, the current code will probably allocate buffers for chroma though (which is just all zeroes in the case of grayscale)

just created an HDR avif, curious how Discord and Chrome render it

okay, they don’t

yet it showed me a preview when uploading

when opened in its own tab, it seems fine on desktop Chrome, albeit with the highlights clipped quite soon when the desktop is not in HDR mode

firefox requires you to download it if you have the flag enabled

For me it didn't

do you mean if you download it and then try to upload it yourself?

Exactly (I'm talking about Windows app, not the website)

ah, that could explain it

for me, in the web version in Chrome, it looks like this:

App is probably an ancient version of electron from before avif support..

https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2021_02_15/index.html#kopenhagen-tivoli-9120&Original=s&AVIF-AOM=b&subset1

Something is wrong. AVIF is much brighter than the original.

for me, it displays a bright overlay while the UI is loading, but then the two images are equally light

Interesting. On another machine it is displayed properly 🤔

<@!794205442175402004> what do you think about AQ modes? Imo some are quite controversial and can make your image worse

AQ modes?

Adaptive quantization

in what codec?

AV1, HEVC, etc

for example in this comparison https://discord.com/channels/794206087879852103/803645746661425173/814003600698900490

used aq-mode 1, which is variance adaptive quantization

is there a description somewhere of how these AQ modes work?

adaptive quantization can only be as good as the metric you're using to select the quantweights

that's the problem when you optimize for e.g. PSNR in YCbCr: that's a poor metric in a perceptually not very relevant color space. Optimizing for that will naturally lead to nice results in PSNR-measuring benchmarks, and poor visual results (blurry, color banding in the darks, etc)

Not a lot, for example this paper http://www.diva-portal.org/smash/get/diva2:906171/FULLTEXT01.pdf

AV1

Source

AV1 `--threads=8 -b 10 --cpu-used=6 --end-usage=q --cq-level=30 --tile-columns=2 --tile-rows=1`

Butteraugli score `10.519846`

<:kekw:808717074305122316>

Original

aomenc cq20 cpu1

`3.674355`

Same image

Source

Encoded

`1.385405`

https://tenor.com/view/breakingbad-science-yeah-science-gif-5954775

https://tenor.com/view/spit-take-laugh-lmao-gif-9271200

https://twitter.com/dericed/status/1364343123271376898

and with butteruagli tune zen

does with butteraugli AV1 AOMENC looks better

like the options i'm doing

i'm not asking like 0.3.0 -s 4 -q 99.2

but at least like the vardct and modular custom settings i'm using

the first did 13 to 2 mb

or is only a bit more tuned

nothing interesting

<@!794205442175402004> Also, about other discord servers discussing compression and formats (I haven't heard about such, maybe some from enthusiasts), but mostly everyone stayed in IRC, maybe it would be convenient for those who use one client and for history to add here a bridge at least to read freenode #daala channel (but I do not know reliable discord bots for that)

it would be nice to have such a bridge

For example, the IRC bridge exists in Matrix, it looks like this: <https://app.element.io/#/room/#freenode_#daala:matrix.org> but for Discord I found only bots that need to be installed on the server <:SadCat:805389277247701002>

Nathan does sound a bit angry in that reddit comment — I wonder what those many details are that I got wrong

It is also somewhat surprising to me that AVIF is apparently not yet finalized

There is AVIF 1.0 from February 2019

Then there is the current "working draft"

https://aomediacodec.github.io/av1-avif/#change-list

Is there a schedule for when AVIF 2.0 (or 1.1 or whatever) will be released?

Yep, it would be better to specify what is wrong, also I noticed that the developers of other formats are more unfriendly when someone compares or point out the flaws of their formats, from the side of JXL it is more gentle

When we say chrome and firefox support AVIF, what exactly does it mean? Do they support AVIF 1.0, or AVIF current working draft, or is it a commitment to staying up to date with the latest version of the spec?

Will there be a different media type than `image/avif` when there is a new version of the spec?

Someone pointed out to me that I shouldn't say that AVIF is limited to 10-bit, because there exists a pull request to the AVIF spec that wants to bump it up to 12.

10 minutes ago: > Hi Jon. Your article has actually started a good conversation on the AVIF standards list about the need for more clarity around 12-bit AVIF. The spec authors were not aware that the wording in the document was ambiguous and lead outsiders to believe that 12-bit was not supported, which of course is not the case as the AOM official libavif implementation supports 12-bit just fine. > > Hopefully that will become explicit in an updated spec document, so stay tuned.

So apparently it is supposed to be 12 bit, they just didn't spec it right.

Seems like a stretch to say it was ambiguous. The spec states quite clearly that you have to declare a profile, and then it lists two profiles: Baseline and Advanced, which correspond to AV1 Main and AV1 High respectively.

But OK, great that 12-bit is officially supported then

or will be

It still bugs me a bit that the AVIF spec remains something that can be modified just like that. Maybe someone was writing an AVIF decoder and was somehow counting on that 10-bit limit (I dunno, using a compact representation of 4 bytes per pixel, which can be done if 10-bit is the limit but not if 12-bit is the limit), and now this "clarification" can mean that they need to start from scratch

AVIF looks to me like creating the format on demand in a big rush and also quickly implementing its support in browsers and where possible, with further development later when it is already in use (without proper time to discuss and develop the specification and needed features before that) Perhaps this is as a quick response to the HEIC replacement or a request from Netflix as they urgently needed HDR

where does that come from?

Reddit

Unlord replied to you

I think it's that: 1. 10-bit support max, while AVIF actually supports 12-bit natively just fine. It just needs better tagging support. Of course, the fact that the AVIF spec didn't explicitly show that, which is odd. 2. Decoding speed isn't actually much of an issue if you use say dav1d. It not only scales very well with resolution, but single-threaded decoding performance is quite good(I still haven't found a way to test out decoding speed with avidec yet though). 3. AVIF doesn't actually use purely independent tiles unlike HEIF. It's actually an AV1 feature that tiles can still access a lot of information from other tiles IIRC. That means that at higher resolution, it doesn't actually pose much of a problem(Scope provides a better explanation in this regard.) 4. You only tested AVIF using the reference encoder. That's not much of a problem when there's only one encoder(in the case of JPEG-XL for example... for now <:Stonks:806137886726553651> ), but there are encoders that actually do a better job in some scenarios in the case of AV1.

Other than that, your article is quite accurate.

It's excellent. 👍 unlord must have replied rather quickly and not provide that information. 🤔

2. For me it is noticeably slower than VarDCT JXL on large images (also take into account that dav1d is already a very optimized decoder and JXL decoder still has a lot of room for further optimization) 3. As far as i know for fast hardware encoding/decoding will use single-image grid or when the image is very large and everything is exactly the same as HEIC, it is not a comparison with regular tiles

Tiles: ```--tilerowslog2 R : Set log2 of number of tile rows (0-6, default: 0) --tilecolslog2 C : Set log2 of number of tile columns (0-6, default: 0)``` Grid: ```-g,--grid MxN : Encode a single-image grid AVIF with M cols & N rows. Either supply MxN identical W/H/D images, or a single image that can be evenly split into the MxN grid and follow AVIF grid image restrictions. The grid will adopt the color profile of the first image supplied.```

What would be the advantage of using a grid image? I've mainly been using --tiles for better threading at higher resolutions, since it didn't create the same discontinuties.

There's av1 level tiling and then there's heif level tiling, no?

At least that's how it is in heic

I'm not sure about hardware encoding (as far as I know the grid implementation is also much simpler), but tiles cannot be used to increase the resolution, for example if the image needs more than 8k

<@!321486891079696385> a standard grid pattern makes hardware encoding and decoding actually worth while

that's why HEICs produced on iphones are actually 256x256 (IIRC) images tiled in HEIF