JPEG XL

Any news?

This page has appeared: https://storage.googleapis.com/avif-comparison/index.html I don't see it being referenced anywhere yet, but it looks like this could be "the data" the Chrome team promised...

(how did you find the link if it's not referenced anywhere?)

I type random urls all the time to see if something pops up 🙂

https://storage.googleapis.com/avif-comparison/images/Kodak/rateplots/Kodak_avif-jxl_t-8_avif-s6_jxl-s7_ssimulacra2__bpp-0.1-3.0.png

that's a funny aggregation method

so this plot shows jxl being 10% smaller in the "web quality" range (ssimulacra2 between 60 and 80), yet the conclusion is that jxl is only 1% smaller overall since at extremely low quality (ssimulacra2 = 30), apparantly avif is 6% smaller than jxl on this image set

presumably the aggregation is done with the standard Bjontegaard delta, although the range is of course debatable

on another image set they get this: https://storage.googleapis.com/avif-comparison/images/subset1/rateplots/subset1_avif-jxl_t-8_avif-s7_jxl-s6_ssimulacra2__bpp-0.1-3.0.png

~~according to the verification links, it is using "encoding point" of 40 at least for some tests, which seems to indicate `-q` in libjxl. the problem is that, `-q 40` is `-d 5.5`...~~ (see below)

so basically avif is 8% better at 0.3 bpp, while jxl is 16% better at > 0.6bpp. Aggregating that by saying jxl is 4.5% better overall is a quite misleading if you know that the median avif image on the web is 1 bpp (source: https://discuss.httparchive.org/t/what-compression-ratio-ranges-are-used-for-lossy-formats/2464)

ah, it seems to use 40 as a representative example *for following tables*, not the main plots

"JPEG XL effort 1 and 2 were not measured because the JPEG XL team discourages their use." really?

well in vardct they are just the same as effort 3

ah, so not actually "discouragement" but pointless to measure, got it.

Probably best not to share this url yet, I don't know if they intended to publish this already so better to wait until they announce it somewhere

yeah, though I think someone in this server *will* share it anyway...

quality tables do seem to indicate which metric libavif and libjxl prioritized over...

I'd like someone at Chrome to take ownership of this data and stand behind it -- that happens best if they share it in the chromium bug or blink-dev like they proposed to

also it can be that this is not the data they plan to publish even if it looks like it could be

interesting, how did they make AVIF encoding perform so fast? really small images? fast preset? lots of threads?

or maybe AOM performance has been optimized a lot since I last tried it

and also the decoding speed, or maybe this is just a chrome decoder implementation https://storage.googleapis.com/avif-comparison/images/main/charts/Main_DecodeMPs_Chrome-92-Win_AVIF_JPEGXL_WebP_900w.png

well tbf jxl decoding in chrome was buggy

and for lossless only slow modes are compared

doesn't change much for decoding speed anyway

of course, aom speed 8 is like 150 fps at 1080p and dav1d is like 900 fps

yeah, but in a such comparison lossless avif looks also not bad (larger, but noticeably faster)

their test only works on chrome

Zoltan also reduced memory pressure if I understood correctly

> The timing was performed using the ImageDecoder API on Windows Chrome version 92.0.4496.2 and Mac Chrome version 92.0.4497.0.

that's a pretty old Chrome version, from July 2021 back when we just had the first initial jxl support in Chrome

how old is the page? is there anywhere a date can be found?

no idea, but they did use relatively new versions of the encoders: https://storage.googleapis.com/avif-comparison/verification-links.html#Codecs-Versions

So they based this on faulty implementation of jxl instead of reporting or fixing it, used an old version of Chrome with relatively recent versions of the encoders, and focused on speed showing AVIF somehow being faster than WebP - something numerous CLI and other non-Chrome tests have shown the opposite results for AVIF (at least for encoding)? Why not use latest Chrome? Why not get the implementation fixed and then test? Why not include comparing features as well? They also seem to focus encoding the images at very low bbp, why not at a range of bbp to show low, medium, and higher qualities? Most of the sites I've seen use higher quality images and would never consider the messy, artifacted, and blurry low quality versions. Something smells fishy with this.

if they used an old version it is likely that they had the results ready and didn't want to repeat

it could also be that the 92 API version is in 109 Chrome, perhaps they have different numberings

They do consider a range of qualities, but if you let that range start at 0.1 bpp like they did, then effectively you are putting a lot of emphasis on the lowest qualities.

https://storage.googleapis.com/avif-comparison/images/Kodak/rateplots/Kodak_avif-jxl_t-8_avif-s6_jxl-s7_ssimulacra2__bpp-0.1-3.0.png

the thing with Bjontegaard delta rates is that they tend to weigh the low range way higher than the high range when bitrate-distortion curves have shapes like they usually do

you take the y range where both curves have values, so here that would be from ssimulacra2=9 to 88 or so

so here it would be from 0.2 bpp to 3 bpp or so

then you look at bpp logarithmically

and basically look for every ssimulacra2 value in the range what the % gain (or loss) is, and average those

this means that ssimulacra2 values 9 to 48 get the same weight as ssimulacra2 values 49 to 88

You are correct. At first glance it looks like they are promoting low bpp.

so basically below 0.45 bpp or so is counted as equally important as everything above 0.45 bpp

git aomenc with git libavif and libyuv. A part of how slow all-intra coding was that converting to RGB to YCbCr was done in floats, which took a non negligible amount of time.

So the majority are using 1 or above... basically the opposite of what AVIF was designed for.

likely many AVIF images in the wild are using more aggressive compression than what an "average" web developer would use, considering these are still mostly early adopters who probably mostly want to save bandwidth

the median JPEG on the web is 2.06 bpp, while the median AVIF is only 0.96 — this is not because AVIF compresses twice as well; I estimate that if JPEG quality and AVIF quality would be the same, the median AVIF would be around 1.5 bpp

but very likely the quality settings used by current AVIF usage on the web are somewhat lower than the quality settings used by JPEG usage

do you happen to have some kind of graph that compares average/median butteraugli distance and bpp between libjpeg, AVIF and jxl? <@794205442175402004>

or some other objective metric

it's really hard for me to imagine what a 1 bpp AVIF image looks like compared to 1 bpp JPEG

I really like that JXL's quality setting is roughly comparable to libjpeg, webp is very similar also but AVIF 1-64 scale is really alien to me

I have lots of graphs here: https://jon-cld.s3.amazonaws.com/test/index.html

this is the plot I got — not the same image set, but interesting to compare

so one thing is that the current git version of jxl produces the blue line, while the version they tested is the red line

(the speed difference is not relevant because I didn't test that accurately so likely it's mostly just different conditions in general load that cause most of that. the compression difference is relevant though)

you can see the red curve has two 'bend' points where there is a bit of a discontinuity, this is also the case on their ssimulacra2 plots

I'm still confused how all the metrics I've seen, and just using avif compared to other encoders I always notice avif being slightly slower (for smaller images) to extremely slower (for large images)... yet Google is saying AVIF is faster than JPEG and WebP?

dav1d is a pretty fast decoder but chrome doesn't use it

encode speed they're comparing with 8 threads, and then they're putting s0, s6 and s9 there for avif

what they don't say is that s9 is basically useless for still images: compression is worse than jpeg at that speed

I've set the CLI encoder to use 4 and 8 threads before and didn't notice that big of a difference.

So they're using the speed of the fastest setting in their graphs - a speed pretty much nobody would normally use?

chrome uses dav1d, at least for desktops

well they're also showing slowest and default

the thing is, in jxl you don't really need to go to slowest speed, in fact probably cjxl effort 6 is basically just as good as effort 9 for lossless

https://storage.googleapis.com/avif-comparison/images/main/charts/Main_TranscodeMPs_vs_Effort_AVIF_JPEGXL_WebP_MozJPEG_900w.png

huh, I thought they still used libaom

Still Doesn't make sense. I use default most of the time and would expect the graph to be JPEG, WebP, JXL, then AVIF. Not JPG, WebP, AVIF by a hair, then JXL.

they used libgav1 initially and then switched to dav1d, no?

what about libavif?

Chrome uses dav1d.

dav1d.

libavif uses dav1d internally?

Yes.

makes sense

does chrome use dav1d directly or via libavif

Via dav1d.

uh

for avif images specifically, not for av1

Yes.

<:KekDog:805390049033191445>

I should check.

does it have its own demuxer and use dav1d or does it use libavif to demux/decode

if I'm not mistaken there is a own demuxer

these are the speeds I get for default speed mozjpeg, webp, jxl and avif

these are their numbers

this is what I get for "fastest" speed (except webp where I only went down to m2, I didn't try m0)

so yeah I don't know what exactly they did there. Perhaps this is particularly poorly compiled jxl build, without proper SIMD or something? no idea how they get avif -s 9 that fast, maybe my avifenc/libaom versions are not recent enough...

I used libavif v0.10.1 with aom [enc/dec]:3.4.0 They used libavif 0.10.1-187 with libaom 3.5.0-246

yes, I agree, and this is also what I measured when I was looking at encode speed. I don't know how they got those numbers but I cannot reproduce anything close to such numbers.

Oh yeah, that alone makes a big difference.

I think they spent those weeks making that stuff not so obvious fake. And not succeeded

A factor 3-4 speedup?

~~Yes, considering CPU-10 or s10 is now a thing ~~ 🙂

They only tested s9 though

Sorry, I always forgot s10 is an RT(realtime) only preset.

What does that mean?

`--cpu-used=<arg> Speed setting (0..6 in good mode, 5..10 in realtime mode, 0..9 in all intra mode)`

I see, so it's only doing something different for inter, or something like that?

Yes. Good mode = slow normal video coding, used in animated AVIF. RT = low latency video coding, used for realtime streaming (as in low latency for stuff like video calls, not video streaming). All-intra = single image AVIF or ProRes like encoding.

Anyway, I started an empty repo at https://github.com/libjxl/codec-comparison Let's put some scripts there to do our own comparisons of speeds and metrics.

their selection of test sets is also a bit weird

https://storage.googleapis.com/avif-comparison/noto-emoji.html

that one in particular

rasterized vector graphics, with the alpha channel dropped? why?

Wait a minute: `noto-emoji`

No wonder it's so much faster! It can do massive amounts of pruning and use screen content tools.

This is not really a representative use case of average performance.

Because otherwise, decode performance would take a fat dump with AV1 😂 Not that extreme, but alpha decode performance is better than JXL than it is with AV1.

that set in particular looks like suspicious cherry picking to me

Yes

It is. Screen content tool analysis is extremely fast with these types of images, and can be pruned to an obscene degree because they only include some patterns.

It's very biaised.

the old Kodak set is also not particularly good. Those images come from 4:2:2 sources, which will help make the 4:2:0 they're doing in avif (and webp) introduce fewer artifacts since horizontal chroma resolution is already reduced anyway

so the only decent test set they have is the "subset1" one

Emoji without alpha? 🤣🤣🤣

yeah, very useful

https://en.wikipedia.org/wiki/List_of_Intel_Xeon_processors_(Sandy_Bridge-based)#Xeon_E5-2690

> All of the timing was done on an Intel Xeon processor released in 2016, Intel Xeon Processor E5-2690 v4 clocked at 2.60GHz.

according to wikipedia, that processor was not released in 2016, but in 2012

The **V4** part is very important. https://www.intel.com/content/www/us/en/products/sku/91770/intel-xeon-processor-e52690-v4-35m-cache-2-60-ghz/specifications.html

ah

why do they number these things in such obscure ways

ok so it is indeed from 2016, and it does have avx2

I was trying to understand where the difference in speeds is coming from

anybody want the kodak image dataset in (almost) 4k?

I'm gonna share it anyway https://drive.google.com/file/d/1lZpLEbTNPOIu_qoJGAFRJBmKvzMQ7OKa/view?usp=sharing

lol, what?

is that AI upsampled or something?

I did this for a blog post a few years ago

no, it's the original files from the disk, I found them on the internet. they are a bit color corrected to match the famous 768x512 files because video cd images always come out really faded

oh really? I didn't know those existed

do you have the original files still?

I think this is the source I used https://www.math.purdue.edu/~lucier/PHOTO_CD/

you can also download the original disk image on archive.org https://archive.org/details/kodakphotocdsoftware1992

Thanks!

It's weekend now so I likely won't have time to do anything until Monday, but if someone wants to start putting codec comparison scripts there, feel free to go ahead!

They posted the link: https://groups.google.com/a/chromium.org/g/blink-dev/c/WjCKcBw219k/m/-cq3Rrk8CAAJ

it would be nice to see some kind of an argument in addition the graphs, only argument on the results page I see is > JPEG XL has better lossless compression compared to WebP, which has better lossless compression compared to AVIF for these images. Here is another lossless comparison, which compares more files and image codecs. The sheet was found on encode.su in a post from 2020.

well, in addition to this which is more like criteria for format inclusion rather than an actual argument > When we evaluate new media formats, the first question we have to ask is whether the format works best for the web. With respect to new image formats such as JPEG XL, that means we have to look comprehensively at many factors: compression performance across a broad range of images; is the decoder fast, allowing for speedy rendering of smaller images; are there fast encoders, ideally with hardware support, that keep encoding costs reasonable for large users; can we optimize existing formats to meet any new use-cases, rather than adding support for an additional format; do other browsers and OSes support it? > > After weighing the data, we’ve decided to stop Chrome’s JPEG XL experiment and remove the code associated with the experiment. We'll work to publish data in the next couple of weeks.

Wow, they are actually somewhat incompetent. Why not test it outside the browser, provide easily repeatable settings and encoder/decoder versions?

Also, I am sure they did not make sure to lock down machine speeds like I do.

Also, I do not believe for one second the lossless decode tests.

Exactly, they are basing it on data that I am not sure is reproducible (would love to see if people can replicate the speed they are reporting) vs a faulty implementation of libjxl in Chrome that is known to be slow, and ignoring all the other factors like features, that the vast majority of images are outside the bbp range AVIF is designed for, that WepP is not perfect at it either and sometimes larger than PNG at lossless while I have yet to see a jxl that is larger than png (that wasn't a bug). They are also trying to push everyone to a random email they set up so they can ignore them? This seems like bad faith all around.

Yes, and this is why I shall finally release something special on December 13th <:YEP:808828808127971399>

For decoding, I think it would be okay the base the results on the browser implementation since it's a web browser we are talking about. But the tests should be done in good faith, acknowledging things like the fact that the implementation isn't currently optimized

The issue is not optimization. The issue is that they are making tests that are hard to repeat on purpose. Using the decoder/encoder directly allows for very easy testing, can't have that.

I'm not too worried about decoding, even a fairly slow decoder could easily up with most things on the web - though very large images would likely be an issue.

yeah, I agree. would you say that using the encoder directly is comparable to rendering the image in browser? I would guess so since it's the same library and I doubt the browser will add that much overhead

These graphs are obviously bogus on first glance. Right off the bat they are trying to tell us AVIF encoding is faster than JXL encoding.

And why would they strip the alpha channel from these emojis...

I dunno what this is supposed to be other than something for everybody to laugh at

that emoji perfectly summarizes their "testing"

Now I am curious!

Tweeted some initial thoughts on the released data: https://twitter.com/jonsneyers/status/1598774837723422721?t=SHQWI3tzhpuXDpkwXO90rA&s=19

What is the most baffling to me is this: 1) Jim says Chrome made the decision by carefully weighing all the data. 2) I say maybe there is a bit of bias here since I observe Chrome decision makers are also AV1/AVIF proponents. 3) Chrome releases the data, which apparently is a website created by "The AVIF team", and we "can submit our feedback to avif-feedback@googlegroups.com".

I mean, how more obviously can you confirm that my concern was quite justified?

I don't think they ever denied that they were AVIF proponents.

But yeah, that is a strong conflict of interest.

how can they say with a straight face that that average avif encode speed is faster than average jpeg xl encode speed

are they assuming hardware encoding?

No, this is presumably just avifenc

They can't be, as the hardware is 9th and 7th gen intel.

they plotted speed and effort on the same axis without mirroring one of them

so these graphs are impossible to read

?????

One thing is that they average speeds over all quantizer settings, if I understand correctly. Since avifenc gets faster as you lower the quality, that might influence results?

Yeah that's an extremely strange thing I also noticed

biasing toward low bpp sounds exactly like the kind of thing they'd do

given that they're avif proponents

Why did they do a whole bunch of different quality settings and then average them, including quality settings that make no sense

they also have a separate setting which is just "mozjpeg" but that's I'm assuming averaging across quality as well

Also they do 8-bit 420, which helps for speed

4:2:0 is like half of the pixels, but also unrealistic

since it doesn't really gain you anything

I mean you could encode JXL as 4:2:0 as well but there's no real purpose

But still, even then the speed results seem a bit strange

avif is notoriously slow to encode

so this strikes me as very odd

> The effort and speed for JPEG XL and AVIF transcodes listed below were selected because the transcoding MP/s are within ~10% of each other.

then they compared the quality at those same "similar speed" settings

which should in theory bias *against* avif if done properly, since avif is much slower than JXL

but apparently not

Well if you look at the plots of the reasonable metrics (dssim, ssimulacra2, butteraugli 3norm), they do look quite good for jxl generally

they do, yea

butteraugli in some ways is an outlier because libjxl targets butteraugli

I don't understand the All Speed/Effort chart, but with the MP/s vs Effort ones with just Best quality, default and on the fly they seem reasonable.

but the others seem like they are good indicators that jxl still is better

Does AVIF scale better with multicore?

But in the BD rate computation the < 0.4 bpp range gets way too much weight, so the overall numbers they put in the table tend to mostly reflect what happens at 0.2 bpp, not what happens at useful quality settings.

Yes, the libjxl encoder is not fully parallelized yet.

I don't think mozjpeg has a speed/effort setting like the other codecs

Exactly, what the heck are they trying to accomplish by testing settings that no one would ever use and then averaging everything?

Also this comparison subtly implies that the only thing that should be considered when comparing these codecs is MP/s.

JPEG-XL completely destroys AVIF in terms of MP/s but that's not the only reason why people want it

And this comparison makes the bizarre claim that AVIF is faster than JXL

So it's just a really weird laughable thing they've released here, no wonder they hesitated to show anyone this

I guess the thing is that you care about speed at a particular quality, as otherwise you could make a speed -1 that is just a memcopy..

The biggest advantages JXL brings over AVIF is features, such as progressive decoding.

I just read the "case for JPEG XL" again on cloudinary blog and it's so well argued that google's effort makes me laugh. they've just said "we looked at the data" and then they give a bunch of graphs and numbers with no context

In my opinion

But the avif devs are implying in this comparison that the only factors they consider is raw megapixels per second. Which JXL still completely destroys AVIF at, despite what this ridiculous "comparison" would have you believe

Maybe instead of laughing at them like they probably expect everyone to do, maybe we should take them seriously and be like, "Oh! The Data! Yes I see now, AVIF is clearly faster than JXL. We have been sorely beaten."

Thanks Google, you proved us all wrong

Really dodged a bullet there, we might have been able to optimize existing JPEGs to take up less space, and we might have had progressive decoding, but thankfully, you showed us all The Data

So now we can all use AVIF forever, the FASTEST image format

Thank God for Google

To an AVIF-specific email address so the discussion won't be public and we can ignore it.

similarly ssim (or ms-ssim) is an outlier, because much of the development process for AV1/AVIF targeted SSIM, and SSIM has weird defects (like applying gamma correction again for already gamma corrected data, leading to misbalance of white and black areas -- if you do the same mistake in the codec, you have huge advantage, and they have those heuristics for local RD control)

https://tenor.com/view/trash-trash-bin-bye-garbage-done-gif-17363404

the avif team "processing" all of your thoughtfully written messages

Yeah, it'll be quite the treat. Jyrki already saw a bit of the text earlier back April, but that was when I was less experienced, less knowledgeable, and a bit dumber than today.

what is the situation with Сhrome currently?

Scroll up 🙂

Yeah when I saw them mention MS-SSIM comparisons I just read that as "here are some random numbers that don't add anything to this comparison lol"

Why is this even here? Okay, skipping, next...

Some people on reddit are confused, saying "Well Google and Cloudinary both have studies that say opposite things!"

Difference is Google's study is a joke and Cloudinary actually had real human judges looking at images and selecting which one they prefer...

And a huge sample size...

Google compares ms-ssim of some emojis with their alpha channel removed. Like something a 15 year old would have done for fun on their laptop

Also, looking at BD rate is kind of misleading, even for those emojis

You can see that while this avif encoder is doing great here in the low bitrates, it plateaus and cannot actually reach the highest qualities (no doubt 4:2:0 plays a big role in that)

For BD rate computation, you just ignore the range that one of the codecs cannot reach

So basically it does not matter what happens at psnr>37, for the purpose of BD rate that range just gets ignored

Basically you could make a codec that can only do a blurhash but it does it with very low header overhead, and it would beat anything else when computing BD rates

Same thing here: what happens at butteraugli distance < 2 simply doesn't count since avifenc 4:2:0 just doesn't get there

(then again butteraugli distance > 11 also doesn't count here since they didn't take jxl beyond that)

The main problem with BD rate computations imo is that they depend heavily on how the metric spreads its values — i.e. you could make a variant of any metric that just applies some monotonic but nonlinear rescaling to its scores (so in terms of Kendall/Spearman correlation nothing at all changes) and you would get drastically different BD rates

In general, metrics tend to produce values that are further apart as perceptual quality goes down, e.g. the difference in JNDs between SSIM 0.98 and 0.99 is larger than the difference between 0.48 and 0.49.

This means the BD rate will in practice effectively put more weight on the lower qualities than on the higher ones

Another thing that is interesting to observe: multithreaded encode in avif does come at a 1-3% cost in compression according to this data

Compare e.g. https://storage.googleapis.com/avif-comparison/images/subset1/rateplots/subset1_avif-jxl_t-1_avif-s7_jxl-s6_msssim_2022-09-01_23d3bb6d_bpp-0.1-3.0.png

versus https://storage.googleapis.com/avif-comparison/images/subset1/rateplots/subset1_avif-jxl_t-8_avif-s7_jxl-s6_msssim_2022-09-01_23d3bb6d_bpp-0.1-3.0.png

Same encode settings, same metric, just single thread vs 8 threads

If you look at the last row of each table (the only one that can be directly compared), you see that basically all the BD rates get 1 to 3.5% worse for avif when using 8 threads compared to 1.

While libjxl produces exactly the same jxl file regardless of the number of threads used.

37 db was a great rule of thumb to aim for with analog SDR/TV times. With correlated noise, modern noiseless reproduction and transmission systems, improved cameras, improved dynamics (HDR), 37 db (something like 5.2 bits of dynamics/channel) is no longer enough -- instead we are upgrading our 8 bit systems to 10, 12, 14 and more bits.

but if you take an old 1990s textbook on video broadcast quality, 37 db comes at you and that's what to aim for

> In general, metrics tend to produce values that are further apart as perceptual quality goes down, e.g. the difference in JNDs between SSIM 0.98 and 0.99 is larger than the difference between 0.48 and 0.49. So basically at high quality targets, it would be better to view the results on a log scale?

I think instead of looking at BD rate, which highly depends on the selected bitrate interval and the way metrics are scaled, it makes more sense to look at specific fidelity points (preferably in terms of MOS but if needed in terms of metric scores) and see what is the improvement at each of them.

Where the selection of fidelity points should be data driven, not "let's pick some points where avif does well"

E.g. you can estimate at the actual distributions of fidelity used on the web by estimating jpeg quality (based on quant tables) or by looking at the bpp buckets for the various formats. You generally have no way to get the uncompressed original to compare against, so it would have to be an estimation, but this should be accurate enough.

Chrome likely has quite detailed telemetry data on bpp of images it sees in the wild (i.e. including the parts of the web that the http archive misses since it's behind login pages)

I would just take like the p20, p50, p80 estimated fidelities the web currently uses, call them low, medium and high "web quality", and compute the compression gains at those points

That would be way more informative than BD rates imo

Why speed7 for AVIF? I thought that 6 is recommended.

well the lower the better of course in results, but these plots are for similar-encode-speed and then they matched jxl e6 to avifenc s7 it seems

I was wrong, the source images are not 4:2:2 here, but I wouldn't say they are completely good images either.

https://en.wikipedia.org/wiki/Photo_CD

If I understand correctly, these images are from 1991 and predate digital cameras. These are scanned analog photos (presumably scanned from negatives).

film based images have different microstatistics than images from digital cameras

in film based photography the microstatistics relate to grains on the film, diffusion processes and photon/capture noise in acquisition

the colors have limitations in film based photography and often they look flat and '1970s' like -- compression is very different in highly saturated colors than in grayer colors

in digital photography there are no grains, but the debeyering algorithm produces its own local statistics -- it can be a 10 % difference between algorithms on how good they are to store debeyering algorithms (in lossless application), particularly WebP lossless is not good at storing debeyering artefact-related microstatistics

I think we should not be using 1990s data/acquisition methods for evaluating what to do in image compression 2023 onwards.

We should use good quality modern cameras to create input

Some of the data is not from modern cameras, but from a image processing software. There, commonly people subsample to make smaller versions and also commonly people sharpen the images a bit. Some people blur the u and v planes to avoid chromacity aberration to be present -- or other kinds of red-green noise which tends to surprisingly often survive the debeyering approaches

ideally a test benchmark would have these both cases: photographs from high quality cameras and photographs that have been say 4x4 or 8x8 subsampled and sharpened moderately

my 'jyrki31' dataset is a 4x4 subsampled and sharpened dataset where originals were taken with a Canon DSLR, i.e., it tries to simulate what could happen in ecommerce

we used 'jyrki31' to do most of the quality evaluations during jpeg xl development

I considered it was a better goal than camera originals, because I didn't want JPEG XL to reflect in its internals some specific debeyering algorithm(s)

i like the sound of this comment https://bugs.chromium.org/p/chromium/issues/detail?id=1178058#c266

Well, it *is* coming from one of the creators of JPEG XL, Jyrki Alakuijala.

~~also i didn't realize this, but they had sent messages right above what i sent lmao~~

~~well i also didn't realize that they weren't a co-author of that attention center model thing~~

oops

lol

> In fact, I'd love to have this bug accumulate somewhere around 3000 (if not 20000) stars so that it would be always reminding developers that JPEG XL is worthy of reconsidering for Chromium's priorities. Let's not use false methods for achieving that however, just spread the message within photographer and web developer communities. but, about that, it does not really show the real interest, is more about format recognition and the number and regularity of articles/blogs on public resources and besides, if new articles or voting requests would stop appearing then the star gaining would slow down drastically and it would take forever to get 3000 stars, and even if Jpeg XL is kept mentioned all this time, based on activity it might take 3-5 years (or even more) to achieve 3000 stars

we can get 3000 stars faster -- it seems to me that every time there is a popular person reaching to a new community of 10'000 we get about 1000 likes to the post and about 100 stars to the bug, it would be about 30 such communities to get 3000 -- it is just critically important to inform the community at that stage that they can star the bug with their gmail account without registration

of course it is a bit childish to set a number goal on the stars, but one can always dream -- making it the top bug in blink would be the right thing to do from my viewpoint at least 🙂

Ah, I see how they arrived at those speed numbers now.

``` $ time avifenc ClassA_8bit_p06.ppm.png t.avif -s 6 -j 1 -y 420 --min 0 --max 63 -a end-usage=q -a cq-level=1 -a tune=ssim real 0m5.621s user 0m5.555s sys 0m0.063s $ time avifenc ClassA_8bit_p06.ppm.png t.avif -s 6 -j 1 -y 420 --min 0 --max 63 -a end-usage=q -a cq-level=63 -a tune=ssim real 0m1.239s user 0m1.195s sys 0m0.044s ```

they take the average going from cq-level 1 to 63

but the difference between those two extremes is a factor 5

some points are higher quality than what is useful, but many more points are lower quality than what is useful (anything above cq-level 40 is pretty much garbage)

jxl has more or less constant speed regardless of quality setting, but avif, mozjpeg and webp all get a lot slower as quality goes up

so if you determine the speed mostly by looking at how fast it is at the unusably low qualities, you get a bit of a misleading number...

That "test a whole bunch of useless quality settings and average them all" seemed really bizarre from the beginning. Now we know why. Because they are lying.

Well, more specifically, they are cooking the numbers so they can make the laughable claim that AVIF encoding is faster than JXL

Testing a whole bunch of completely useless settings and averaging them into the results is called cooking the data

Again, this might not be intentional. Doing things from cq-level 1 to 63 in steps of 1 is the most straightforward thing to do and just averaging numbers is also the most straightforward thing to do so it is entirely plausible that they did things in good faith (but still in a not very relevant way)

doing the speed testing at use-case relevant data rates is central to getting it right -- at JPEG XL we spent a lot of thought on this, and JPEG XL tends to get faster (not slower like AVIF and WebP) at high bit rates

well not really faster, at some point as you go higher quality than d1 it does become a bit slower since there's just more entropy coding to be done

but the speed curve is very flat compared to other codecs

E.g. this is mozjpeg: ``` $ time cjpeg -quality 10 ClassA_8bit_p06.ppm.png >/dev/null real 0m0.521s user 0m0.486s sys 0m0.024s $ time cjpeg -quality 89 ClassA_8bit_p06.ppm.png >/dev/null real 0m1.507s user 0m1.463s sys 0m0.030s $ time cjpeg -quality 90 ClassA_8bit_p06.ppm.png >/dev/null real 0m2.074s user 0m2.047s sys 0m0.027s $ time cjpeg -quality 100 ClassA_8bit_p06.ppm.png >/dev/null real 0m6.798s user 0m6.754s sys 0m0.044s ```

measuring it at q10 will make it look 4 times faster than measuring it at q90

above q90 it gets another 3 times slower

the problem with the avif team's methodology to measure speed, is that they implicitly consider mozjpeg q1-q20 to be just as important as mozjpeg q80-q99.

while in practice of course nobody would use mozjpeg q10 for anything at all while the upper end of the range is where most of the use cases are concentrated

Regarding decode time: does anyone know if 420 avif gets decoded to a yuv420 buffer in chrome, effectively leaving the chroma upsampling and color management to the renderer, so it would not be included in the decode timing?

<@795684063032901642> maybe you know this?

I don't know, maybe take a look at https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/image-decoders/avif/avif_image_decoder.cc to figure it out?

yep

***WHY HAVE I ONLY DISCOVERED THIS NOW***

When libavif decodes AVIF image, output is usually YUV. User can call avifImageYUVToRGB() to convert it. If libavif is built with libyuv, the conversion is very fast.

It even works on Facebook Messenger... wow.

for proper color management (and alpha blending, if applicable), something will have to convert the yuv to rgb and do the color conversion to display space. I guess this can be done on GPU and very efficiently, but still, it's not really an apples-to-apples comparison if you just ignore that part...

looking at that code, it looks like they indeed have code paths that keep things as yuv420, and then other code paths for cases where they cannot do that (like when there is alpha, the colorspace is not one of a few supported ones, or it's HDR)

so it would be interesting to replicate their browser decode timing experiment with: - a recent Chrome version - different kinds of avif: {sdr, hdr} with {420, 444} and {no-alpha, alpha} - same different kinds of jxl, except there is no 420, so just {sdr, hdr} with {no-alpha, alpha} - trying on both x86 desktop/laptop and on an ARM phone

can you measure decoding latency from javascript?

there must be a way

https://storage.googleapis.com/avif-comparison/verification-process.html

they have some example js code there

well, that *does* measure YUV420 conversions and the like, then

``` function renderImage(result) { const imageDecoded = performance.now(); decodingMilli = imageDecoded - imageDecodeStart; const canvas = document.getElementById('canvas'); const canvasContext = canvas.getContext('2d'); canvasContext.drawImage(result.image, 0, 0); renderingMilli = performance.now() - imageDecoded; } ```

I don't know if it has anything to do with it, but I noticed it renders faster when using `ImageData` vs supplying a pixel array. I haven't had the time but I am still getting some images show up as oversaturated (only with HDR).

they have both `decodingMilli` and `renderingMilli` there but they are only reporting one thing on the main page

so I assume they are only showing `decodingMilli`

Advantage of libavif is that it can output in various pixel formats, including ABGR, ARGB, ... With libjxl we can get RGB and RGBA. If application need different order, up to 10% time could be lost by converting between buffers on application side.

yeah we need to fix that — though for Chrome I suppose this is not an issue and we can produce buffers in the format needed

https://sneyers.info/browserspeedtest/

looks like jxl is indeed somewhat slower

w.jxl: Fetch: 5.600000023841858| Decode: 189.5| Render: 14.5 w8.avif: Fetch: 6.5| Decode: 94.39999997615814| Render: 35.90000003576279 w10.avif: Fetch: 6.200000047683716| Decode: 109.39999997615814| Render: 18.100000023841858 w12.avif: Fetch: 6.5| Decode: 130.69999998807907| Render: 18.700000047683716 w.webp: Fetch: 6.5| Decode: 67| Render: 13.800000011920929

As can be expected, 8-bit avif decodes faster than 10/12-bit avif

render time is higher for 8-bit avif than for 10/12-bit avif though — so probably this is related to yuv2rgb conversion being done as part of decode in 10/12-bit and as part of render in 8-bit

still, avif is about 33% faster at worst, right?

can you try with --faster_decode?

(also, what chrome version is this?)

added fd1,2,3 variants

w-fd1.jxl: Fetch: 237.60000002384186| Decode: 166.69999998807907| Render: 12.399999976158142 w-fd2.jxl: Fetch: 4.800000011920929| Decode: 152.19999998807907| Render: 12 w-fd3.jxl: Fetch: 4.699999988079071| Decode: 138.30000001192093| Render: 11.900000035762787

mhhh interestingly this looks very different on my workstation (jxl is somewhat faster) but then I also have AVX512

Version 108.0.5359.94 (Official Build) (x86_64)

yeah I don't have avx512

can you try on 109?

or canary

numbers look pretty much the same for me on canary

different still on the phone (12bit avif looks like it is not SIMDfied)

109.0.5414.25 ```w.webp: Fetch: 4.799999997019768| Decode: 60.5| Render: 12.599999994039536 w.jpg: Fetch: 187.90000000596046| Decode: 32.099999994039536| Render: 9.700000002980232 w.jxl: Fetch: 511.5| Decode: 201.5| Render: 12.299999997019768 w-fd1.jxl: Fetch: 370.59999999403954| Decode: 243.79999999701977| Render: 7.100000008940697 w-fd2.jxl: Fetch: 363.90000000596046| Decode: 233.20000000298023| Render: 6.899999991059303 w-fd3.jxl: Fetch: 360.40000000596046| Decode: 196.69999998807907| Render: 6.100000008940697 w8.avif: Fetch: 369.59999999403954| Decode: 134.79999999701977| Render: 26.30000001192093 w10.avif: Fetch: 356.20000000298023| Decode: 162.29999999701977| Render: 6.300000011920929 w12.avif: Fetch: 378| Decode: 153.29999999701977| Render: 6.700000002980232```

yeah this is going to differ quite a bit depending on hardware

bottom line is avif and jxl are both quite a bit slower to decode than jpg, and webp is somewhere in between

110.0.5449.3 ```w.webp: Fetch: 465.29999999701977| Decode: 71.6000000089407| Render: 19.299999997019768 w.jpg: Fetch: 62| Decode: 20.30000001192093| Render: 13.299999997019768 w.jxl: Fetch: 355.90000000596046| Decode: 199.5| Render: 13.199999988079071 w-fd1.jxl: Fetch: 353.79999999701977| Decode: 214.20000000298023| Render: 9.099999994039536 w-fd2.jxl: Fetch: 452.29999999701977| Decode: 228.90000000596046| Render: 7 w-fd3.jxl: Fetch: 348.70000000298023| Decode: 181.29999999701977| Render: 7.799999997019768 w8.avif: Fetch: 357.3999999910593| Decode: 103.6000000089407| Render: 26.19999998807907 w10.avif: Fetch: 395.20000000298023| Decode: 112.5| Render: 7 w12.avif: Fetch: 465.19999998807907| Decode: 164.40000000596046| Render: 8.400000005960464```

doing multithreaded decode in chrome would take away much of the difference though, since jpeg and webp benefit way less from MT

you can stop doing the epf at high enough qualities, or do less iterations of epf

yeah but epf is a decode-only filter, no?

https://sneyers.info/browserspeedtest/small.html here is the same test but for a more typical "web-sized" image

fd3 disables epf

https://sneyers.info/browserspeedtest/small.html ```s.webp: Fetch: 243.40000000596046| Decode: 8.599999994039536| Render: 3.4000000059604645 s.jpg: Fetch: 185.20000000298023| Decode: 1.1000000089406967| Render: 0.5 s.jxl: Fetch: 243| Decode: 42.400000005960464| Render: 2.8999999910593033 s-fd1.jxl: Fetch: 140.90000000596046| Decode: 31.799999997019768| Render: 1 s-fd2.jxl: Fetch: 128.70000000298023| Decode: 23.799999997019768| Render: 1.4000000059604645 s-fd3.jxl: Fetch: 276.1000000089407| Decode: 36.3999999910593| Render: 1.2999999970197678 s8.avif: Fetch: 193.5| Decode: 15.699999988079071| Render: 2 s10.avif: Fetch: 195.29999999701977| Decode: 35.29999999701977| Render: 4.299999997019768 s12.avif: Fetch: 131.5| Decode: 18.299999997019768| Render: 1.2999999970197678``` yeah, the numbers are very fluctuating ```webp: Fetch: 243.5| Decode: 11.5| Render: 4.5 s.jpg: Fetch: 72.19999998807907| Decode: 1.1000000089406967| Render: 0.5 s.jxl: Fetch: 123.29999999701977| Decode: 15.299999997019768| Render: 1.0999999940395355 s-fd1.jxl: Fetch: 121.29999999701977| Decode: 14.899999991059303| Render: 0.9000000059604645 s-fd2.jxl: Fetch: 170.1000000089407| Decode: 16.299999997019768| Render: 1 s-fd3.jxl: Fetch: 123.29999999701977| Decode: 12.399999991059303| Render: 0.9000000059604645 s8.avif: Fetch: 184.90000000596046| Decode: 7.399999991059303| Render: 1.1000000089406967 s10.avif: Fetch: 137.20000000298023| Decode: 8.299999997019768| Render: 1 s12.avif: Fetch: 145.70000000298023| Decode: 9.5| Render: 0.9000000059604645```

s10.avif: Fetch: 195.29999999701977| Decode: 35.29999999701977| Render: 4.299999997019768 s12.avif: Fetch: 131.5| Decode: 18.299999997019768| Render: 1.2999999970197678 uh, what

maybe it should average 10 runs or something

yeah single run measurements are kind of noisy

anyway it does look like jxl is somewhat slower to decode on avif (except on avx512), though I wonder how much that really matters in practice

in the sense that network is likely to still be at least an order of magnitude slower in most cases, and for larger images / slower network it's not really the full-image-decode time that matters most imo, but rather the time to first preview...

on the phone it might matter

esp on lower end phones

we could consider making a libjxl-turbo that implements everything in int16_t simd...

pretty hard to do that

or we could make it use 2-4 threads per image?

but for a subset it can be done

say only for files encoded with fd 1+ or 2+

maybe (depends on DCT sizes)

it's still a pain though

I think the phones are not that slow, even very low-end starting from ARM64, for older phones most likely there are no more updates and new Chrome versions and many other apps for lossy, but lossless can be very slow

eh, I'll try it on my family's phones at some point in the next month

ugh wait a second, that javascript code first loads the whole bitstream, then decodes it

so it doesn't do any kind of streaming decode

decode in reality can start while data is still arriving

at least libjxl can certainly do that, I dunno if they can do that for avif already

this makes quite a big difference for any image that is more than one network packet large

You can pass streams into `ImageDecoder`. Will probably have to use `fetch()` though.

Using a `ServiceWorker` you can intercept the fetches from `img` tags.

https://sneyers.info/progressive/avif-jxl.html

simulating a 6x slower cpu on a "fast 3G" network, this is the kind of thing I get

what does 6x slowdown mean...

no idea but I suppose it somehow makes the cpu do nothing 5/6th of the time?

the point is: even if the jxl decode time is longer, it starts decoding during transfer and the 1:8 cat is on the screen well before the avif is on the screen

what is the current status of google chrome and jpeg-xl? are they adding it back in?

no

I believe that Chrome communicates about JPEG XL through blink-dev mailing list at this time

They receive feedback from the industry about JPEG XL through two channels

JPEG XL/Chromium bug: https://bugs.chromium.org/p/chromium/issues/detail?id=1178058

and avif-feedback@googlegroups.com email identified in Yaowu's last post

the latter relates only to improving the AVIF/JPEG XL comparison

You're awfully kind. But no one would argue in good faith that AVIF encoding is faster than JXL encoding.

And averaging irrelevant data into the results to skew the numbers to say the opposite of what they would normally say, smells.

You'd be surprised.

I prefer to assume this was a honest mistake that can be corrected by pointing out what went wrong. We need to attempt to be constructive and build bridges that allow premature conclusions to be rectified, rather than pointing fingers and accusing one another of deception. It is tempting to become cynical, but I believe we can make more progress by patiently discussing things based on a scientific method. We don't need character assassinations to win the argument, we can just point out the facts in a friendly and diplomatic way.

The people building AV1/AVIF, many of them come with a long history (like 25+ years) of compression, longer than mine or Jon's. Some of them were early employees of On2 Technologies, and represent the best expertise in compression. They are highly professional data-driven people. It is not only unkind and unprofessional to attack them personally, it is also just wrong.

Why we have different numbers on performance can be because of different context. Also possibly because of not wanting to invest a lot of effort in benchmarking has contributed to not understanding each others' context.

well said

I don't think people have the issue with the way the data was gathered. The issue is that they made it harder than it was needed to actually get repeatable results.

I think people take an issue with how data was gathered

AVIF is decently fast at encoding qualities that are unusably low

and it's not really realistic to include them in your aggregate

Don't be evil

i guess they haven't removed it yet as of 110.0.5449.3 Dev i haven't checked canary yet though

https://chromium-review.googlesource.com/c/chromium/src/+/2915684

This may be the reason why they did the decode speed comparison using M92. At M93 they started using two threads for avif decode, while jxl decode is still single-threaded...

Jxl decode became faster, but avif decode too, by using two threads instead of one, so in current chrome versions the relative timings are still similar — avif being faster than jxl — but if both would use the same number of threads, jxl would be faster, I think.

Anyway, this kind of invalidates the timings we have above: those are comparing 2-threaded dav1d to 1-threaded libjxl, which is not really a fair comparison.

We should probably make jxl decoding also use 2 threads, or even make it depend on the image dimensions (1 thread for small ones, 2 for medium, maybe 3 or 4 for large ones)

What kinds of speeds is everyone getting on this? https://sneyers.info/browserspeedtest/small.html

This is what I'm getting: s.jpg: Decode+render speed: 204.80 MP/s | Fetch: 2.70| Decode: 1.20| Render: 0.40 s.jxl: Decode+render speed: 21.70 MP/s | Fetch: 2.60| Decode: 14.60| Render: 0.50 s-fd1.jxl: Decode+render speed: 22.91 MP/s | Fetch: 3.20| Decode: 13.70| Render: 0.60 s-fd2.jxl: Decode+render speed: 24.82 MP/s | Fetch: 3.00| Decode: 12.60| Render: 0.60 s-fd3.jxl: Decode+render speed: 26.21 MP/s | Fetch: 2.90| Decode: 12.10| Render: 0.40 s8.avif: Decode+render speed: 40.96 MP/s | Fetch: 7.60| Decode: 7.40| Render: 0.60 s10.avif: Decode+render speed: 35.62 MP/s | Fetch: 2.90| Decode: 8.70| Render: 0.50 s12.avif: Decode+render speed: 30.91 MP/s | Fetch: 3.70| Decode: 10.10| Render: 0.50 s.webp: Decode+render speed: 71.23 MP/s | Fetch: 2.80| Decode: 4.10| Render: 0.50

so comparing jxl without faster-decoding to 8-bit avif, avif is almost twice as fast — but it is also using 2 threads while jxl uses 1

(I clicked the buttons a few times and took the fastest result for each file)

The drop in s8.avif decode time is extreme: 1. s8.avif: Decode+render speed: 12.51 MP/s | Fetch: 72.00| Decode: 22.30| Render: 3.90 2. s8.avif: Decode+render speed: 49.65 MP/s | Fetch: 5.80| Decode: 5.60| Render: 1.00 I suspect that at some point caching might kick in. Or does it also start decoding while fetching?

s.jpg: Decode+render speed: 364.09 MP/s | Fetch: 1.70| Decode: 0.70| Render: 0.20 s.jxl: Decode+render speed: 33.44 MP/s | Fetch: 1.90| Decode: 9.60| Render: 0.20 s-fd1.jxl: Decode+render speed: 33.78 MP/s | Fetch: 1.90| Decode: 9.50| Render: 0.20 s-fd2.jxl: Decode+render speed: 36.82 MP/s | Fetch: 1.90| Decode: 8.60| Render: 0.30 s-fd3.jxl: Decode+render speed: 40.96 MP/s | Fetch: 1.90| Decode: 7.70| Render: 0.30 s8.avif: Decode+render speed: 71.23 MP/s | Fetch: 2.00| Decode: 4.20| Render: 0.40 s10.avif: Decode+render speed: 64.25 MP/s | Fetch: 1.90| Decode: 4.90| Render: 0.20 s12.avif: Decode+render speed: 51.20 MP/s | Fetch: 2.00| Decode: 6.20| Render: 0.20 s.webp: Decode+render speed: 117.03 MP/s | Fetch: 1.70| Decode: 2.60| Render: 0.20

s.jpg: Decode+render speed: 252.06 MP/s | Fetch: 3.00| Decode: 0.90| Render: 0.40 s.jxl: Decode+render speed: 26.86 MP/s | Fetch: 4.80| Decode: 11.30| Render: 0.90 s-fd1.jxl: Decode+render speed: 27.08 MP/s | Fetch: 3.20| Decode: 11.10| Render: 1.00 s-fd2.jxl: Decode+render speed: 28.74 MP/s | Fetch: 6.20| Decode: 10.30| Render: 1.10 s-fd3.jxl: Decode+render speed: 31.81 MP/s | Fetch: 3.20| Decode: 9.40| Render: 0.90 s8.avif: Decode+render speed: 53.72 MP/s | Fetch: 5.90| Decode: 5.50| Render: 0.60 s10.avif: Decode+render speed: 44.28 MP/s | Fetch: 4.90| Decode: 6.60| Render: 0.80 s12.avif: Decode+render speed: 35.62 MP/s | Fetch: 5.00| Decode: 8.30| Render: 0.90 s.webp: Decode+render speed: 93.62 MP/s | Fetch: 4.40| Decode: 3.00| Render: 0.50

On a weaker laptop, I get: Jxl: 20.23 MP/s Avif 8-bit: 20.48 Avif 10-bit: 18.51

On my phone, I get: jxl: 4.93 MP/s avif 8-bit: 13.37 avif 10-bit: 10.34

Chrome 108, desktop (incognito): s.jpg: Decode+render speed: 30.06 MP/s | Fetch: 1048.90| Decode: 1.40| Render: 9.50 s.jxl: Decode+render speed: 19.86 MP/s | Fetch: 785.30| Decode: 14.90| Render: 1.60 s-fd1.jxl: Decode+render speed: 14.31 MP/s | Fetch: 780.20| Decode: 22.10| Render: 0.80 s-fd2.jxl: Decode+render speed: 16.22 MP/s | Fetch: 790.10| Decode: 19.50| Render: 0.70 s-fd3.jxl: Decode+render speed: 25.40 MP/s | Fetch: 783.90| Decode: 12.20| Render: 0.70 s8.avif: Decode+render speed: 34.13 MP/s | Fetch: 780.90| Decode: 8.60| Render: 1.00 s10.avif: Decode+render speed: 32.77 MP/s | Fetch: 794.20| Decode: 9.30| Render: 0.70 s12.avif: Decode+render speed: 22.91 MP/s | Fetch: 782.70| Decode: 13.20| Render: 1.10 s.webp: Decode+render speed: 86.23 MP/s | Fetch: 783.00| Decode: 3.40| Render: 0.40

things do fluctuate quite a bit between runs though so maybe my method of clicking a few times and then taking fastest might not be ideal

I think you need to run them in a random order and remove outliers and average remainders (so to say)

yeah that would be a better way, I can't really write javascript though and this requires some better script code

eyeballing the numbers, I would say avif is typically about twice as fast at 8-bit and more like 1.5 times as fast at >8-bit

factoring in the number of threads used, I'd say they are pretty much comparable in decode speed (maybe jxl a bit faster for HDR)

would that be a fair assessment?

On my crappy phone, the difference is quite big. (On a Mediatek Helio P25, a ~6 years old midrange SoC.) s8.avif: Decode+render speed: 5.84 MP/s | Fetch: 226.30| Decode: 45.70| Render: 10.40 s8.avif: Decode+render speed: 6.21 MP/s | Fetch: 11.30| Decode: 46.60| Render: 6.20 s12.avif: Decode+render speed: 3.08 MP/s | Fetch: 209.80| Decode: 97.80| Render: 8.60 s12.avif: Decode+render speed: 3.78 MP/s | Fetch: 15.10| Decode: 82.90| Render: 3.90 s.jxl: Decode+render speed: 1.23 MP/s | Fetch: 243.50| Decode: 262.40| Render: 4.60 s.jxl: Decode+render speed: 1.31 MP/s | Fetch: 17.00| Decode: 244.90| Render: 5.10 s-fd3.jxl: Decode+render speed: 1.87 MP/s | Fetch: 244.90| Decode: 172.60| Render: 2.60 s-fd3.jxl: Decode+render speed: 1.64 MP/s | Fetch: 14.20| Decode: 196.60| Render: 3.00

It feels too variable, but here is Pixel 6 Pro FWIW: ``` s8.avif: Decode+render speed: 11.00 MP/s | Fetch: 15.30| Decode: 26.60| Render: 3.20 s8.avif: Decode+render speed: 11.34 MP/s | Fetch: 24.20| Decode: 24.80| Render: 4.10 s12.avif: Decode+render speed: 6.21 MP/s | Fetch: 21.10| Decode: 49.30| Render: 3.50 s12.avif: Decode+render speed: 9.39 MP/s | Fetch: 10.80| Decode: 33.30| Render: 1.60 s.jxl: Decode+render speed: 3.16 MP/s | Fetch: 14.30| Decode: 101.50| Render: 2.20 s.jxl: Decode+render speed: 7.41 MP/s | Fetch: 18.80| Decode: 40.10| Render: 4.10 s-fd3.jxl: Decode+render speed: 10.85 MP/s | Fetch: 15.10| Decode: 27.10| Render: 3.10 s-fd3.jxl: Decode+render speed: 9.36 MP/s | Fetch: 16.60| Decode: 30.30| Render: 4.70 ```

```plaintext s.jpg: Decode+render speed: 36.41 MP/s | Fetch: 2.20| Decode: 1.10| Render: 7.90 s.jxl: Decode+render speed: 28.01 MP/s | Fetch: 2.00| Decode: 10.30| Render: 1.40 s-fd1.jxl: Decode+render speed: 34.13 MP/s | Fetch: 1.90| Decode: 9.10| Render: 0.50 s-fd2.jxl: Decode+render speed: 36.01 MP/s | Fetch: 2.20| Decode: 8.70| Render: 0.40 s-fd3.jxl: Decode+render speed: 39.01 MP/s | Fetch: 2.30| Decode: 8.00| Render: 0.40 s8.avif: Decode+render speed: 55.54 MP/s | Fetch: 2.30| Decode: 5.30| Render: 0.60 s10.avif: Decode+render speed: 48.19 MP/s | Fetch: 2.40| Decode: 5.90| Render: 0.90 s12.avif: Decode+render speed: 49.65 MP/s | Fetch: 218.90| Decode: 6.10| Render: 0.50 s.webp: Decode+render speed: 91.02 MP/s | Fetch: 452.30| Decode: 3.10| Render: 0.50 ```

On old phones, I expect the difference to be larger since libjxl uses floating point arithmetic and old ARM is particularly bad at that

but I think on modern hardware it looks like the gap is basically just 1-threads vs 2-threads and likely if both would use 2 threads, jxl would be at least as fast if not faster (since it would almost double the speed compared to the numbers we see now)

we _could_ theoretically make a libjxl that uses only 16-bit integer arithmetic (at least when only 8 or 10-bit precision is needed, like on the web), and it could likely be close to libjpeg-turbo speeds, but the question is if this is worth the implementation effort — it would probably only make any observable difference on old phones

what about a comparison in firefox? because it supports multithreading for both formats

None of the images are loading on my phone with Firefox mobile, it'll just create the text for the respective format with no timings

Just so happens that every time I try one of these I'm not home to use my computer

The api used to decode the images isn't supported by Firefox. https://developer.mozilla.org/en-US/docs/Web/API/ImageDecoder

You'd need to write an alternative benchmark using either the <img> element, or <https://developer.mozilla.org/en-US/docs/Web/API/createImageBitmap>. But I'm not sure if you could get the same the same kind of results. Particularly, a seperate number for the rendering step might prove difficult. For that matter, `createImageBitmap` looks to be the closest replacement for the currently used api as it can load `blob`s (so, binary data). But I'm not sure what else that function does besides just decoding.

OK. A general comment, but I hope not totally off-topic… I must say I’m excited to see the activity, ideas and urge to improve JXL. And some of it seems new (to me!, I only read msgs here) and possibly influenced by Chrome team’s decision+publication of their results. Useful knowing speed might be another factor along the known problem at the low quality end (which I still, personally, think warrants most attention). Almost as if competition is a good thing. All boats will be risen etc. 🤣 Another reason for Chrome to reverse (or “reevaluate”) their decision. If not now, then “one day soon”.

Absolutely. Competition can be very useful — it is what gave us mozjpeg, for example, and I'm sure also a lot of the recent improvements in both jxl and avif are inspired by it.

single thread emulation (not sure how accurate it is) ```s.jpg: Decode+render speed: 192.75 MP/s | Fetch: 1.80| Decode: 1.10| Render: 0.60 s.jxl: Decode+render speed: 19.98 MP/s | Fetch: 1.70| Decode: 15.40| Render: 1.00 s-fd1.jxl: Decode+render speed: 21.01 MP/s | Fetch: 2.00| Decode: 14.60| Render: 1.00 s-fd2.jxl: Decode+render speed: 18.94 MP/s | Fetch: 1.80| Decode: 16.40| Render: 0.90 s-fd3.jxl: Decode+render speed: 23.92 MP/s | Fetch: 1.90| Decode: 12.60| Render: 1.10 s8.avif: Decode+render speed: 32.13 MP/s | Fetch: 2.20| Decode: 9.40| Render: 0.80 s10.avif: Decode+render speed: 34.13 MP/s | Fetch: 1.90| Decode: 8.20| Render: 1.40 s12.avif: Decode+render speed: 27.54 MP/s | Fetch: 2.80| Decode: 10.90| Render: 1.00 s.webp: Decode+render speed: 76.20 MP/s | Fetch: 1.80| Decode: 3.80| Render: 0.50```

Here is using the wasm polyfill I am working on (4 thread): ```plaintext s.jxl: Decode+render speed: 10.34 MP/s | Decode: 11.16 | Render: 7.01 s-fd1.jxl: Decode+render speed: 11 MP/s | Decode: 10.17 | Render: 6.63 s-fd2.jxl: Decode+render speed: 11.19 MP/s | Decode: 10.01 | Render: 6.42 s-fd3.jxl: Decode+render speed: 11.67 MP/s | Decode: 9.25 | Render: 6.52 ```

are you somehow forcing chrome to use only 1 thread, or what did you do?

Welp, looks like the speed test is not working in Firefox and Pale Moon. We haven't implemented ImageDecoder yet...

yeah, thread affinity

I changed the scripts on the test page to use `createImageBitmap` for the decoding instead. Here's the numbers I get in Firefox Nightly. Of course, there's quite some fluctuation. But overall, jxl seems to be as fast as 8-bit avif and a bit faster than 12-bit avif. s.jxl: Decode+render speed: 36.41 MP/s | Fetch: 4.00| Decode: 8.00| Render: 1.00 s8.avif: Decode+render speed: 32.77 MP/s | Fetch: 4.00| Decode: 9.00| Render: 1.00 s12.avif: Decode+render speed: 27.31 MP/s | Fetch: 6.00| Decode: 12.00| Render: 0.00

Nice, this seems to confirm the hypothesis that speeds are comparable and it's just the thread count that makes the difference.

And here's Firefox Nightly on my crappy phone: s.jxl: Decode+render speed: 6.97 MP/s | Fetch: 83.00| Decode: 42.00| Render: 5.00 s8.avif: Decode+render speed: 7.80 MP/s | Fetch: 61.00| Decode: 40.00| Render: 2.00 s12.avif: Decode+render speed: 4.82 MP/s | Fetch: 56.00| Decode: 64.00| Render: 4.00 For comparison, the chrome numbers on the same page are much worse, for both avif and jxl. I think chromium had some issues with that api, which might explain why it wasn't used for the comparison. s.jxl: Decode+render speed: 1.01 MP/s | Fetch: 11.50| Decode: 323.90| Render: 0.40 s8.avif: Decode+render speed: 3.35 MP/s | Fetch: 20.10| Decode: 97.40| Render: 0.30 s12.avif: Decode+render speed: 2.03 MP/s | Fetch: 40.50| Decode: 161.10| Render: 0.30

In case anyone might be interested, here's the changed test page using `createImageBitmap` instead.

I will try making 1, 2, 3, and 4 thread wasm packages and pick based on image size to see what happens.

Your 8k art image takes 4 seconds to decode & render.

yeah 8k is kind of large for a web image 🙂

Also I think lossless is a lot slower than lossy.

oh yes

at least funky lossless like jxl art, which cannot use any fast path

on my Firefox Nightly, I get these numbers: s.jxl: Decode+render speed: 32.77 MP/s | Fetch: 2.00| Decode: 9.00| Render: 1.00 s8.avif: Decode+render speed: 29.79 MP/s | Fetch: 2.00| Decode: 11.00| Render: 0.00 s10.avif: Decode+render speed: 23.41 MP/s | Fetch: 2.00| Decode: 14.00| Render: 0.00 s12.avif: Decode+render speed: 21.85 MP/s | Fetch: 2.00| Decode: 14.00| Render: 1.00

also 8k jxl art requires the nonseparable upsampler typically which is very slow

yes, that one uses the 8x upsampler

is it that slow? I thought it was simdified and threaded quite well?

it might be but the algorithm itself is slow

Is it much worse than bicubic upsampling?

Speed is not even a valid angle to approach this argument at all. dav1d is already optimized to the max and reached its potential. We don't expect any further significant breakthroughs in AV1 decoding speed.

I doubt that the same can be said about the libjxl decoder.

I think that the jxl format has been barely explored and exploited to take advantage of it's performance potential both in speed and in psyvis optimization

Well, the fjxl encoder is pretty cool, if you want ludicrously fast encoding, thanks Luca!

Now we need fjxld :D

Personally I still believe it's impossible for a knowledgeable person to sincerely argue in good faith that AVIF encoding is faster than JXL.

Well, you'll see a knowledgeable and experience person on the subject proving that AV1 encoding can be both absurdly slow and absurdly fast. Next week (I hope...) that is, not today.

But a knowledgeable person at Google seems to be suggesting exactly that and most people can immediately recognize how absurd that looks like. And it would be nice to believe it was somehow an "accident" that most of their tests, that they took the average of, were done using useless settings that produce unusable output. But that would require me to visit fairy magic land and believe in Peter Pan.

The span between s0 and s9 in libaom is very large, both in speed and in compression results

My understanding is that AVIF currently can be very fast, or it can look good. One or the other.

It's a lot more nuanced than that 🙂

Maybe it can be very fast, it can look good, and it can have a small filesize. Choose two.

You can make all 3 work if you know what to do. Hence, very nuanced stuff 🙂

Well hopefully that will also benefit video encoding too, when that happens.

I wonder if XYB can benefit AV1 compression at all

Or if anyone's tried using XYB for a video codec

It seems kind of archaic now, by comparison, for a format to have a 10-bit and 12-bit mode when JXL only has one mode.

Well, not with current encoders. Everything is well tuned for YCbCr, so if you just insert XYB through X mean(ICC profile or future color metadata), you'll get barely any performance improvements, or none at all.

It's not archaic when you need to have low cost power efficient dedicated HW support though.

16-bit int fixed point lossy JXL will become a thing eventually if you want wide support for it on all kind of devices.

Is JXL floating point?

libjxl uses float32. The format itself is defined with real numbers, and how you approximate those is up to you (e.g. for hardware implementations they will use fixedpoint)

For real hardware, the number of precision bits can be chosen arbitrarily. For FPGA, 18-bit is convenient. For CPU, 16-bit is convenient.

Well the real numbers have to be encoded somehow when they are put into a file.

Video codecs need to be defined exactly in fixedpoint, since errors can otherwise accumulate too much over frames. So you need to have different modes for each bit depth.

little endian, big endian, signed or unsigned, float or int, etc

They are always encoded as integers, since all entropy coding is encoding ints. But those ints are quantized coeffs that represent reals which are in libjxl represented using float32.

Oh

That's a good explanation actually, thanks :)

So they can be decoded into whatever output format you like

So why would we ever need "16 bit int fixed point JXL" if the format doesn't care how you represent the real numbers?

Only in some optional header fields numbers are represented directly (as half-float or uint, typically). The image data itself are integers, entropy coded using Huffman or ANS.

Because of speed and memory usage (bandwidth and size).

Afte entropy decode, the decoder needs to do all the inverse transforms and filters and blending etc, and it can do that with various kinds of precision.

But it sounds like the format would not have to change at all since it's completely inconsequential how you represent the decoded image data.

Libjxl currently does everything with float32

Are you talking about a hypothetical decoder implementation?

For SDR images, maybe also 10/12-bit, you could likely make a jxl decoder that uses (mostly) int16 instead of float32

At first I thought you meant a new defined subset of JXL, the format

Yeah, I hope there are a whole bunch of decoder libraries in the future optimized for embedded use and/or speed.

The bigger dct blocks (above dct32) probably need some more precision. We might at some point define a 'light' profile where some things are not allowed, to make it decodable in limited precision.

I'm talking about specifically dedicated HW support. It would be expensive and rather power hungry to design an ASIC that currently supports every advanced software JXL feature. Lossy float operations are the ones that need to be cut, because lossy int operations are faster, more power efficient, and most importantly, don't trash CPU cores that don't have strong FPUs 😄

Well first someone ought to get started making an embedded-oriented library.

In a 'light' profile (or 'hardware friendly'), we would likely only allow the coding tools used in libjxl-tiny.