JPEG XL

What I thought was that it was first developed to replace WebP, and that it was abandoned for JXL, which suited best the use cases. I made some tests and I found that WP2 has a compression method quite similar to JXL, except it’s a bit heavier (sometimes even more than WebP). And the support for WP2 is, as I’ve seen almost non-existent (not by GIMP, IrfanView, not even one browser have the option, and it’s not even mentioned in Can I Use), as it’s a nearly discontinued recent format. > WebP 2 is the successor of the WebP image format, currently in development. It is not ready for general use, and the format is not finalized so changes to the library can break compatibility with images encoded with previous versions.

—https://chromium.googlesource.com/codecs/libwebp2/

They say it’s still in active development but I can’t think it will ever be a widely used format with jxl.

https://twitter.com/pandoras_foxo/status/1433856228011110404

I think this is why ecosystem support outside the browser is so important imo, and why image codecs specifically made "for the web" are a bad idea, even if it is easier for companies that control both ends of the web to push adoption on the web.

Images have a life cycle that is not (always) just from server serving it to client viewing it.

people are allowed to download images nowadays? **schocked**

For video, it's kind of inevitable that different things are used for production than for end-user delivery. I think it's video people who have this concept of an image format that is "good enough for the web", as if the flow is always that an image is 'produced' once and then served a million times (basically the youtube/netflix/broadcasting model of distribution)

For images we can now afford to use the same format for both authoring and delivery, and the line between those two is way more blurry than for video

are you saying that people are able make their own images, and can encode them and even distribute them without paying someone to do that by watching ads?

where is economy in that?

i mean that probably isn't the case but sometimes i wonder what goes in heads of ones who are pushing for avif so hard

Lol when I was trying to convince professional photographers that generation loss is a problem, some of them considered it a feature, not a bug, since it acts like a rudimentary DRM that prevents unlimited unauthorized sharing of their photos

gotta be careful about advertising liberty-enabling features

I think for now the lossless Jpeg XL encoder still needs improvement in speed/efficiency for game content and I think that lossless WebP is still more universal and preferable for that purpose (unless more than 8 bits per channel are required), although it might be better to start implementing earlier, but it might not have any noticeable advantages when comparing the current Jpeg XL implementation

However: <https://github.com/godotengine/godot/pull/47835>

https://user-images.githubusercontent.com/32321/120538887-dcff5400-c39b-11eb-8b68-8a65726413e9.png

I think we need to come up with encoder heuristics for e4-6 that do something more than just using a fixed MA tree and RCT, and something less than the full MA tree learning algorithm (which is what it does now)

That and making the tree learning algorithm more parallel/SIMD'd (for e7-9) should help a lot to get better speed/density trade-offs

webp mode

😛

(I suspect we can make a "webp mode" that works up to e8 by default, and at e9 try both that and what we do today)

Yes, but also e.g. atm modular basically does global MA trees all the time (where group id is just a property it can use). Wouldn't it make sense to use local trees/histograms and possibly lose a bit in density but gain a lot in parallelizability (and possibly getting better / more locally adapted trees?)

why not parallelize the tree learning?

can be done...

Btw, WebP2 as far as I understand should have almost the same decoding speed (after the final optimization), but at the expense of slow encoding speed, because basically there remains the same model as in WebP1, but some things are changed to more advanced and may be worth to do also something like this? And for games most important is decoding speed (mostly in single-threaded mode, because there are a lot of images and they are loaded in parallel) and moderate memory consumption

In the case of (parametrized) fixed trees and no weighted predictor, I think we could make decode speed very fast

Like -e 2 now, but without the unnecessary conversions from int to float and back to int that we're doing now

unlord, unlord, this was certainly unexpected from you: https://old.reddit.com/r/AV1/comments/phhhhk/any_way_to_improve_the_quality_of_dark_areas_of/hbkg0ta/

Quite expected, since this is not the first such statement

I know, but that's not what unlord and me have been discussing before in a more private setting <:Thonk:805904896879493180>

web images are one thing, but i am quite happy with how jxl performs as an offline store (lossless only)

i use for only lossy

and i do get the expected average -35% over png and jpg

no i want 70% lower at least

with rav1e i get 30% lower than mp4 on video

for screenshot i want 80 kb from 420-1000 kb

500-1000 kb

for some images i want a similar file to the original even when specifing d 1

Nathan seems so bitter about jxl

Does anyone know what he's doing nowadays (after mozilla did that big sacking thing)?

tbf, if you use JPEG recompression on mozjpeg files... it is much closer to 7% than to 20%

waiting for -d 0.339 -I 0.574 -s 2 --patches=1 --epf=2 --use_new_heuristics 08092021

to ruin some images

with it

and destroy completely picture quality

Maybe we should run some benchmarks to see how it performs on mozjpeg, libjpeg-turbo, 420/444, various qualities.

hope the bpp choice will be good

That's what is weird 🤔

for photos

especially some photos same size

some photo way less

like 80 less

i want miracles

He seems to have a different public facing opinion about JXL and AVIF than what I've seen in "private"

another thing is that what "internet quality" means is not quite clear - is it mozjpeg -q 85? -q 65? -q 95?

mozjpeg -q 10 <:kekw:808717074305122316>

😛

we're trying to figure that out, but "nontrivial" doesn't even begin to describe that

I know some big websites use 65-75

but other places use higher

depending how you sample and weight, you can get medians around 80-85

i want that settings

wikipedia uses 88 I believe?

https://discord.com/channels/794206087879852103/805176455658733570/883752627744169995

don't care for more -d

Wordpress uses 82 by default iirc

Jon can probably tell us what *they* use

😛

We don't use fixed q most of the time

rav1e at speed 10 with 01092021 build from releases do all

with notenoughav1encodes

the quality is blurred

yes I mean what's the distribution

but the bitrate is automatic

it doesn't delete any detail

I don't have super great statistics on it, but the bulk of what we do is in the q65-q90 range

mozjpeg have different quant table preset and trellis quantization, I think libjpeg q and mozjpeg q have different quality define.

Yes, that's correct

It's not hugely different, but we do a mapping to convert libjpeg q to mozjpeg (or webp for that matter) q

(can you share that, and how you obtained it?)

also -- any such knowledge about photoshop? 😛

(I mean by email, just to clarify :P)

Dunno if I can share that, it's not public info

We basically just aligned using ssimulacra on some corpus, iirc

And made a lookup table

Photoshop: we don't encode with photoshop, only libjpeg-turbo and mozjpeg :)

Judging by some people, yes, although not quality, but some people want to encode in something like mozjpeg -q 10-30 sizes when encoding in AVIF

fair enough

well, some images in AVIF at sizes of mozjpeg q30 are not *too* bad

Mozjpeg q30 size is something very different from mozjpeg q30 fidelity

jpeg ImageMagick, mozjpeg, Guetzli quality test, use PSNR、SSIM、butteraugli compare, this comapre say for q90 up quality, no reason choose mozjpeg. > japanese site > JPEG画像をImageMagick,mozjpeg,Guetzliでquality > を変えつつ生成した場合のファイルサイズ、入力画像に対するPSNR、SSIM、butteraugliを比較する > https://qiita.com/kazinoue/items/38b34f9d798400c0d129

https://qiita-user-contents.imgix.net/https%3A%2F%2Fqiita-image-store.s3.amazonaws.com%2F0%2F149350%2Fa9c2c1e9-8fbe-eaca-3335-98b8642e5475.png?ixlib=rb-4.0.0&auto=format&gif-q=60&q=75&w=1400&fit=max&s=213cc0933504ceb3a536438902b4055f

I get near result on mozjpeg from my drawing test.

We usually use mozjpeg when 420 is ok and for the lower qualities (q60-85), and just libjpeg-turbo (but of course with huffman optimization and using progressive) for the higher qualities (q85-95)

Guetzli is too slow for us

yes, agree, Guetzli is too slow...

At the higher qualities we didn't really see a big enough benefit for mozjpeg to justify its speed

as in, "it isn't better", or as in, "it isn't better enough"?

Iirc the difference became smaller or nonexistent as q goes up

But the speed difference remained the same

(but that's not default mozjpeg vs default libjpeg-turbo, it's our specific mozjpeg settings vs our specific libjpeg-turbo settings, so ymmv)

I think things like trellis quantization and overshoot-clamping-based deringing and whatever else mozjpeg has that libjpeg-turbo doesn't, just becomes less effective / important as you quantize less aggressively

Sometime mozjpeg need higher q to reach maxButteraugli 1.0 quality, (libjpeg q95 444 and mozjpeg q98 444)

Yes, mozjpeg uses different quant tables and does more sophisticated quantization

mozjpeg default use PSNR-HVS, I think for high quality quantization choose libjpeg is a better option. (sjpeg use PSNR optimize, also can't get better maxButteraugli on high quality quantization.)

Before we switched to the lookup table, we used the following to map "libjpeg q" to a mozjpeg q:

`mozjpeg_q = libjpeg_q + 4 + (70-libjpeg_q)/8`

That mapping probably only makes sense above q50 since we don't care much about lower than that

aka 12.75 + 0.875 * libjpeg_q ?

I guess :)

rav1e doesn't have good ratio

compared to libaom-av1

fwiw

it's generally not better, it's just, written in rust

so rustaceans will use it for no other reason

Rav1e is also a real community encoder, while others are mostly developed by companies, and it usually has better quality at high bitrates than aom, but for now it is slower and has less features

Isn't rav1e only not "developed by a company" because Mozilla fired 25% of their staff including all their people who were working on rav1e?

Also, I think it was originally intended to be faster (but more limited in the coding tools it uses) than libaom. Now it seems to be aiming to be slower-but-better. It's a bit unclear to me what the overall goal of the rav1e project is, tbh.

As far as I know now Xiph exists and is funded separately, also mostly now Rav1e developers under VideoLAN and Vimeo funding

Yes, xiph is separate, I guess it's good that they put it under that umbrella and not the mozilla one

this is actually not true

libaom-av1 outperforms rav1e at higher bitrates

the biggest weakness of libaom at this point for an end user is that it multithreads fairly poorly

although av1an can fix this

just encode each GOP in its own single-threaded process

as for "developed by a company" it depends on what you mean

funded by a company and all the decisions made by a company providing funding are not the same thing

Depends on the content, but with the latest builds and in visually lossless quality, I like the result of Rav1e mostly more, the same applies to AVIF, although before I did not consider Rav1e as a real competitor

define "visually lossless quality"

you can't like the result of rav1e more if it's visually lossless, you should like it exactly the same

and just look at the filesize/speed

if you're comparing the actual files visually then it is not visually lossless

We don't care much about multithreading in production, tbh. For video, we parallelize in the way you describe (split in segments, encode them separately), which is needed anyway if you want to do streaming. For still images, we do everything in single-threaded processes since that's the most effective if you need to encode thousands of images per second anyway.

yea, that's what I do

I figure cjxl multithreads by default because the average joe wants that

but anyone batch processing will just disable threading

For an end-user who just needs to encode/decode a single image, multithreading is nice to have

For those who want to use it in more demanding use cases, they can figure out how to add `--num_threads=0` 😄

exactly

I take size into account (like quality at the same size), but I don't take speed, Rav1e is still slower than Aom Also for discussions about video codecs and AV1 there is a quite active discord community <https://discord.gg/HSBxne3>

also it really depends on what you define visually lossless to be

I've seen people claim x264 on preset veryslow is visually lossless at CRF=18

which is just isn't

It depends a lot on testing methodology

AIC-2 still image flicker test (what they used to test JPEG XS, which targets video production workflows) is very strict, corresponds to cjxl -d 0.3 or so

I mean what testing methodology is there beyond "a viewer can fairly easily tell the difference on most content"

If your metric marks something as "visually lossless" when a viewer can fairly easily tell the difference on most content, then your metric is wrong.

BT.500 side-by-side eval is much more forgiving, you can get within 'indistinguishable from the original' probably at -d 2 or so in most cases

Yes, but I just clarified what I mean by high bitrate, this is where the difference is quite difficult to see when carefully comparing even frames with each other (although for video to compare frames is not always the best solution)

Video eval methodologies are even more forgiving, since you don't even compare frames, you first watch one clip and then watch another.

the quality settings of rav1e btw don't necessarily map to those of libaom

Libaom current version also doesn't map to libaom older version

and since constant quality will give you better ratios than ABR, to truly compare them you either have to do ABR anyway and compare visual quality at a specific bitrate, or you have to tweak the quality settings until you figure out what's roughly comparable on most content

this second one is very hard cause there's a lot of variance

On a random test image, I got a 2 MB avif, then I updated my libaom/libavif to most recent version, and with the exact same command line it now gave me a 800 KB avif (which looked much worse too, of course)

yea, and some viewers notice certain things more than others

`cjxl -d 1 lenna.png lenna.jxl` to me is noticeably different but because I tend to look at grain

it's something that catches my eye

I'm guessing most viewers don't notice it as much

like if you showed me lenna.jxl without context would I know? probably not. but side-by-side it's pretty apparent to me

Also -e 8 -d 1 might be a bit different visually from lower effort -d 1

well I tried -e 9 and got a higher filesize

so

I'm not entirely sure why that happened

At e8+ we do iterations of Butteraugli

So likely the e7 one is not actually reaching d1

so you're saying at e9 the "d=1" metric is different-ish

or rather, it's more correct

At e8+ you get more guarantees that it actually reaches the target

but at d7 and lower, it's only approximately matching d=1?

that makes sense, trading accuracy for speed

At faster settings it is more heuristics-based so it might be a bit over or under target

I suppose in this case d=1 is really d<=1

Well BA is a maxnorm metric so

It's the worst spot that counts, not the average spot

I mean, if you run cjxl d=1 and you end up getting an image with d=0.5 from the original I'm guessing the encoder goes 'ok'

or are your heuristics good enough that this is uncommon

Well at d8+ the iterations are not per image, they are per block

Since we do adaptive quantization

So it will adjust quantization (in both directions) until it's close enough to the target

At e7 and faster, it selects adaptive quantization heuristically without actually verifying the result

It works well enough in practice, but in general: the lower the encode effort, the more perceptual variation in the result

To the point that at -e 3 it's basically just a quantizer setting you're specifying, not a perceptual target

huh

(so it becomes more like any other encoder when you dial down the effort, setting quantization, not quality)

that sounds very different from the way x264 handles CRF

Yes, I don't mean quality in the form of -q/-crf/-cq option, I mean visual quality at the best settings of both encoders at the same image or video size and to put it simply, Aom has a very noticeable detail loss and blurring even at high bitrates (although this can be somewhat improved with some settings), while Rav1e at high bitrates works better, but the lower the bitrate, Aom is mainly more visually appealing and at medium and low bitrates it almost always looks more pleasant than Rav1e

best settings for quality is lossless

you have to make some compromise

I don't mean the bitrate or lossless settings, Aom by default has far from optimal settings and some things can be improved to the same bitrate

you mean slowest/highest effort, ah

Well some people like <@416586441058025472> might disagree on that - some people like the compression artifacts and it does happen in subjective evals that distorted images get higher scores than the original

liking artifacts is not higher quality

quality in the visual sense is based on distance from the original

a compression algorithm that sharpens the output after decoding might appear "better" cause people like sharp things

i like simplyfing of original image

but it's not really better

fab is also not a good metric

even if it retains some of the noise or compression artifacts

I think that's not the job of an image codec, but some people don't mind it if an encoder "denoises" an image

but an hdr image and shooten better is best especially for a phone

but that's just placing a higher subjective emphasis on the damage caused by certain types of artifacts

even if compresses more

I think an image codec should be neutral and transparent, image in = image out

by design, yea

a simplified image to the original or a image that look same as original to the bytes

look stupid

is not encoder job

at least is encoder job but there is a limit

is not to the extreme

some people don't care if it encoder denoises but that's ultimately just a different subjective weight to certain artifact types

but i like even at d 0.339 new heuristics

i like to see the image simplified

Other tools can do denoising and sharpening and whatnot. If I want to have a noisy image, I want to be able to keep it that way.

exactly

that's why I say quality is nto "how nice it looks" but distance to the original

Not only the speed, but something like `--tune` from x26x (but unlike x26x, Aomenc does not even have optimal default settings, not just for certain content) Perhaps because the default for Aom developers considered something like YouTube quality

well x264 doesn't either. it has sane defaults but not optimal defaults for all content

x265 is a piece of work and I won't use it

it's not really like x264, it's just got the same CLI user interface and the same "name"

but it's a different thing that works much more poorly

choose the default for youtube makes no sense cause they're going to set all those settings manually anyway

The way x264 handles its constant quality setting is basically by making a (psychovisual) decision on how much the extra bits are necessary at a specific quantization parameter setting. this is based on the qcomp value curve, which defaults to 0.6 (scale between 0 and 1) at qcomp=1, it considers them *always* necessary and it works like CQP (constant quanization) at qcomp=0 it considers them *never* necessary and it works like CBR (or very close) the qcomp determines how aggressive it is at throwing away data that it thinks is probably unnecessary for that target quality

I know, and I agree. It's just a complication when doing subjective evaluation, and it's one of the reasons why "visually lossless" is sometimes (incorrectly imo) defined to occur at very low bitrates. On BT.500 MOS scales (1=worst, 5=best) we often see original images get 3.5 or 4 when compared side-by-side against themselves, and then if you look at confidence intervals around MOS scores of distorted images, you can claim images to be 'visually lossless' at 0.5 bpp, while in a fliptest anyone would see clear differences.

yea, my personal take is that objective metrics are a tool to make sense of what is ultimately human subjectiveness

since the true goal is human visual quality

and if the metric is inconsistent with human interpretation, then IMO the metric is wrong

so simpler images isn't the ultimate goal

no

the goal is entropy compression

also no

I think Butteraugli is a good metric to estimate whether some artifact can be seen or not. It's not a very good metric to estimate which artifacted image will look better, if both of them are clearly artifacted

well that's a much harder problem

cause some people care more about different artifact types

Yes, it's more about image semantics than about the visual system itself

Yes, but the default settings in x264 are still quite good if there is no way to adjust them for known content and encode blindly, but there are some things for Aom that when enabled, will give better quality at about the same speed and size

I call that "sane defaults"

People don't care if the grass is artifacted, but they do care if the face is artifacted.

Also they tend to assume background blur is intentional/artistic, while foreground blur is more annoying. Everyone will hate blockiness, but for smoothing or ringing there is more variation in what people think about it. Some are fine with some ringing because it can improve apparent sharpness, others hate it. Same with smoothing: some like how it denoises stuff, others hate the plastic look

And for example for AVIF, there will be even more such settings and they almost always give better results than the defaults, something like that: `-a color:denoise-noise-level=5 -a color:enable-dnl-denoising=0 -a color:sharpness=2 -a color:enable-chroma-deltaq=1 -a color:deltaq-mode=3 -a color:enable-qm=1` (although this may vary with changes in Aom from build to build) I mean Aom still has not optimal defaults, but x264 mostly doesn't need something like that, which gives better results on any content (not considering speed) and defaults are pretty ok

It would be really convenient if they would make whatever they think is the best general-purpose setting the default

Command line tools that require <@416586441058025472>-level parameter tweaking are just user hostile imo

Also JXL for lossy, maybe besides enabling noise in some cases, it has no settings that could noticeably improve the quality (so the default settings are ~optimal, at least for those that are available for the user)

That's the point. If there is something that is a good idea to do by default, then I consider it a bug if we don't do it by default.

Also, this discussion reminded me that it would be a good idea to add rav1e to this comparison: <https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2021_08_10/index.html> (it was useless before because old rav1e builds were always worse than aom, but now it makes some sense)

That comparison is done by the webp2 folks, I dunno if they will keep doing it

Doesn't hurt to ask for it

It might be good if someone would fork that comparison project and add a bunch more things to it, e.g. besides rav1e I am also interested in multiple encoder speed settings

I think it's all default speeds in the webp2 comparisons, which makes sense if you test only one thing, but it's easier to do apples-to-apples comparisons if multiple speeds are tried

together with the usual `-min 0 -max 63 --end-usage=q --tune=ssim`?

`--min 0 --max 63 -a end-usage=q -a cq-level=` and I think `--tune=ssim` is not always better, so I don't change `--tune` unless there is a way to use `--tune=butteraugli`, but butteraugli doesn't work with 10 bits

I think it updates when Jyrki asks for it, also in the last comparison the methodology for adjusting the size of images has changed

Yes, that was a good thing, the way they did it before gave rather variable perceptual results for the same size setting. Would probably be even better if size was just made to match jxl size at various d settings, but at least it's better now than how it was.

yes, it’s all too easy to lose track of that fact

tip #3 of https://thecooperreview.com/10-tricks-appear-smart-meetings/ is relevant here

Non-ironically though

I saw someone on the dpreview forum with a nice signature, along the lines of: “Beware correct answers to the wrong question.”

Most of humanity's problems are caused by accepting neat solutions that are solving the wrong problem. Whole fields of study are based on doing that, like most of economics, all of theology, and big chunks of medicine or law.

E.g. the invisible hand of the market certainly is an elegant solution, but unfortunately not for the right problem.

Also about metrics https://videoprocessing.ai/metrics/ways-of-cheating-on-popular-objective-metrics.html

https://videoprocessing.ai/assets/img/metrics/psnr_and_ssim/pic2.png

modern economists don’t believe that the free market solves everything, quite the opposite

https://noahpinion.substack.com/p/is-economics-an-excuse-for-inaction

something that is arguably a correct answer to a generally irrelevant question is confidence intervals

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4742505/

and then it is interpreted _as if_ it were the answer to another question

I love the reading material I get here, thanks for the interesting pointers, <@604964375924834314> and <@111445179587624960> !

That was an interesting read. And yes, academic economists are (fortunately!) no longer the high priests justifying laissez-faire for the neoliberals of the 80s and 90s, though as the article says, the "economists" you typically see on TV or other popular media are often still stuck in the 80s.

It's in particular good that they're turning more empirical now.

In my opinion, after Marx there was basically a century of non-empirical economics where they studied mathematical toy problems instead of reality because studying reality did not lead to the desired political conclusions.

Those mathematical toy problems (though fun and interesting) are what I meant with "neat solutions that are solving the wrong problem".

Anyway, sorry for derailing this channel

https://www.reddit.com/r/AV1/comments/picypn/the_av1_encoding_situation_sucks/

unlord: "If you are complaining the open-source tooling that others are generously investing in just isn't good enough, then you should ask for a refund."

I don't like that attitude - basically he says FOSS is allowed to suck because it's gratis?

I don't think libaom/rav1e/svt suck, btw. But it is a concern to me that if you want best encode performance, the proprietary encoders are better (maybe? I don't really know)

What's the point of making a royalty-free codec if you still need to get (expensive) closed-source proprietary software to make good use of it?

Not saying that av1 is in that situation atm, but it is a risk imo

Also, as I wrote in AV1 discord, about the significant improvement in quality even in real-time software AV1 encoders compared to the AVC that some companies were showing a couple of years ago, judging by this comparison I have big doubts <https://www.reddit.com/r/AV1/comments/pi411a/av1_still_the_current_future_of_video/> <https://blog.xaymar.com/2021/08/19/av1-still-the-current-future-of-video/>

Yes, it is possible that some closed encoders have better speed and quality, but I don't think it is that significant, and judging by these comparisons, software AV1 encoders (in real time) are still not much better quality than existing AVC, with a huge increase in computing requirements

Even many real-time streaming platforms still haven't switched to AV1 even though they wanted to do it years ago, most likely software encoding makes not enough difference in efficiency and maybe they are waiting for the availability of good hardware encoders

Afaik only Google meet android use av1? That too with very low resolution and on slow networks.

Yep, maybe for a very low bitrate it is reasonable

you might want to look into using `jpegtran` to do lossless crops

also you can try `identify -verbose` to see what quality imagemagick estimates the jpeg has, but that's a heuristic thing, since different jpeg encoders do different things

jpegtran is part of libjpeg-turbo/mozjpeg, identify is part of ImageMagick

avif in firefox postponed again: https://bugzilla.mozilla.org/show_bug.cgi?id=1682995#c70

And it will also once again postpone the review of JXL patches

it's somewhat frustrating from my point of view: the biggest advantage AVIF has compared to JXL, is that they "were first" - they finalized the bitstream first, they got browser support first, so they "won the race" in that sense: they were the first to beat what was there before (JPEG/WebP) and now according to some people, the game has changed and JXL doesn't just have to beat JPEG/WebP, it also has to beat AVIF. But on the other hand, AVIF support is still lacking in firefox and safari, and probably a big reason of why jxl support is not advancing more rapidly in browsers is because browser devs are still busy with AVIF.

I find it funny how the AOM is constituted by the same entities that own the internet and still can't impose their codecs due to lack of support, etc

AVIF is also first in slowness.

hey guys, 👋 I'm having trouble trying to find / remember a website / web app... months ago, while perusing some comparisons between codecs (still images and video, like JXL, AVIF, HEIC, etc) I came upon a webapp which had a few images (the first and default one was a car racing track) which allowed the user to choose: 1. whether chroma subsampling was applied to the image, 2. whether interpolation was used on chroma, 3. and some sliders to lower the resolution of the chroma components (it could go REALLY low); now I need it to explain chroma subsampling and just can't find it anywhere 😦 it was a very nice demonstration of it (of course, it transformed the image using js) but I can't remember does it ring a bell?

lol i've gone through this same thing with the same site... i might be able to find it in my history

https://goo.gle/yuv <@!818626963639238668>

Note that in that demo, the effects of lossy compression after subsampling are not taken into account

thank you so much!! that's the one (saving it)

this is actually just to show "you could severely subsample chroma in most images without anyone noticing", so I guess "works for me"

I know, and it's a neat demo to show that, but in practice you also get artifacts from doing lossy on the subsampled chroma and they do make problems easier to notice - any ringing, blur or blockiness will get magnified by the upsampling factor and become more problematic than what it would be with nonsubsampled chroma

yeah, I see wha1t you mean

I don't fully undestand the intersection of chroma subsampling and Chroma from Luma in a way, CfL could basically undo a lot of chroma subsampling's artifacts, to some extent? (if a good prediction can be found for a given block) CfL works on the full resolution image, after all

are you talking about av1's CfL or jxl's ?

we both have a concept of CfL but it's not quite the same

I don't know the details of CfL in av1 - but I assume that it doesn't work on the full resolution image, because that would mean you are returning a yuv444 image when using yuv420 mode

you're right, it's not quite the same... in jxl, can djxl use CfL to reconstruct a full-resolution Chroma plane?

or does it downsample luma when predicting? that would make it less complex I assume

CfL in jxl is only really meaningful when doing 444 (but we only do 420 when recompressing 420 jpegs)

oh

in jxl, CfL is signaled as two local multipliers (per 64x64 region) that get multiplied with the luma AC coeffs and added to/subtracted from the two chroma AC coeffs

for DC there's a fixed multiplier for the whole image

or the whole DC group, I don't remember

for that to work, you need the DCT transforms to be the same for the 3 channels - so it doesn't work when chroma is subsampled

in av1 I think they do things in the pixel domain, and I think they have two local constants per chroma channel (an additive term and a multiplier). I assume they downscale luma to make it match chroma in case of 420

not sure though about av1

yeah, so it's in frequency domain so no impact on chroma subsampling... I'd guess, in general, Chroma Subsampling has much better compression efficiency than 444 + CfL, right?

nah not really

chroma subsampling is basically exactly the same thing as zeroing 3/4 of the chroma coeffs

you don't need chroma subsampling to zero those coeffs if you want to zero them

it's just a bit of a memory optimization if you can skip the upsampling and give yuv420 buffers to the gpu

and in codecs with poor entropy coding, like old jpeg, it does help because you have less chroma DC to encode and chroma AC can also be encoded more efficiently because you're using 16x16 blocks for chroma instead of 8x8 ones

but in codecs with good entropy coding, the compression advantage of chroma subsampling is quite small

while the disadvantage of chroma subsampling is quite large: you HAVE to zero 3/4 of the chroma coeffs, even in spots where that's a really bad idea

what is this

Av1 wallpaper i do not have the lgbt one and the Halloween one

They exist

The font i used is elektra text pro

The one with the snow the author is Alex of notenoughav1encodes

Lgbt was not made by him

can svt av1 with libavif do qp 18 min qp 65 mx speed 3 sharpness 2

is any efficient and there's any recent build 09 september 10 september

or i should wait for libwebp2 and libjxl

they look square so probably a bad wallpaper (unless u want a massive white or black border)

good for a square monitor though

Right

well most ppl don't have square monitors lmao

my coworker actually has one in the workplace, I was amazed

damn

what res?

I don't know

yes it is really a square (1:1), the coworker complained about its resolution so I guess its resolution is not too high (1600x1600?)

the complaint was that, the coworker really wants to use a square display but it is super uncommon so one with higher resolution or dimension is non-existent or very expensive

found an eizo one on amazon

looks like it's only a singular seller tho

https://www.eizo.com/products/flexscan/ev2730q/

eizo is generally only used in enterprise tho afaik?

well makes sense

cdw sells it for $1400 <:monkaMega:809252622900789269>

it only has 1 dp and 1 dvid input

60hz

300cd/m^2 but advertised as 100% srgb?

There are also monitors you can rotate, might be useful I guess when working with portrait stuff

rotating messes up subpixel font stuff though unless its grayscale

Subpixel rendering is a neat trick but it's very fragile, you need to be sure of the display RGB subpixel layout

Phones and tablets have rotating screens anyway, so I think subpixel rendering is on its way out

grayscale one is not but for rgb id say yes

Grayscale is not really subpixel rendering, but just doing anti-aliasing properly

subpixel rendering with rotation would be possible but it would be a hassle

phones have such dense displays that it’s not worth the trouble

regular antialiasing is good enough

oh yeah was abt to say that myself its not subpixel if its grayscale

To me, subpixel rendering is a neat hack, but on high density screens you don't need it, and on low density screens it can cause visible color fringing imo. There's a sweet spot of screen density where it's a good idea, but there are disadvantages like screenshots no longer being portable to other screens.

generally it is portable enough except when things are rotated

at least looking at majority of displays sold rn

At some point I considered adding a Modular transform that basically converts subpixel rendered text to 3*width grayscale text and back

It helped a bit for compression

interesting. how exactly it works

But it's a bitstream complication, and I am happy we don't have to deal with it

Well basically just take the RGB data and reinterpret it as 3 grayscale pixels per pixel

Or the other way around

That basically seems to be the way most subpixel font renderers work

If we would define the transform at a higher level in the bitstream (not in modular), then patches would also benefit, because now we're missing lots of opportunities in case of subpixel rendering since you get 3 totally different pixel patterns depending on where the letter starts exactly

But it would be a big bitstream complication

And it would be hard to make it work well with mixed content that is not just subpixel rendered text

hmm essentially inlined rgb channels

You can literally take a ppm file with black text that uses subpixel rendering, change the header to that of a pgm with 3x the width, and it will make more sense for compression

PNG doesn't suffer much from subpixel rendering because it is basically gzipped ppm, and for gzip it looks exactly the same as the 3x width pgm

jxl and flif do suffer from it because they encode in a planar way, not an interleaved way like png

And subpixel rendering causes weird colored edges that make no sense for neither lossy or lossless compression

(it messes up color decorrelation attempts completely)

Somewhat funny to me that someone encoding a movie in av1 is considered reddit-worthy news

https://www.reddit.com/r/AV1/comments/pmpzok/big_bucky_bunny_2160p_60fps_is_now_av1_10bit/

Also about the resolution and that 16k is enough for everything, after some experimentation with random Manga/Manhua/Manhwa images

Yeah, I've often seen, for example, guides which are greater than 16K vertically.

OK, not quite actually.

This image is 21.6K:

Discord does not like that, lol

looks like op is self advertising a video he encoded <:kekw:808717074305122316>

If it is a transcode that isn't very interesting. If, on the other hand, they got Blender outputting AV1 directly then that is more interesting.

looks like it's just a transcode

oh wait

blender can use ffmpeg to output video apparently

nope appears to just be a transcode

anyway afaik to get the big buck bunny files u have to pay the blender foundation?

oh nvm

u can download the blend files for bbb

it's 800mb <:kekw:808717074305122316>

That's some very good compression for a lossless version.

Though I think the decode complexity is a little high.

Lol

https://encode.su/threads/3701-GDCC-21-T2-Multispectral-16-bit-image-compression

``` --------- pngcrush ------ flif ----- ccsds -- pik-gray --- pik-rgb ------ lzma --- tpnib C -------- 301,032,449 257,964,169 296,274,782 278,858,839 260,087,773 278,943,805 272,547,287```

So these are images and can be compared with Jpeg XL 🤔

interesting

I wonder how they passed 32-bit floats to those image codecs, because none of them can do lossless float

probably just as two 16-bit ints per sample, or something

is there a way to do

for %i in (C:\Users\Utente\Documents\Facebook\*.jpg) do avifenc -q 93.13 -sharpness=3 -enable-chroma-deltaq=1 -s 5 "%i" -o "%i.avif"

or av1enc

why it fails

i copied parameter not listed by the encoder such as quality shaprness and delta quantization

Hmm, it was sad to discover that some sites started using even FPGA WebP encoders (with something like 50% more compression than Jpeg and sometimes with transcoding only on the fly) with a future move to FPGA AVIF, but they never heard of Jpeg XL <:SadOrange:806131742636507177>

Fpga vp8/av1, I wonder how that performs compared to software

For the web, I suspect software encode is a better idea than hw encode

For camera, you don't care too much about density, so stupider hw encode is ok

Usually it's some kind of hybrid FPGA which also does cropping, resizing and other operations and as I understand on very high-load sites, speed of software encoding/processing is not always enough or they are quite expensive for the CPU

And until recently I also hadn't heard much about FPGA WebP encoders and also didn't think anyone used them, but they do exist and are used 🤔

I wonder what kind of site that is then

Cloudinary does do quite a lot of encodes but so far cpu works well for us

Afaik it's the same for twitter and facebook

Youtube does hw video encode for low traffic / quick encodes, and switches to sw encode when there are more views

Afaik

It's kind of like photo sites or Instagram with a lot of mobile users, maybe it's somehow more cost-effective

Instagram is facebook, they also don't do that

I think hw transcode is done by some who do it on the fly on every request

Yep, I also don't know what FPGA solution they use, the first thing I found is <https://www.inspursystems.com/fpga/use-case/webp> but as far as I understand they use something else that was developed with Intel

I still doubt it really makes sense to try to save on storage/cache so badly that you need hw to just repeat the encode every time you need the image

45 watts 😧

Maybe they have a cache on the most used images, but they don't want to transcode all available images, but do it in real time for rarer content (and software encoders can't do it with the needed quality, speed and latency)

You can do that in cpu too

The thing with custom hw is it's a pain to deploy in a decentralized way (which is essential if you want it to work well not only close to your serverfarm)

As far as I know they only use their own servers and do not use third-party CDNs

That is a really bad idea unless you only target a geographic region around your servers

Hmm, probably, perhaps this is also one of the main reasons why some kind of custom FPGAs is not used as often for certain very specific purposes (other than for example offline encoding of some content long before distribution) 🤔

Hmm, as far as I understand it is something based on Intel Arria, perhaps with some custom features/designs 🤔 <https://www.intel.com/content/www/us/en/products/details/fpga/arria/10.html> So maybe this is something more flexible for programming and Jpeg XL can also be "accelerated" in a similar way if needed

Likely. Also 'generic' DSP chips are available on many SoCs and could probably be used to accellerate stuff like DCT or color transforms

🤔 https://www.ipol.im/pub/art/2021/325/

https://twitter.com/OliverJAsh/status/1441095716227543042

🤔 https://twitter.com/OliverJAsh/status/1441107335171043334

AMAZING

So when can we get JPEG XL in unsplash? <:FeelsReadingMan:808827102278451241>

at least for higher resolution not the preview

https://twitter.com/alyssarzg/status/1441595107615064065?s=19

I get 38.2 kb jxl modular

cjxl -d 0 chroma-444.png test4444.jxl

>stat -c "%s" test4444.jxl 39151

JPEG XL encoder v0.5.0

Disabling patches takes it down to 24Kb

Size 38917 - JPEG XL encoder v0.7.0 cdd3ae6

``` cjxl -m -E 3 -I 1 -e 8 --patches=0 JPEG XL encoder v0.7.0 cdd3ae6 [SSE4] Compressed to 17165 bytes ```

Still bigger than webp

```cjxl -s 9 -d 0.0 -g 1 --patches=0 JPEG XL encoder v0.7.0 cdd3ae6 [AVX2] Compressed to 16227 bytes (0.141 bpp)```

```cjxl -s 9 -d 0.0 -g 1 --patches=0 -P 8 -I 0.65 JPEG XL encoder v0.7.0 cdd3ae6 [AVX2] Compressed to 15622 bytes (0.136 bpp). ```

Yeah, lossless webp is too good. 😄 ``` cjxl -m -g 1 -I 1 -P 8 --patches=0 -e 9 -Y 0 --palette=9 JPEG XL encoder v0.7.0 cdd3ae6 [SSE4] Compressed to 15477 bytes (0.134 bpp). ```

``` $ ../build/tools/cjxl chroma-444.png chroma-444.png.jxl -q 100 --patches 0 -e 9 -P 0 -X 0 -Y 0 -I 0 -g 3 JPEG XL encoder v0.7.0 59851dc [SSE4,Scalar] Read 1280x720 image, 62.7 MP/s Encoding [Modular, lossless, tortoise], 4 threads. Compressed to 9580 bytes (0.083 bpp). ```

it's one of those cases where lz77 matching helps a lot, which is the core thing png/lossless webp do, but in cjxl it only seriously attempts to use lz77 matching with `-e 9 -I 0`

yes, not to mention having to use very non-default options 🙂

just shows there's room for improvement in the default options

lossless is always a very tricky thing if you have these specific images that can be losslessly compressed to 0.1 bpp

``` cjxl -e 8 -m --patches=0 -P 0 -Y 0 -I 0 -g 3 JPEG XL encoder v0.7.0 cdd3ae6 [SSE4] Compressed to 12863 bytes (0.112 bpp) TotalSeconds : 0,2933563 cwebp -mt -m 6 -lossless -z 9 Lossless-ARGB compressed size: 12238 bytes TotalSeconds : 0,3493875 ```

It could be smaller 😄 ``` cjxl -e 9 -m --patches=0 -P 1 -Y 0 -I 0 -g 3 Compressed to 9472 bytes (0.082 bpp). TotalSeconds : 14,3653558 ```

a webp-like encode mode could be made that is faster and better for such images - basically skip tree learning and instead use fixed trees with one context for every 16x16 block or whatever

so e 9 is 50x slower than e 8?

does anyone know if and how you can get a partial webp decoded?

or otherwise emulate progressive loading of webp

<https://developers.google.com/speed/webp/docs/api>

Seems like there are no easy methods

Doubtful. Can't do it with AVIF either. They are considering including a low res version of the image in the metadata to use as a placeholder while the image downloads. I created a demo that uses a small separate image with a blur filter as an example: https://thumb.avif.techie-jim.net/

Could do the same with WebP

What kind of efficiency hit would that take though?

The point of progressive is that the efficiency loss it pretty slight since it is simply rearranging the data that is already needed to be encoded so that it can be decoded partially

Would it even be worth it to use two separate AVIFs/webps over jxl or even legacy jpeg in situations where progressive is something you want?

Probably, but it only makes sense if the preview is low enough quality to not make the total size too large

And of course it's a very limited kind of progressive, only one intermediate preview...

It's really not efficient at all; It adds a few extra kilobytes to the payload. But for larger images it at least displays a preview early on so you don't just have an empty space or content being pushed around later (when `height` and `width` are not defined). Also, web speed tools are considering image "time to first render" as part of the page speed metrics... so AVIF may tank a site's metrics. Having a small preview show up early is a [slightly gimmicky] solution.

i wonder if same approach could be used for lossless non-progressive jxl, because ll progressive is hella inefficient

Yes, you can. See my preview link above. I wrote on it that it can be used with any image format, including jxl (though I would favor progressive or responsive loading). All you need is to produce your regular image and a small resolution preview image and with a bit of extra html (and javascript if you want a smoother transition), and viola! As far as embedding the preview in the metadata, the encoder & decoder would have to be written to handle that.

extra js is eww

As I said, the JS is only for a smooth transition - there is no way for CSS to detect when the image is loaded. Without the JS the larger image still "pops in" when loaded, just not smoothly.

As I said before, a pretty large percentage of people don't like the very low quality blurry preview image and they even more prefer solid color, but the incremental loading of the image is accepted well by most people and it is already there in lossless non-progressive jxl

True, but when your image format doesn't support that... gotta go with something less desirable. You can go with a solid color but if you want it to be the primary color of the image there needs to be some sort of backend/server processing to find it first.

i actually kinda like decoding square by square it looks less boring than jpeg/png's smooth line by line

as for avif, eh stuff should be just tiled, will improve encoding/decoding perf too

For me it will be jarring at first but I will get used to it.

I believe AVIF is looking to possibly do that in the long term but as their codec is written now it would take a lot of rewriting to support progressive loading.

They wrote it to be part of the video codec - were not really thinking about how the keyframes could be displayed on the web.

In my opinion the main advantage of progressive decoding is the near full quality stage or the ability to use lower resolutions in jxl from the same image

its kinda mess, im not surprised some people just like jpegs and pngs

Yeah it will remain a niche thing that fewer sites will support. Should work fine as video capture thumbnails but larger images, no. It should work fine as a lower resolution background. I don't think CSS backgrounds support progressive rendering.

less likely to be usable for css background as iirc theres no such thing as <picture> for that

What do you need picture for?