JPEG XL

There's an open issue by Jon

adopted it where?

they have thousands of open requests

unless someone actually makes a PR for it, it's not gonna do anything

In the browser? because that's decided on by Jim Bankowski, lead of the Chrome codec team

In the library

Well, I don’t think they’d want to use AVIF as an image capture format unless they know that the phones will all have hardware encoding chips. The encoding speed is just too bad.

like I said afaik there isn't even a standard camera or gallery app included in the base android distribution

as far as capture goes, it mostly just relies on the camera firmware itself for handling that, so that wont change anything anytime soon

Theoretically, the camera app could request the raw pixels from the camera sensor and then do the encoding itself

I know like, 2 camera apps that do that and one is defunct

I thought JPEG comitee was busy with exactly that. Or was it standalone camera devices only?

no idea, gonna take a long while before they land in any android phone though

Ah, I see... Yeah, avif codec seems to be built into the OS with android 14. But I still don't think it looks like it's anything more than this one guy that for some reason decided that jxl must be stopped...

aosp is a very much "if a vendor want's it we can do it if they submit code" kinda thing

He is the project lead responsible for the decision and he has not named anyone else after all

Also given the original chrome bag report, it seems like the vendors do want to use the format

android vendor is a different beast

like I said, I havent seen any traffic at all in a PR or anything

Yeah, vendors like Samsung are already including a jxl codec on their own

Has Jon made a pr?

Are there any examples where they accepted a big feature like that from a random user? Not just an optimization or a bugfix.

I would think apps abandoned by Google during Jelly Bean era would be not-dead if they accepted PRs...

are they? I havent heard of anyone that samsung is going to have JXL support 0.0

I cant think of any times a user has added a big feature at all

then again, AOSP is a massive project that has tons of sub dependencies, so it really wouldnt surprise me if it has happened

Samsung is using jxl-in-dng, afair

ah so not anything platform related

Gallery app has a working decoder for jxl codestreams but only loads it for DNG 1.7 raw images. Even budget phones' gallery app received it so that demonstrates itd be smart bypassing google and going for vendors with their own apps preinstalled and autoupdating

Unfortunately the only options are completely raw bayer data, or jpeg at quality 100. There's no way to get processed, but uncompressed data from the sensor

They've had WebP since 4.2.1 and AVIF since Android 12

Just tested it in Android 14 with this: https://github.com/link-u/avif-sample-images (pretty cool that AVIF have this, *unlike* jxl). Built-in Files app doesn't recognize them, Google's Photos app shows some of them, very slowly, but most are very broken. Looks like those files are too advanced.

Webp are also used in Android development, i.e. as in-app resources format

The AVIF in Android 12 is quite a limited and poor implementation though. No alpha, only 8-bit 420, old version of the file format. Rather rushed, if you ask me. Which is annoying now, because we are seeing Android app devs assuming they can use AVIF in Android 12+ and then they complain that some images don't decode or look wrong.

I suppose it's the same people who rushed avif into Chrome who also rushed it into Android. Problem is: bugs in the initial rushed version can easily be fixed if it's Chrome, since most Chrome installs get updated regularly. But in Android, there's a really long tail of old Android versions out there, and they don't get updated. So the bugs 'stick' for a really long time.

(it's for that reason that many apps and app dev frameworks don't rely on Android system libraries but just ship their own copy of everything they need)

Interesting, I did not know that. That would explain a bug report I received a few years ago on my Paint.NET AVIF plugin about Android 12.

I think we made a wise decision not to pursue getting system-level jxl support in Android back when AVIF was boasting about it. It creates more problems than it solves. Maybe it's good PR, but it's not a good approach from an engineering point of view.

I don't know if it has already been mentioned, but the Windows 11 24H2 preview SDK has JpegXL support in the OS imaging framework (WIC). Credit for spotting it and the following image goes to Paint.NET developer Rick Brewster:

Looking at the other SDK headers, it appears that they may also support animations.

it's been brought up in here a few times and there has even been registry entries spotted in fresh installs as well, although it seems like there isn't any actual decoder or encoder shipped with Windows or on the MS store yet.

I think it is a feature in the Win11 preview releases, or perhaps Microsoft is shipping the public headers in the preview SDK while they work to finish the OS implementation. I have no idea how their release process works.

It was first spotted in the SDK, then registry entries on Windows 11, and most recently Wincodecs.dll can create instances for the encoder and decoder, but the MS Store codec hasn't been released yet so they don't do anything

SDK https://discord.com/channels/794206087879852103/803574970180829194/1201015191492640818 Registry https://discord.com/channels/794206087879852103/803574970180829194/1216042795643568278 Wincodecs https://discord.com/channels/794206087879852103/803574970180829194/1255914607235829852

Ok, so they are using the store to deliver the actual implementation. I guess that makes sense given that they did the same thing with AV1/AVIF. I just hope that this time around they release and support a fully working codec implementation. IIRC their AV1/AVIF implementation is effectively abandoned and doesn't even support transparency, which is ironic as that means it can't load all of the images that they contributed to the AVIF test suite.

It's set to support Animation, Lossless and 32 bpp with HDR10 based on the linked messages

I'm mostly hoping they use multithreaded decoding, WebP feels pretty sluggish on my CPU due to slow single core speeds with 15 threads doing nothing

I really really hope they actually release the codec and doesn't cave in to Google.

jxl must be stopped! Nooo! Stop adding support! There's not enough interest! The future is webp and avif!

The future is non progressive decoding yuv420p images!!

We need search engines to punish progressive images and reward serial-scanline webp instead!

generate your photographs as jpegs - with the now traditional visual artifacts and huge filesize - warning, must choose a high quality % to look acceptable

still salty anyone thought just adding a lite gainmap makes it better than capturing straight to hdr/jxl

people with SDR screens or devices would be salty at you for saying this [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&quality=lossless&name=av1_dogelol)

I did do a little bit of investigation, and I didn't see anything spectacularly dumb in the AOSP source code, so im thinking it might be a bug in libgav1 itself.

I have an sdr screen but gainmaps seem like a smoothbrain solution that creates more problems than it solves. Especially when you consider that nobody generates gainmaps by hand, so why save a bitmap when you could just save the instructions on how to generate the bitmap?

when creating a gainmap you do so in mastering process so you can verify it looks right

I'm pretty sure gainmaps are chiefly created by computers and not humans, and even if a human was involved in the creation, different regions/pixels of the gainmap aren't being edited independently from each other.

a gain map is more or less a bad difference map between two images, the two images are still independently masteted, and gain map fills the difference

In other words the gainmap is the output of some algorithm and it makes no sense to store it as a separate bitmap instead of just describing what algorithm was used to generate the bitmap. Using less bits.

Why store 2 separate bitmaps when you could just store 1 image and a tiny algorithm that efficiently describes how to derive the 2nd image

except for now you have to have either a really complex algo you need to compute which causes massive complexity and security risk, all for saving maybe a few kb

you pretty much are just making a computer generated shader at that point

As mentioned previously, having 2 separate bitmaps creates lots of headaches with editing, and just plain doesn't make any sense that the HDR image could be a picture of a dog and the SDR image can be a picture of a banana.

It's just a really dumb solution

sure it could be that, but is that *really* an issue?

Well, the issue is, why have such a stupid solution to a problem, when there's a far more obvious solution that doesn't have any of those problems?

the far more obvious solution you present is far more complicated to implement

gain maps give per pixel, and possibly per channel tonemapping

if you want to do that with a single algo, thats gonna be one long complex algo

No reason why it needs to have unbounded complexity either. I'm pretty sure the algorithms that generate gainmaps all run at realtime speed in order for them to be viable for use in cameras

you need to parse and render that at client side

that's an issue

Pretty sure gainmaps are typically generated at realtime speed

In editing software and camera firmware

yeah, but I dont need to implement an entire shader parser just to decode a gainmap

It's just dumb to encode a gainmap in the first place when it's the output of a simple, realtime-speed algorithm.

There's no need to

its not simple

not if you want the flexibility gainmaps provide

You don't want that flexibility. That flexibility is the problem and the source of editing headaches.

You don't want the SDR and HDR images to be completely separate images with no automatic relationship with each other and require separate editing.

thats outright wrong

gainmaps are the *minumum* that people are willing to concede

You want to edit the same image twice, one for HDR and one for SDR

?

And not have any portion of the work automatically carry over?

Artists have already been willing to do this, distribution is the issue, Gain maps allow generating two properly distinct images from one base, and have it look actually good. Gainmaps are perfectly fine from a creators standpoint as long as the application you use isnt trash

Still a waste of bits to encode a whole bitmap that could be described in a small algebraic formula...

With reduced resolution too

you loose per pixel or per pixel group luma mapping with that

so its less flexible

and more prone to errors

A reduced resolution bitmap that could be algebraically generated in full resolution

I don't think it's common for individual regions of the gainmap to be edited independently of one another.

At that point you have 2 separate images. A dog and a banana.

you wouldnt edit a gainmap, you edit two different pictures

At that point it's not the same image anymore.

thats literally the point

It would make more sense to store them in separate files since it's 2 separate bitmaps anyways

It's 2 separate uncorrelated images, not 1

gainmaps let you efficiently serve two images, an HDR and SDR rendition, to the user without the user needing to worry which is which

its not some limited algo, it's a fully fledged two seperate images in one

But if those images are not correlated with each other anymore, it's deceptive to have them in the same file and have a different image show up depending on the environment.

That's like the ambiguous multithreaded PNG decoding

if the images arent correlated, it's just going to look really stupid when a app tries to decode it

https://www.da.vidbuchanan.co.uk/widgets/pngdiff/

the gainmap builds ontop of the base image, so it's just a suped up moddification of it, ofc its possible to do really stupid things with that, but just dont distribute stupid images

They're 2 separate uncorrelated images pretending to be 1 image

It's kinda deceptive

who cares? not artists, and not general users who get the image

It doesn't make sense from a technical point of view

people have "abused" gifs and apngs with the same thing

in the end it never became a real issue

To design a format like that doesn't make sense

It's a dumb idea

well, heres the thing, we had no alternatives, and we still have no alternatives

Especially when there's a more obvious solution that doesn't have those problems

gainmaps are the best we have, and no one bothered with this "obvious solution" so clearly it's not that obvious

Well, the alternative is tone mapping... and most gainmaps are just the bitmap output of a tonemapping algorithm. There should just be an agreed upon standard way to encode tonemapping parameters.

tonemapping sucks and is extremely limited in comparison to gainmaps

Tone mapping is something that's built into image viewers already and runs at realtime speed. There should just be a standard for specifying custom parameters for the tonemapping algorithm

since that's what gainmaps are being used for

gain maps give you per region tonemapping, ofc you could do this with an "algo" but then at that point it's just a bad gainmap anyways

It would make a lot more sense and take up less bandwidth than encoding a separate low-resolution bitmap to generate an HDR image from an SDR base image...

you get less bandwidth and less flexibility/detail

Apart from manually generated bitstreams, all gain maps will be the result of _some_ local tone mapping algorithm. At least at the moment I am not aware of any editing tool that lets artists produce custom gain maps manually.

Is it really that common for people to generate an SDR image that has strong region-specific differences from the HDR counterpart?

Yes, local tone mapping does provide much better results than global tone mapping

In a way that can't be described by an automated process I mean...

Imagine say the northen lights at night, the SDR rendition will need a much stronger or weaker rendition for the stars vs the northern lights

you also have content specific tonemapping too

many people will find that some images look better with bt.2446a, some look better with ACES' mapper, some 2390 etc. because dumb algorithms inherently suck for this

sometimes you need to make a muted section of an image in SDR when it should really be popping like sun + screen + shade

If there's no editing tool that allows custom gain maps then the gain map is generated by custom parameters to a tonemapping algorithm that would be more efficient to embed as metadata than an entirely new bitmap

I'm sure these tonemapping kernels all run at realtime speed too

> At least at the moment I am not aware of any editing tool that lets artists produce custom gain maps manually. Isn't adobe lightroom working on doing diffs? I thought I read that

I guess the problem is that gainmaps exist right now but standardized algorithms and parameter encoding does not

even with parameter encoding, I fail to see how this could be done in a way that is competitve to encoding a secondary image when it comes to regions

I don't see a way to manually adjust the gain map locally in Lightroom. I can adjust global parameters to the presumably local tone mapping algorithm it implements, but I don't see a way to adjust the result in specific regions.

When you encode a gainmap bitmap, does it even take advantage of redundancy and prediction from the base image data?

What's the advantage of having 2 images pretending to be 1 file?

I wouldn't be surprised if they just havent added it yet

No

would there even be much redundancy at all?

Conceptually I don't like it. It is like shipping movies as a Blu-ray disk and VHS tape taped together.

At that point just have 2 separate images, it's more honest and logical and predictable that way...

I mean 2 separate files, it's already 2 separate images...

No reason to pretend it's 1

except now you have to worry about everyone distributing it wrong

does html even support distributin both SDR and HDR and swapping depending on a tag or something?

I dunno, it's just my 2 cents. I don't have all the answers either, but on the surface gainmaps seem like a smoothbrain idea to me

Especially after hearing wb's explanation of them

I mean the only real solution is to force everyone to migrate to HDR fullstop. but thats not happening any time soon

The thing is, gain maps is an approach that tries to kill two birds with one stone: 1. Graceful degradation/interoperability: making things work in legacy SDR-only applications, while also offering an enhanced experience for applications that do support HDR. This is the main focus of the UltraHDR approach (and JPEG XT a decade earlier). 2. Artistic control over the tone mapping. Point 1 is very much about solving a temporary, transitional problem, while point 2 is fundamentally changing the concept of an image to be such that pixels do not have fixed sample values but the actual values depend on the viewing conditions and display capabilities, being interpolated between two variants of the image.

But these are kind of contradictory goals to some extent. Implementing gain maps based image rendering properly is very complicated, since you cannot just produce a single decoded image buffer: you have to keep both the base image and the alternate image in memory, and dynamically update the interpolation between them since the viewing conditions will also change dynamically (background illumination and display brightness can change over time). So it's more complicated than just having one HDR image. So my feeling is that very few applications will actually do that. Most will just use point 1: not implement it at all, since that also 'works'. This does give artists control over the tone mapping, but it does not give them HDR images. Other applications will partially implement it: use the gain map to reconstruct an HDR image, and then consider the image as if it is just that HDR image. That means artists do not get control over the tone mapping, but they do get HDR. I don't really see it happening in practice that points 1 and 2 are actually both achieved in a satisfactory way.

I disagree that it's really that complicated for the majority of consumer applications. Most applications won't need to balance out between SDR and HDR. It is true that brightness changes and in context needs to be accounted for, but even then, with a gainmap you have pretty much "HDR" mode and "SDR" mode. It's far more likely that applications would only need to, at most, render the rendition it needs, then leave mapping up to the operating system as per the current status quo. Applications that do juggle between (such as ios browser) them would far more likely to simply render out to HDR then do SDR -> HDR mapping for the rest of the content, much like how HDR is currently handled Currently "SDR only application" applies to the vast majority of systems and operating modes today. and for the foressable future will be necessary, if we have learned anything from technology progression, "SDR only consumers" will be around for at least another decade.

The gain map spec does specify interpolation between the two images, and that would be quite important if you have e.g. a PQ image that actually goes up to 10k nits and you want to render it on a 1k nits display that is moving between outdoors viewing conditions and indoors.

But yes, I don't think applications will actually implement that kind of dynamic interpolation, and they will rather be just decoding the SDR image when using an SDR display and reconstructing an HDR image just once when using an HDR display.

I thought the "android specification" for gainmaps more or less mapped to PQ with a hard nit value, Ill have to re-read it,

I was talking about the Adobe specification for gain maps

oh right, I forgot we need to deal with this, it really is "HDR 2.0" [av1_cheems](https://cdn.discordapp.com/emojis/720670067091570719.webp?size=48&quality=lossless&name=av1_cheems)

Anyway, the goal of the current ISO 21496 is to have just a single gain maps spec instead of all the proprietary single vendor approaches we have now, which I think is a noble goal. But I think the gain maps approach in general is just not a good approach if the goal is to get consistent rendering of HDR images.

In practice, the actual image an end-user will see will not be consistent at all: - they may see the SDR base image (in applications that don't implement it) - they may see the HDR image, tone mapped with whatever default tone mapping the application or OS or display is doing - they may see an interpolated image based on the display brightness at decode time - they may see an interpolated image that dynamically changes when the display brightness is changing

this seems like a lot larger scope then is really necessary or wanted. the latter 2 parts shouldn't even be in scope IMO. It should be left entirely up to platform handling. The second point I suppose *could* be an issue, but that is the current status quo anyways, unless you mean HDR->SDR, I don't forsee that being an issue since any application gainmap aware in the first place should be platform aware anyways.

IMO simple is solid, and solid is good. a gainmap's scope should be limited to taking an SDR image to a typical HDR transfer, or vice versa, (though HDR -> SDR I suppose might be harder )

wait, with gain maps, is it SDR+Gainmap=HDR or HDR+Gainmap=SDR?

i.e. is the source image in SDR or HDR?

in theory it is both ways, but IMO SDR->HDR is the only one that makes sense

no idea about the spec tho, Im not sure if it's public or not

Both ways are possible in the current spec. For 8-bit formats like de facto JPEG or Main profile HEIC, only SDR+gainmap->HDR makes sense. For formats that natively support HDR, I think the other direction is the one that makes the most sense.

does it make sense? I think the likely hood of an application supporting HDR being gainmap aware is significantly higher then the alternative, even if potential quality is higher, the loss in app support is far more significant, which pretty much breaks gainmap's single greatest usecase

but most software supporting new formats (ie jxl) should probably be hdr aware, and if not, the decoder can take that burden

are iPhone HEIC photos Main profile?

this is not true due how libraries work in the first place, many applications "just work" with JXL when their decoding library adds support for it

also very little linux software is HDR aware in the first place anyways

Developing an HDR application is often not "update your app's gui framework" sadly

shouldn't the decoding library support tonemapping? Couldn't you could have the libjxl (or whatever) decode function have an `bool is_hdr` argument which defaults to `false`?

this is a highly unique feature to JXL

the answer is no

you may have rudimentary support in your colour management system, but well, tonemapping sucks

I suppose it could be a decent fallback, but it's not reliable at all

Yes, as far as I understand. In principle, HEIC is always Main profile — if you want to use other profiles, you need HEIX.

Basically the question boils down to: do you want the SDR image to be reliable/well-defined, but you don't really know if and how the HDR image will be rendered since that will depend on the application (that's what you get with the SDR+gainmap->HDR approach), or do you want the HDR image to be well-defined, but you don't really know if SDR renditions will look good (that's what you get when encoding just an HDR image)? With inverse gain maps (HDR+gainmap->SDR), in principle you can have both images well-defined and looking good, but I think it's overkill, and it would be better to have a well-defined parametrized "rendering intent" for HDR images, which basically normatively defines how to do the tone mapping (i.e., parameters of a global or local tone mapping algorithm) so it is consistent.

But then people have to agree on those parameters and on an on-disk format to serialize them.

if the specification properly defines on how to render the gainmap out then the rendition should more or less be as well defined on the SDR, and any applications failing to do so would simply be spec in-compliant

I think ICC profiles could be extended with a more elaborate rendering intent description than the current 4-value one (which is meant to describe gamut mapping rendering intent, not dynamic range mapping).

iccMAX? https://www.color.org/iccmax.xalter

https://github.com/InternationalColorConsortium/DemoIccMAX

I don't see anything related to localized tone mapping or tone mapping of any kind though

`Programmable transforms (e.g. direct encoding of device models) will be supported, with functional operators, conditional evaluation, persistent variables and vectorized operations for improved performance.` So, executable code in ICC profiles? Great idea

no no, ICC 5 is not something anyone really wants to implement. They're working on a new revision of ICC 4 which should get some kind of tone mapping intent, but this is still very preliminary work

iccmax is a crapshoot, ~~feels a lot like tiff tbh~~

Interesting...

I guess in the meantime gainmaps are in a much more mature and usable state despite its flaws

Not really, there are different single-vendor ways of doing gain maps, there's the JPEG XT way that was defined 10 years ago but nobody really uses it, and then there's the new TC 42 spec but that one is only at CD stage so I wouldn't call it mature yet.

I wonder why XT isn't more common...

How does XT’s work ?

It's just a regular JPEG file, compatible with all existing JPEG decoders. But it adds extensions for alpha channel, lossless, and floating point HDR, and cool stuff like that.

https://github.com/thorfdbg/libjpeg

Looks like it's literally just a drag and drop replacement for libjpeg too. Seems like a no brainer, I'm kinda surprised it didn't get any traction

And instead people are literally re-inventing the wheel like with Android JPEG HDR which does the exact same thing but doesn't follow the existing standard way of doing it

Hmm, it doesn't actually seem like such a drop-in replacement after all...

I wish this was resolved <:PepeHands:808829977608323112> https://github.com/mozilla/mozjpeg/issues/437

no, it's not a drop-in replacement like mozjpeg and jpegli. But it does implement the full JPEG standard and also JPEG XT.

I guess the main problem with JPEG XT has been that if you're doing things that won't work with non-updated existing jpeg decoders, you can just as well use a newer codec that will compress better.

https://www.digikam.org/news/2024-07-14-8.4.0_release_announcement/

Problem is JPEG is still king...

not for long !

It's the gold standard. You can even use newer codecs and still be compatible with JPEG. :)

Since people are still struggling to improve upon it today, especially with the new arithmetic coder patents expiring

And new codecs like jpegli or even pik and lepton

jpegli isn't a new codec? (**EDIT:** I'm trying to say that jpegli isn't it's own codec it's just a new encoder for an old codec)

I wonder how viable itd be to generate reconstructible jxl directly, without requiring a jpg already created

lepton devs push users to turn to JpegXL instead, and pik became JpegXL

Android just made "HDR JPEG" and I struggle to see how or why it's different than jpeg-xt part 2 or part 7

It's a new codec for an old format 😂

pretty weird to put in the same list as Pik or even Lepton

pik and lepton are "new" ish codecs that leverage existing jpeg compatibility

Lepton and Brunsli were new formats for losslessly packing JPEGs

Pik was an entirely new image format meant to replace JPEG

sometimes people use the word codec to mean "format" and I think that's a valid use for the word but I think it's even more commonly used to refer to a "coder/decoder"

I was thought of back at launch, but not implemented yet

https://res.cloudinary.com/cloudinary-marketing/image/upload/Web_Assets/blog/Encoder_diagram.png

I haven't heard of Brunsli, I'll look it up. As for Pik, I'm pretty sure it was designed to be compatible with JPEG too?

jpegli is just a new encoder and decoder pair, not a new format at all, and so quite different from those three above.

No, PIK was essentially what JXL is now. Jpegs could be 'upgraded' to PIK with a 20% reduction but it was a new format

Brunsli was Google's JPEG packer

a coder/decoder pair is often properly called a codec

Early versions of JXL used Brunsli for JPEG compat instead of VarDCT + extra metadata.

https://github.com/google/brunsli

I haven't looked at the development history of Pik, but presumably it also used Brunsli for JPEG compat.

Although, while looking in PIK it pointed out low generation loss by encoding Jpeg lossily with the same 8x8 DCT blocks. Makes me wonder how much of an impact that would have in VarDCT if they were matched

And also losslessly converted back to JPEG

So it's a new way of storing jpegs like lepton was

but that's irrelevant to the point

like you're putting "new" in scare quotes because a new, much more capable format also has some compat features

From what I can see, it was only one-way, and then it was moved to JpegXL instead https://github.com/google/pik/issues/40

lol I'm putting "new" in "scare quotes" because it's debatable if lepton should be called "new" when it's already considered "obsolete do not use"

but it's still a "new codec" relatively speaking

A new and already-obsolete codec

anyone checked librewolf? oddly people think it has jxl but its disabled by default and as bas as in firefox nightly (bad alpha, no animation)

librewolf has patches in it's repo for better JXL support but they aren't included/compiled into librewolf

funny thing is they don't seem to test if the browser works with them they just check if they can apply

tried stable librewolf 128, had to enable the jxl flag but it works

any way to get em merged?

its just lw is gaining mindshare over floorp and waterfox so might as well win the derivatives

the one they have in their repo is currently messed up a bit but I made a fixed version

although it seems that's not the reason they don't include it by default

it's been discussed in here before but there's a chance librewolf won't ever enable JXL support unless regular firefox does, which kinda ruins the point of them even having patches for it in the first place if they aren't ever going to actually use them

odd given improvements like jxl could diminish the performance impact of the other compromises they make in the name of security

ie sure jit is disabled but we still load faster than upstream

my theory is they are afraid of fingerprinting if they enable JXL support

because regular firefox does not support JXL

The problem is http accept headers. https://wiki.whatwg.org/wiki/Why_not_conneg

Browsers are an abomination and their poorly designed standards practically encourage and invite fingerprinting and privacy violations at every corner. Because the people in charge of making browsers and writing the standards all profit from it

accept headers themselves are great though, it's a shame CDNs are bloody retarded with them, around 70% of CDNs I come across will just crap themselves if you remove webp from the accept header, almost guranteed because whoever implemented it though it would be good to just chain `if not then break` statements

Also a small part of me wonders if someone would submit a PR to rip out libjxl and replace it with jxl-oxide if they would be less hesitant since they like rust and all.

huh? I don't think I've ever had websites break with this installed https://addons.mozilla.org/firefox/addon/dont-accept-webp worst that's gonna happen is they ignore it and still give you a WEBP ...it does mention they've had to add exceptions for Patreon and Reddit though

one example I have saved ``` ➜ ~ curl -s 'https://www.hollandamerica.com/content/dam/hal/inventory-assets/ships/RN/gallery/rotterdam-main-dinning-rom-c10.jpg' -H 'Accept: image/avif,image/webp,*/*' -o - | file - /dev/stdin: ISO Media, AVIF Image ➜ ~ curl -s 'https://www.hollandamerica.com/content/dam/hal/inventory-assets/ships/RN/gallery/rotterdam-main-dinning-rom-c10.jpg' -H 'Accept: image/avif,*/*' -o - | file -/dev/stdin: JPEG image data, JFIF standard 1.01, aspect ratio, density 0x0, segment length 16, progressive, precision 8, 1542x867, components 3 ```

I come across a lot of CDNs like this

hmm is the problem that the 2nd command is prioritizing JPEG over AVIF?

no, or else the top one would serve a webp

the issue is that it checks for webp in the accept header, then bails before it can check for avif

neat, betterdiscord has a plugin to embed images, so if you use bdbrowser with an enabled fork like thorium or a custom built electron with JXL support you can make discord embed the images, its pretty buggy since for some reason if you open a jxl image it will just open it 1:1 which makes nothing clickable on larger images EDIT: sadly it doesn't work with jxl-crx

``` bash-3.2$ curl -s 'https://res.cloudinary.com/jon/f_auto/sample' -H 'Accept: image/avif' -o - | file - /dev/stdin: ISO Media, AVIF Image bash-3.2$ curl -s 'https://res.cloudinary.com/jon/f_auto/sample' -H 'Accept: image/jxl' -o - | file - /dev/stdin: JPEG XL codestream bash-3.2$ curl -s 'https://res.cloudinary.com/jon/f_auto/sample' -H 'Accept: image/webp' -o - | file - /dev/stdin: RIFF (little-endian) data, Web/P image, VP8 encoding, 864x576, Scaling: [none]x[none], YUV color, decoders should clamp bash-3.2$ curl -s 'https://res.cloudinary.com/jon/f_auto/sample' -H 'Accept: image/jpg' -o - | file - /dev/stdin: JPEG image data, JFIF standard 1.02, resolution (DPI), density 100x100, segment length 16, progressive, precision 8, 864x576, components 3 ```

it's always nice to see good CDNs [av1_PepeHappy](https://cdn.discordapp.com/emojis/654081052012314643.webp?size=48&quality=lossless&name=av1_PepeHappy)

btw <@106423637103353856> , one of the CDN's im talking about is shopify, which is one of the larger ones ```ps ➜ ~ curl -s 'https://shop.vote.org/cdn/shop/products/mockup-of-a-baby-wearing-a-sublimated-onesie-30023.png?v=1652303987&width=360' -H 'Accept: image/jxl,image/avif,image/webp,*/*' -o - | file - /dev/stdin: JPEG XL codestream ➜ ~ curl -s 'https://shop.vote.org/cdn/shop/products/mockup-of-a-baby-wearing-a-sublimated-onesie-30023.png?v=1652303987&width=360' -H 'Accept: image/jxl,image/avif,*/*' -o - | file - /dev/stdin: PNG image data, 360 x 360, 8-bit/color RGBA, non-interlaced ➜ ~ curl -s 'https://premioinc.com/cdn/shop/products/ACO-3000_1_50x.jpg' -H 'Accept: image/jxl,image/avif,image/webp,*/*' -o - | file - /dev/stdin: JPEG XL codestream ➜ ~ curl -s 'https://premioinc.com/cdn/shop/products/ACO-3000_1_50x.jpg' -H 'Accept: image/jxl,image/avif,*/*' -o - | file - /dev/stdin: JPEG image data, Exif standard: [TIFF image data, little-endian, direntries=6, orientation=upper-left, xresolution=86, yresolution=94, resolutionunit=2], progressive, precision 8, 50x28, components 3 ```

so this will prevent a LOT of websites from serving avif or jxl

I informally passed the above bugreport to someone I happen to know at Shopify

I hope this is fixed 0.0.

looks like it checks for webp support first before *conditionally* checking for other formats

why is an accept header better than a <picture> element?

If you want to make variants based on both format and responsive breakpoints, it causes a bit of a combinatorial explosion that will make your html pretty tedious to write (or if automated, still pretty verbose and adding some overhead)

Ideally, with Accept headers and a client hint that says something about the viewport width, you can just use a single img tag and let the server worry about which variant to serve for each case.

AFAIR, it also checks for AVIF support second before *conditionally* checking for JXL.

I dunno, the arguments in "why not conneg" sound pretty strong to me. Especially wrt cache/cdn

I just tried completely stripping all "Accept" and "Accept-Language" headers from my browser's requests. So far I don't notice any issues... Imagine how much useless data could be saved and stripped from HTTP GET requests if browsers stopped sending it for every request...

User agent crap should be slimmed down a lot too and contain less fingerprint data like CPU and kernel type...

accept headers mean you don't need to go back and update stuff to serve new and netter images

you are likely to get a lot more jpegs and pngs without accept headers, which would really increase the data used

You are likely to get less webp, which in itself is a victory...

can't deny that [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&quality=lossless&name=av1_dogelol)

Let the browser choose a variant. That's what <picture> is for after all.

There ought to be a standard HTML tag for describing a simple TOC for a progressive image too. So browsers know how much data to request to get a certain size.

the picture element requires you to go back and edit every single HTML page to add support for new images, and requires you to have dedicated paths for each image, accept headers are a lot more simple for web developers and way better for websites that can simply set and forget. accept headers are why the majority of sites are actually serving JXL, without them, maybe you would have a hundred sites or so, maybe less. thanks to accept headers, literally millions of sites (4,794,511 made with shopify according to builtwith for example) support and serve JXL.

accept headers save a LOT more data then picture tags due thanks to every single web site developer not needing to update every single page/template

I remember trying to manually send a partial request to fetch half a JXL image, but I couldn't find a way to do it from the browser tools, only scrips for video players, ect where you can buffer a few seconds

does JXL have a method to signifify that you only need to download x amount of bytes if you want y image?

But the different formats are often at different URLs anyways and so the web server has to serve a different page depending on what the headers say.

Yeah, the JXL header has a table of contents

in some cases

But you need to already be reading the JXL file to read the header/toc

Is that only when using the container?

I don't think so no

It's all JXL files

If I recall someone was doing tests to figure out the average percentage needed for the first progressive pass in a set of files

The web page itself would need to contain some information in an html tag to tell the browser how much bytes to request to get a preview

Not all JXL is progressive either. Most lossless jxl is tile-by-tile

One tile at a time

The TOC is part of the frame header, it is obligatory. Decoders can use it to do parallel or ROI decoding. It could also be used to efficiently determine truncation offsets for the progressive passes.

well that's still "progressive" compared to other formats

This is an excellent "first step" I would say. You would STILL need a way to signal to the browser beforehand that the image is progressive-coded. But if a web browser knows that a file is progressively coded, and knows the size of the file, then it can request the first x% of the file, x being a simple heuristic informed by statistics and the file size and dimensions. There would need to be some statistical analysis based on a large corpus of images at various dimensions and bitrate levels to see what the average ratios are.

But again... there would STILL need to be a standard way to signal whether or not an image is progressive, in the HTML

I don't think there's any way for the browser to know whether an image it's about to fetch is progressive or not, and no one did any statistical analysis to establish a good heuristic to determine what % to fetch based on known information about the dimensions and filesize. Maybe there's a way for the browser to tell a server to stop sending the rest of the file?

The HTTP "I don't need any more thanks" signal

Is that a thing?

yeah this would be the best, if a browser could easily query how much bits are needed for the various offsets it could trivially stop the download.

Browsers could just fetch the first few kB of any image, and this would always be useful. For non-progressive images, it tells you the image dimensions which is useful to avoid layout shifts, for progressive jxl it tells you the offsets of the passes and perhaps you already get the first pass too, for small images.

You don't want too many round trip requests before displaying content. That creates unacceptable latency on certain connections.

The information should be included as part of the embed tag.

it can be done asynchronously

If you have to wait on a round trip before content is displayed, it's a bad idea.

the other thing that would work out well would be a browser simply telling the decoder what resolution is needed, then let the decoder send signal to stop the transfer, that would need some kind of state manager but well, browsers are full of that shit anyways

The problem with that is that http does not really have a way to abort a transfer. You can break the connection but that's a rather drastic thing to do since there will be many transfers going on through the same connection.

The best way to do it is server-side, prioritizing the transfer of the initial segment of each image above the fold (or close to the fold) corresponding to the first pass of each image. This is very nontrivial to do though, since servers generally serve requests in a stateless way and this requires per-client coordination of how all the requests coming from the same client are prioritized, i.e. it requires state.

Can’t the servers just send 206 partial content and then terminate the connection?

This is what I was trying to use, as just said above https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/206

Sure, but that doesn't allow the client to choose where to stop.

Theoretically some new header could work

206 responses would be a way to prioritize the initial pass of each image, but it still requires multiple requests. Server-side prioritizing of the initial pass would be a way to have only a single request per image.

What do you mean exactly by “prioritising the initial pass”?

It does oncw you have the toc and lqip and make a new request.

Then it can at least load what was sent and make a new request if it wants more. But then the server determines how much to send at first

And the server doesn't have dpi info

Or any kind of idea what the client wants

that's where Client Hints are useful, they can inform the server about dpr and viewport width

http stream prioritization can be used to send e.g. the first progressive pass with a high priority and the rest with a low priority, so if there are multiple images on a page, instead of sending them mostly sequentially, you would first get all the previews and then all the detail

But having multiple requests for each image causes a lot of unnecessary overhead

Anyone with an Apple device: please open a Feedback Assistant (https://developer.apple.com/bug-reporting/) bugreport about HDR images not working in Safari and in Preview. I have been told that the relevant Apple team is aware of the issue but has a hard time getting this prioritized, and for that it apparently helps if the bug gets reported by many people.

this is not JXL-specific, right? it’s all HDR images?

(including Apple-produced ones?)

Is it Safari-specific, or webkit-specific?

Could be Core Image specific

it works in the Photos app

Does preview use WebKit?

HDR images don’t seem to work in preview

webkit, I doubt it, but probably Core Image?

That’s why I was thinking it was CoreImage

and why I mentioned Photos which probably also does (?) and in which it works

Maybe it’s an issue with how WebKit uses Core Image (?)

If it does at all?

that would be my guess

that Core Image doesn’t automatically make it work, but also doesn’t inherently prevent it

that it’s up to the application to use it properly

So photos is probably using Core Image + something else?

To view HDR JXLs I use chromium because finder and preview tonemap them to SDR

Correct. I can't get any image format to render in HDR in Preview or Safari.

My guess would be that CoreImage by default returns 8-bit tone mapped pixel buffers, and only with a specific option returns HDR image buffers, as a way to avoid breaking existing applications that use it. And then they probably updated Photos to use the new option or new API, but not the rest.

Update: Ever since setting my browser to stop sending HTTP "Accept" and "Accept-Language" headers, most things still seem to work just fine, but I noticed that certain websites are a little bit weird if you aren't logged in. Like github. I guess some HTTP servers assume bot traffic if they don't like the HTTP headers.

Everything should work if you don't send Accept headers, you should just get the fallback for everything. No gz/br compression on your html/css/js, only jpeg/png/gif for images. It works, it just uses more bandwidth.

I kept the accept-encoding header, just not the other two.

Compression should be a mandatory part of http that's inferred in the version string. The other two are even more useless and dumb

The problem it creates for caches and CDNs seem like the biggest problem with the accept header

A few servers actually ignore your requests if you don't have webp in your accept header.

yeah, shopify T.T

If you don't like webp, you don't deserve to access their services :)

https://github.com/Eclipse-Community/r3dfox/releases/tag/v128.0.2

I should mention, it has JXL decoder

theoretically, `image/webp;q=0` in the accept header should tell websites that you do not accept webp, but in practice I think most web servers just see if webp is mentioned in the accept string and assume webp is supported if it's found, without checking the q=0

This seems to work pretty well. Less webp, you advertise/ask for jxl, and servers that look for and require `Accept: text/html` are happy. Best of all worlds.

idk if it actually matters but you can change the bottom one to `text/html,image/jxl,*/*` if any server actually cares about that. I don't think so, though.

unfortunately if you omit the `Accept:` header entirely, some websites will rarely behave weird. They're just expecting `Accept: text/html`

correct, since checking for the existence of substrings is much faster than fully parsing the Accept header

ehhh, it's not really any actual perf savings, more like dev time savings. this might be an issue if you are running on a pentium II or something xD

faster for the devs to code XD

also some web servers may not have modules to parse Accept headers at all, so for those web servers `*image/example*` might be your only call

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/source

Who needs to parse accept headers anyways?

CDNs that don't have access to upstream HTML

it's been explained why content negotiation is better than <source> elements

Accept headers cripple caches...

The CDN doesn't need to change the HTML...

Parsing accept headers and serving dynamic html is a waste of time and bad design

You're putting way too much work on the http server

whut

This isn't new or radical information here

Everyone already knows this

well if you want to add a new image option, how do you do that without changing the HTML

the CDN cannot do that without changing the HTML unless it has content negotiation

"Everyone already knows this" and yet many people here disagree with you, including professionals

I see, so if they add support for a new format, they have to modify the page to add it... the CDN has no control over that, it becomes the website owner's responsibility.

Shopify added JXL images to its repertoire and all the apple devices that can view them started seeing them. None of the shopify sites had to update their HTML to allow this to happen.

If you had it your way, literally all the individual sites would have to do that, even though they hire shopify to deliver what's best

There is a reason the experts do it the way they do it

Changing responsibility around isn't really a good or a bad thing, unless you assume website owners should have more or less responsibility

but it isn't just "changing responsibilities"

it's limiting the agency of website owners

Regardless of your position on this no one can argue that dynamic web pages are the root cause for performance problems with web servers and the first thing to get optimized is reducing the amount of work the server has to do.

And making things more static

right now, website operators can *choose* whether to use imagesets or whether to the server/a CDN go off of the Accept header

most of the performance issues with dynamic webpages are due to the number of roundtrips

You're saying that website owners shouldn't have that choice

not due to processing time from the accept header

why does it feel like every time I have a discussion with you I have to defend accepted industry practice developed by experts to counteract your opinion that everyone should just do things your way instead and everything would be better

I'm not talking about latency, I am talking about performance problems due to increased demand and waiting on CPU

parsing an accept header is fairly cpu-light

I think you're vastly overestimating the amount of cpu cycles that content negotiation requires

pcre is very fast, and you don't even need to use regex to do it

Making everything as static as possible is the universal first step to solving those problems

your perceived problems*

which problems though

DDoS problems or server can't keep up with demand

DDoS is not going to be fixed by avoiding an accept header

servers that can't keep up with demand aren't going to fix it by ignoring the accept headers

in fact, they can! they just need to serve one image for one URL

which is already possible to do

things that are slow on serverside are things that can't be made static, like database lookups

for me to log into a website requires the http server to contact its SQL server and compare the received credentials, generate a crypographic token, and send it over

that's the kind of stuff that is slow

accept header parsing is not

anonymous fetches are quite low-cpu intensive even if they have to parse the accept header. as you are aware, most sites don't do a fully compliant parse and just search for specific things like `image/webp`

I mean, things aren't absolute either, and there is a point and merit to what you're saying, just like there's also merit in saying that there are downsides to dynamic web pages and that accept headers are kinda stupid.

accept headers aren't "kinda stupid"

there's no merit to saying that anyway, as it's fairly unproductive and means nothing

I think <source> is a good alternative to them and most of the time they are just wasted bytes and another source of errors and wasted time in configuration.

if the server side render is server side render, an accept header isn't going to make it not a server side render

20 bytes saved on request is entirely negligible

and if they use a source tag with not server side render, it's not going to magically disappear

There is merit to what you're saying that they provide choice and some benefit when serving up dynamic web pages, but dynamic web pages are something that everyone optimizing for performance wants to avoid in the first place.

people aren't worrying about performance for server side rendering

True, but server side rendering is something that's avoided, cached, and usually protected against spam

web 2 relies on server side rendering

people want to avoid client side rendering because that's the slow one

What?

phones are slower than servers

any cpu time that can be moved from client to server is good

wikipedia for example, server side rendering, works without javascript

all the device has to do is render it once

avoiding JS as much as possible is the biggest bonus to perf

You don't need javascript to render a basic article with some text and pictures... But they do use some dynamic elements for logged in users.

YouTube, client side render, constant loading boxes, horrible on mobile and requires the app to be usable

the dynamic elements you are thinking of take like no processing power

It's super fast for servers

this has been solved

Jokes on you, their app is unusable too ;)

the UI might not be amazing but it's not slow

And it's only done for logged in users. Regular users are served a cached static page.

no they arent

well you can say everything is cached

Even for logged in

But it's not an issue

that's definitely not true

accept header isn't going to make it faster or anything

you can find out yourself by loading up an incognito/private window and then shift reloading

Really? I haven't examined the software that closely but I assume it does what everyone does and tries to make as many things as static as possible and limit the number of server side calculations it has to do before sending the web page

why would it need to do that when it's already so fast and takes no processing power

The only reason for caching is for bandwidth

I know because I've done it a lot

Because of how many people visit wikipedia, even a 5% difference in speed could be pretty useful

and how would it do that when the content is dynamic

And having a non conforming accept header if anything will miss the cache

Servers that serve dynamic pages usually try to cache and serve static pages whenever it gets identical requests. Parsing more headers and junk in the requests means there are less matches.

one of the fields for cache is the entire header

generally

at least aws has that

so if you modify your header it will do the opposite of your goal

The User-Agent header is probably the worst offender

Way too long and detailed

That doesn't matter

It just hashes it

The entire thing

Then it still matters because the more detailed it is the more variants there will be

yeah and you think only putting one thing isn't unique?

It gives details about your CPU and whether you're using X11...

what are you talking about

holy shit

The User-Agent header

Which part is confusing you exactly? Do you not know what x11 is?

I'm just saying that's too much info in the headers...

As someone who had to deal with this exactly for work, I'll let you know this is not an issue (on AWS)

Wikipedia does in fact cache pages and fetch already-rendered pages from geographically close servers when you're not logged in.

Mostly for fingerprinting/privacy reasons

And if you're logged in, *most* of the page is pre-rendered, but the response is not from a geographically close server, unless you happen to live in the right place.

So CPU-wise logged in and logged out are basically identical

But logged out users get a lower latency response

if you modify your accept header, you are making yourself more unique

It would be good if browsers could agree on common user agent and accept strings just for the sake of less fingerprinting. With the bare minimum of info.

they already do?

what is your user agent

Like "iPhone" or "Firefox" instead of the whole shebang

No need for version numbers, cpu type, etc

they're meant to indicate... the user agent

They could be made a lot more generalized

Look at curl's user agent

they are already most generalized

if you are using a normal browser

No...? They include a lot of unnecessary details.

like what

You're making stuff up

???

Do I have to repeat myself again? Or is there no point because I'm making it up anyhow? Do I need to go find and print out my user agent string for you?

Yeah sure

`Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0`

X11; Linux x86_64 is the name of the os

My OS isn't X11...

Yeah it's generalizing

That's not generalizing, that's being awfully specific about my machine...

how

It's a Linux machine I think that's useful