JPEG XL

didn't it become popular exactly because the small file size made it easy to share on the internet?

But that was later. And if that were the case, then by the time the Internet, the AAC format would have been the main one. but no

look

or maybe more like...

All of it can not only CD audio but also mp3 on CD

The first MP3 CD player was released in the year 2000, a few years after MP3 became a popular format online.

<:monkaMega:809252622900789269>

well

yeah in the 2000s :p to be fair, the first MP3 player in general, that used flash instead of a CD, was launched in like 1997, so around the same time early online music distribution/piracy was getting off the ground

Was the internet so bad that packet corruption was so frequent? Why then does mp3 have so many degrees of integrity protection, crc32 per block. absolutely every block of 20 milliseconds is marked with another 32 bits indicating the beginning of the block, which makes it easy to skip any amount of corrupted data

Which also makes it ideal for bittorrent. But it appeared in 1999

idk. You'd have to ask the researchers at the Fraunhofer Society that developed it in the early 90s

Not sure what applications they were thinking of for MP3

Even despite the block structure of OPUS. Only in MP3 I can replace 100 random bytes of files, crop it anywhere, and it will still play as it can on any player. OPUS will shut down immediately and will not be decoded

<:NotLikeThis:805132742819053610>

if the floppy disks had frequent damage, mp3 would be ideal for 1.44 floppy disks, but it wasn't used like that...

maybe DVB-T

The only audio format I'd consider pretty resilient to data corruption, is Opus.

encode the opus file. Changed 1 byte after 1% of the data. A dead file. It will not be decoded

well, that can equally happen with MP3 as well

you can still decode it, though, just not every program can

I understand that it is used in data transmission networks using UDP. But only its stability is achieved by the transmitting soft, if the file has not reached it, it will be known. Not that the file is corrupted.

that sentence doesn't compute...

no, the mp3 will be played, no matter how many bytes you damage, it will skip all the damaged blocks, the opus file does not do this

how do you decode the Opus file?

VLC MPC-HC Audacity ffmpeg

maybe try opusdec directly

Opus is made to be resilient even to high degrees of packet loss

arguably, that’s somewhat at odds with compression

when you compress data, you want to make the best use of the bytes you do have

and having bytes that don’t matter goes against that

if you destroy part of the data and the image is largely unchanged, what was the data even for?

uh oh

I big tests in 2020 and 2022. But now I don't see any problems with playing corrupted Opus files

interesting

Literally the same files that I was doing tests with back then

Ideally, error correction...

I think that’s probably best left to another layer

I don't think so

I think stuff like fec can be implemented in container level without too much issue

PLC techniques afaik needs to be at a format level tho no?

If you really integrate it into a lossy media format, you can even use it to ensure graceful degradatation in case of severe data corruption, and have overall a lot more control over the whole thing.

the more interesting thing is that opus Seamlessly combines the package between (a broken file on top)

You'll probably have fun playing with the ` --packet-loss n Simulate n % random packet loss` parameter of opusdec

less quality... I can reduce the amount of data if I compress harder. Having received 10% of the void instead of the data that was previously useful. But if I just mess up 1 byte I will lose all the data.

To make Opus really resilient to packet loss, you have to encode using this parameter: ` --expect-loss n Set expected packet loss in percent (default: 0)`

Harms efficiency in case of no packet loss, of course.

I know. And I've never been able to check it

the smart way is to use another layer, but make it capable of using information from that outer layer regarding the data. That still requires designing the codec with that in mind, but it's more flexible since you can swap out the error correction and detection layer without re-encoding.

In general, I did not understand something, I found the opus sound that I compressed in 2020, I found ffmpeg 2020. So... just removed the damaged packages, it works as it should...

<:monkaMega:809252622900789269> dementia already now

Good point. Ideally the protection data can be easily removed later.

okay, I had to go back to the time when I opened my first AV1 file

in 2020, I was experimenting with compressing music on broke paper, then I couldn't encode OPUS files. But I was able to do it with MP3 and, accordingly, adPCM

from what I see, as I understood, since I compressed the block size to 60 milliseconds, no one OPUS block could be restored due to constant linear damage

adPCM

I don't remember which one, some specific codec for mobile networks

well yes

It sounds funny

<:monkaMega:809252622900789269> sounds creppy

https://github.com/libjxl/libjxl/issues/3712

<:megapog:816773962884972565>

considering that jpeg XL can have up to 8 layers of progressive decoding, this would be very useful

by the way, the last time I had to make my fork for 8 layers of progressive encoding, is this still not in the encoding settings?

well nope

about jpegli. are conversions to xyb colorspace normally lossless or is there some unavoidable color shifting that precedes lossy compression?

context: xnviewmp updated its jpegli support and its now a togglabe option in export in place of whatever it used before (mozjpg?) but enabling conversion to xyb colorspace there results in perceptible color shifting even at 99% quality

how wide-gamut is the image? jpegli’s XYB might use a range that only contains sRGB

since unlike jxl which uses float, jpegli is constrained by jpeg’s 0-1 range and some decision has to be made as to which “raw” XYB values that range can encompass

png with alpha, 32bit, mostly reddish, colorspace managed to monitor. doesnt seem its an edge case though but limited to use of xyb in export

dev had subsampling options switched around earlier so i was wondering if it wasnt another implementation oopsie somewhere

do you know what the original colour space was? Adobe RGB and P3 both have more colourful reds than sRGB

every image checked color shifts with xyb toggled for jpegli export in xnviewmp 1.8.2

guessing its an integration issue

What subsampling are you using? Because as mentioned the other day, XYB uses different settings to normal

XYB using weird chroma settings is a bug :(

Do they look noticeably darker? It's probably something to do with the color profile.

I notice it makes images darker too, but I think it's probably because without XYB it doesn't attach a color profile so it renders differently.

The XYB chroma subsampling only affects the blue channel

but yeah, the behavior is inconsistent, I made an issue about it

jpegli+xyb export reports odd icc-header and icc_profile (Profile CMM Type, profile creator = Unknown (jxl ) ) and (chromatic adaptation). not sure what to think of that. could such export actually require being decoded using jpegli ?

XYB JPEGs do not require jpegli to be decoded correctly all they need is proper ICC color management handling

https://github.com/libjxl/libjxl/issues/2284

> Now if jpegli is going to produce a new kind of exotic JPEG that we cannot recompress in JXL, then I think we are shooting ourselves in the foot. > > Are there any benchmarks that give insight into the advantage of subsampling the B component in XYB? If this is a good idea, then why didn't we do it in jxl too? Unless there is a huge advantage to it, I would suggest to stop producing bitstreams in jpegli that cannot be represented in jxl. > The consensus is that we should change jpegli not to subsample the B component. > This has been fixed at some point so I'm closing the issue. So... It was fixed at some point, and then reverted?

Seems to work for me, do you have a file where the lossless transcode fails?

What kind of input file will trigger the xyb conversion?

https://github.com/libjxl/libjxl/issues/3605#issuecomment-2132435084

xyb jpegs from cjpegli fail to transcode becuase they are implictly chroma subsampled

which is odd, since the default for yuv is 444

keep in mind these are the defaults for the CLI, the API defaults to yuv 420

Specifically blue channel RGB chroma subsampled, which isn't in JXL spec

According to some testing I've done, 4:4:4 xyb scores higher than chroma subsampled xyb in most quality ranges

I'll post metric numbers soon

And yes, it does look better to the human eye as well

Here are ssimulacra2 scores on a 2914 x 2186 photograph at the same bpp: ``` YUV ------- 80.532 YUV-noaq -- 80.562 XYB ------- 81.672 XYB-noaq -- 81.740 XYB444 ---- 82.007 XYB444-noaq 82.305 ``` noaq = `--noadaptive_quantization`

Interestingly, disabling adaptive quantization (it's on by default) was the more noticeable improvement

It tended to preserve fine detail better

Well... Yeah... Not cutting the image in half does tend to help the quality a bit. It's done for filesize

https://research.google/blog/hdr-photo-editing-with-machine-learning/

good news, they figured out how to do editing of hdr images with gain maps properly

...by just using AI to figure out how the gain map should be modified

lol

another idea, just scrap the gainmaps lmao

neat concept, if it's actually decent quality this would certainly make gainmaps worth using since running AI tonemapping locally just wouldn't make sense

Jpegli-related https://github.com/immich-app/immich/issues/13603

is that an issue with jpegli, immich's integration or the jpgs generated by that phone?

i recall some phones create OEM-specific variants of whats essentially ultrahdr (non-standard jpeg with gainmap), could that be an issue for decoding and conversions?

Xiaomi phones have a HDR option in the camera settings.

Left a comment linking to https://github.com/libjxl/libjxl/issues/3882

how would tail data cause thumbnails for jpeg/ultrahdr to not generate in immich ?

The sample images the OP linked in the thread were converting fine using cjpegli in my case. Has anyone else tried?

https://mkcqb.dynv6.net:666/$/nR3Yo https://mkcqb.dynv6.net:666/$/wwHPb https://mkcqb.dynv6.net:666/$/d94Op https://mkcqb.dynv6.net:666/$/wQiib

My point was just that phones are creating invalid files

i wonder, sounds like itd be a massive issue. whats special about those files that causes breakage? uhdr and rebranded derivatives are supposed to degrade gracefully and return a dimmed base jpeg no matter the stack

https://youtube.com/shorts/T8dVdye559E?si=tZYVcR6PHVHiBpDb

Justice for my boy webp

<@238552565619359744> while not entirely unexpected, seeing APNGs smaller than AVIF is always funny

Thanks again for the link

Already comparing them ey? haha

Gotta try out new things xd

https://svt-av1-psy.com/avif/

I see you noticed the new stuff we've been adding recently 😄

we still have work left to do, but we'll get there

Just wondering with SVT AV1 PSY what preset does it beat 265/vp9?

well, it really depends on multiple factors (as of right now)

but usually, if you stick to the low to mid-high quality range, with preset 4 or slower, that usually beats 265 and almost always beats vp9

What about high fidelity / visually lossless?

that's still x265's territory

Huh, what preset can you do real time?

I can only do preset 8 and above

preset 13 😂

I'm using a macbook air

Ah, I’m using MacBook Pro

What about preset -2?

P -2 is used for meme 8MB encodes in practice

Like an entire movie in 8MB?

it's like 5 FPM on mine lol

an entire movie will most likely be unwatchable

Even just audio

but a 4-min music video is doable

You get 10 Kbps for an 100 minute movie at 8 MB

Imagine that level of compression

You would need to use like Speex 2kbps

Codec 2 anyone?

yeah, getting anything recognizable out of it with video will require going beyond DCT-based codecs

Lossy modular?

🤷‍♂️

AI video compression?

Sorry I said the buzzword

something that can reconstruct the entire Shrek movie from just a detailed transcript 😂

Just from the title!

lookup tables go brr

discord sent a notice of copyright infringement in this message, small restricting my account for 2 years

lol

literally 727 days ago message

It seems to be running in test mode Discord global bot (hello YouTube and twitch), which will analyze all existing files, music and videos that have ever been uploaded to discord in general, for copyright infringement

<:discord:1294466779686371368>

oh boy, my account is fucked then lmao

now ?

not yet

but i have uploaded hundreds of songs etc.

Some are saying it's only if the message is reported

I'd guess it's only if the content itself is reported

I should be fine then xd

Same lmao. The amount of FLACs I've shared <:KekDog:884736660376535040>

<:discord:1294466779686371368> going 1984 NOW

fr lmao

And the amount of mega compressed movies <:KekDog:884736660376535040>

Maybe I should delete some once I get home XD

I wonder what database it would use

how obscure content needs to be to escape this

would they have, say, https://discord.com/channels/794206087879852103/806898911091753051/1068628871735418901 ?

what does small restricting do?

I have absolutely no idea.

I looked at other restrictions on the Internet, it usually says something like that you can send a maximum of 1 file per hour or something. That you can't add friends, for example. Etc. I have absolutely nothing written

"You broke the rules" ok and <:galaxybrain:821831336372338729> <:discord:1294466779686371368>

on support.discord https://support.discord.com/hc/en-us/articles/18210965981847-Discord-Warning-System#h_01HD4SVGAFKAQKYVGEBHS14HSQ like this

I have no idea what I was limited for, and why, and why the message was 2 years ago, and why it was limited for 2 years ahead. It looks like - oh yes, well, the classic of "gigacorporations"

you are limited, we don't owe you anything, you can't appeal against it (we won't read your appeal, and in general, we are not a court, and we don't have a deadline for considering the request) 👉 <:discord:1294466779686371368>

Oh, it's much better, discord responds with a bot. A non-real person writes that, you violated, and we deleted the content. like https://www.reddit.com/r/discordsucks/comments/lbzdbs/my_experience_with_discord_support_positive_ending/

It turns out that discord does not have technical support, the Clyde bot responds to any message — https://www.reddit.com/r/BannedFromDiscord/comments/1evq6em/screw_you_clyde_and_screw_discord — https://www.reddit.com/r/BannedFromDiscord/comments/1eyt0um/theyve_locked_this_ticket_lol_might_as_well_make — https://www.reddit.com/r/BannedFromDiscord/comments/1f2esly/definitely_helped — https://www.reddit.com/r/BannedFromDiscord/comments/1f7yv1m/ehh_may_become_the_last_update_of_this_situation — https://www.reddit.com/r/BannedFromDiscord/comments/1g14sgn/what_should_i_do — https://www.reddit.com/r/BannedFromDiscord/comments/1g5ea3y/bruh_im_so_confused — https://www.reddit.com/r/BannedFromDiscord/comments/1g81i3v/not_deleted_but_ai_cant_find_my_acc — https://www.reddit.com/r/BannedFromDiscord/comments/1db9evy/i_got_suspended_may_29th_and_i_didnt_do_anything — https://www.reddit.com/r/BannedFromDiscord/comments/1fkhus6/literally_did_nothing — https://www.reddit.com/r/Kinguin/comments/1g2sbdk/nitro_1_year_subscription_code

<:This:805404376658739230> sorry but <#806898911091753051> resting today

https://pbs.twimg.com/media/GanplpNWQAA2wcV?format=png

Now? it's been that way ever since it was first conceived by a US intel agency incubator fund :)

(I don't know if discord was born from US spy agencies, but it sure behaves and quacks like a duck)

discord is too trash for it to be us spycrap

These comparisons put JXL to shame. It looks almost as bad compared to AVIF as webp looks to JXL

and webp looks about as bad as JPEG

Actually in the minecraft screenshot JXL looks even worse than webp... wow, that's terrible.

losing to webp lossy should never happen...

Something is seriously wrong if that happens

Bad testing methodology?

Not sure how to read this but this is what they say they did https://svt-av1-psy.com/avif/methodology/index.html

To be fair there are a few regions of the minecraft image where jxl looks better or almost tied but I would rank the jxl as overall worse than the webp

Obviously the only conclusion is JPEG XL sucks and adoption is a waste of time /s

truth

` cjxl input output.jxl -q XX -e 10` it says

AVIF sure needs lots of arbitrary parameters lol

Basically libjxl is extremely bad compared to other codecs at preserving grit and sharp edges.

That’s not a good sign if you have to pass 4-10 advanced parameters to get good results

And it's not a problem with jxl inherently, just a problem with the libjxl encoder

Yeah, video codecs tend to have way too many pointless knobs

and bad defaults

Can jpeg xl photon noise be change around the image or is it full on the image

Like can I apply noise to the top left but non to bottom right

noise is per-frame, so it would be possible with frame blending

but it will also limit the range of epf and gaborish

or... how about having noise-only empty frame and blend it onto main image

<#1278292301038227489>

<:megapog:816773962884972565>

BTW, this will change soon 😎

Is noise only on the RGB channels? If so you could have RGB noise with a modular alpha as a “mask” and then using a blending option

What are the supported blending options?

You already get RB noise if you use the photon option in cjxl while set to lossless. Seems to be assuming the colorspace is always XYB or something

<:Thonk:805904896879493180> https://github.com/AOMediaCodec/av1-avif/releases/tag/v1.2.0-rc

I think that the most salient new thing from the 1.2.0 spec is that one-frame AVIFs no longer have to have the `still_picture` bit turned on in order to be conformant

it's enforced in previous versions, but actual implementations didn't care

What does that bit do?

from what I know from the spec, it's a semantic bit that tells the decoder that the payload must only have one frame (as opposed to one or more frames if the bit is off)

turning the bit on also allows enabling `reduced_still_picture_header`, which saves a few bytes by not signaling some syntax elements

the amount of white space change in this is annoying

oh hey, they formally adopted the hack that we came up with lol > The [=AV1 Image File Format=] supports progressive image decoding through layered images.

too much of a pain to keep looking at the diff files, but here is the diff link for anyone curious https://github.com/AOMediaCodec/av1-avif/compare/v1.1.0...v1.2.0-rc wish github made comparisons more easy to do via ui

Ah maybe you need more than one frame to signal layered images? Or not?

im fairly sure each layer == a frame

https://aomediacodec.github.io/av1-avif/#layered-items

looks like AVIF redefines how frames are output (from AV1 behavior), so stuff like progressive decoding can work

It's not quite progressive decoding, more like a preview frame, right?

(which is a mechanism we also have in jxl but never really use since it's just wasted bytes if you have actual progressive anyway)

iirc yes

The thing with spec changes like this is that it makes it a bit hard to define what exactly it means for something to implement avif. E.g. if a browser says they accept `image/avif`, then what does that actually mean? Can you send them animated avif which is arbitrary av1? (early versions of the spec did not allow that) Will it support 'progressive' decoding? If the spec can have these kind of relatively big changes, it becomes a bit problematic to assign the an IANA media type to something that is a bit of a moving target...

(of course proprietary vendor-specific formats like PSD do that all the time but they only get vendor media types)

speaking of AVIF being a moving target... https://github.com/AOMediaCodec/av1-avif/issues/129 https://github.com/AOMediaCodec/libavif/issues/2077

image/avif.v1 ? 🤷‍♂️

yeah, all 'passes' are independent of each other, so there's some overhead in practice

and in the future there will also be AVIFs with AV2 payloads

that stuff will be more confusing than arithmetic coded JPEGs if they don't define a new type or something

the good thing with AVIF though is that newer versions of the standard tend to relax constraints that weren't really needed in practice, either because decoders weren't effectively affected at doing their job of displaying the image/sequence, or (with new features like progressive decoding) experiences can gracefully degrade if the decoder doesn't support them

I do really hope AV2 AVIFs will get like, a new type or something

I don't think the average person sees AVIFs like MP4s or MKVs

I bet their first instinct will be "this shit is broken/corrupted" instead "oh, I can't view it here", and it's understandable

they shoulda done av1f av2f etc.

😂 no one is ready, neither for av2 nor for avif2. no one is ready to switch to it, not desire , not from a technical point of view. these codecs are dead, no one wants to switch to its. unfortunately, this is the end of human civilization and progress. jpeg xl is the end of all. and over the last 6 years of observation, I am only becoming more convinced of this.

Google has recently been recognized as a monopoly. If the division of the company begins, then there will be no further promotion of this format. and why is it needed if there is a jxl <:logo:829708783336816671> in a fair fight jxl will win

How do I make lossless animated AVIF files?

You can use ffmpeg or avifenc. Handbrake will likely work I assume

What would be the command for them?

What is your input? a range of images or a video?

video

`ffmpeg -i INPUT -crf 0 -cpu-used 4 OUTPUT.avif`

I convert a video into a lossless animated AVIF file, but now I can't play it on Windows Photos or XnView MP. Irfanview only plays it as a still image, and when I put it on a web browser it didn't play all the frames. How do I fix that? Here is the command I used: `ffmpeg -i input.mkv -vf format=yuv444p10le -c:v libaom-av1 -crf 0 -cpu-used 4 out.avif`

Converting to AVIF creates a thumbnail of the videostream as the first stream. You'll have to check if you can change the videotrack in the viewers you've tried

Execute `ffprobe` on the AVIF file you've created and have a look at the lower section

WebP may give you better results than AVIF btw, even with lossless

To create a lossless WebP you can compare to the previously created AVIF: `ffmpeg -i INPUT -lossless 1 -loop 0 "OUTPUT.webp"`

<@386612331288723469> this is my site. I’m actually a big JXL fan so nothing was done to intentionally hamper JXL performance here. The Minecraft image is kinda rough but the ocean image looks quite good with JXL imo. All e10 btw

That’s only with aomenc (currently - this is changing soon). SVT-AV1-PSY just requires Tune 4 and that’s it

Do u have any sample images that favor JXL?

The ocean image looks better than the other codecs except avif is clearly miles ahead of jxl in the ocean image.

By the same amount webp is behind jxl

I’ll take that as a compliment - Tune 4 in its current state was a lot of work and we are proud of the results

Hopefully, some of what we did to make AVIF good can be ported to libjxl, but I think libjxl already does a lot right

You helped to tune the avif encoder? I wish you could help make libjxl less embarrassing too lol 😂

I know the jxl format has a lot more potential than the current state of libjxl

Yeah, <@297955493698076672> and I implemented everything that makes Tune 4 special - alongside <@138804598365224961> and <@321486891079696385>

I’m the maintainer of SVT-AV1-PSY

No offense to the people who made libjxl and provided it to everyone for free. There are just some extremely competitive tools and codecs out there and I really want jxl to succeed and have tools available that are competitive with other existing encoders

To be fair libjxl has been facing some regressions lately

I think the focus should be on lossless or very high fidelity

we took inspiration from libjxl and jpegli in our implementation, and one of our key goals was to defeat some of AVIF's embarassing perceptual weaknesses that didn't show up in metrics like extreme smoothing. In a lot of cases, I prefer jxl results visually over aomenc avif

And resistance to generation loss

I think "low quality mode" should be a separate "are you sure" type feature

anything higher than distance 4 is locked behind `--allow_expert_options` 😅

And I really think there should be some spectral based noise shaping algorithm that pushes distortion into the higher frequencies, like an audio codec would

A noise normalization feature might be able to help with that

I don't know too much about libjxl

So low-frequency artifacts should be avoided at all costs and artifacts are made to resemble blue noise as much as possible

Uniform, uncorrelated blue noise that is easy for humans to "see through"

Ever since 0.9, libjxl has had a serious blurring problem, it also generates too much noise in the wrong places: https://discord.com/channels/794206087879852103/1278292301038227489/1297641128333410404

I wonder, has anyone plotted the rd-curves comparing all major versions of libjxl?

Probably very dumb question, but this isn’t affecting lossless is it?

Lossless has only been improving due to the fact that the metrics are purely objective

JXL is best. Without JXL, a lot of AV1 tools would still be awful.

ssimu2 and 🧈 FTW.

Also, some of the algos used in svt-av1-psy were inspired by libjxl, vorbis and x264.

<:This:805404376658739230> Unironically, JXL pushed avif to be as good as it is right now

I wonder if the reason no one has tried using perceptual noise shaping for image codecs is because it would take a massive hit according to objective metric scores (even though it would look obviously better to human eyes and more of the original signal would be objectively retained below the noise floor)

It is possible to preserve signals that are weaker than the strength of distortion and noise if you push the noise to different frequency bands

This noise shaping theory works on visual data too but it's usually applied only to audio

I feel like it's a seriously underutilized thing in image codecs and I think maybe retarded objective metrics are to blame for that

Because the way current metrics are (badly) designed it would heavily penalize noise that's hard to notice and easy to see through, but not punish smudging and smoothing that utterly deletes and obliterates everything beneath it

The other day I was wondering if anyone actually bisected to find when it got so bad, unless it was just a cummulative effect from many other tweaks

I found a few commits actually

gimme a sec

add Theora to the list 😛

libjxl has had some lossy regressions but it's also been improving those regressions so overall it hasn't improved or gotten worse.

But it's kind of infuriating how bad visual metrics are at guiding quality decisions and to think that it could lead to a lot of potential quality gains being left on the table.

Reminds me of the theora ptalarbvorm update that never got released despite major earthshattering improvements in quality

the main issue with improving noise shaping is that it's very hard from the encoder side to tell "good-looking noise" from "bad-looking noise", because it's contingent on underlying edges, contrast, saturation, and even surrounding areas

but it's definitely doable

Good looking noise is uniform, uncorrelated with the original signal, and contains no strong low frequency content

It can be mathematically defined so idk why good metrics haven't been written

Noise shaping is extensively relied on in all audio codecs

There's maybe only 1 more layer of complexity to take into account which is noise masking. If you get noise masking wrong it can undo the good effects of noise shaping

Noise masking is actually much more difficult to get right but it's also something that codecs already pay a lot of attention to and get right because metrics punish bad noise masking hard

https://people.xiph.org/~xiphmont/demo/theora/demo9.html

This update is in their dev branch but was never pushed to release because vp8 caused everyone to immediately lose interest in squeezing more potential out of theora

I think this commit was the decisive moment, Jyrki changed the tuning of butteraugli to be p-norm rather than max norm. https://github.com/libjxl/libjxl/commit/45096d1ea739d81833370fd4713f5b5a70273e24 Yes, I'm aware this is just the benchmark program, but Jyrki based all of his tuning from the stats given to him from `benchmark_xl`. After this commit, it was death by a thousand cuts (or in this case around 12 commits). Most if not all of the psychovisual hand-tuned algorithms were changed to *heavily* favor `pnorm*BPP`. These changes usually came at the expense of lower max norm & ssimulacra2 scores, which I think is really bad idea. Here is another notable commit, this changed how butteraugli is handled internally. https://github.com/libjxl/libjxl/commit/b468126e7d22a42737c7a669adb861c1a5aa6473 I don't like the look of pnorm-3 butteraugli compared to max norm, but the devs say that it's not consistent enough for lower quality images. That's why I choose pnorm-5 (or 7) for my regression testing, it's a good middle ground. Max norm is still the best metric for higher quality images.

What I don't understand is why butteraugli for lower quality images? I thought ssimulacra2 was more accurate in that quality range. Tuning the heuristics for pnorm*BPP makes no sense to me.

Here is what the documentation has to say about max norm and p norm: > Butteraugli Distance (max norm) indicates the maximum psychovisual error in the decoded image. > > Butteraugli p-norm (3-norm) is a similar summary of the psychovisual error, but closer to an average, giving less weight to small low-quality regions. iirc ssimulacra2 is 4-norm.

It's likely that libjxl has been "cheating" on one metric, improving the rd-curve for pnorm at the expense of true psychovisual quality. This is why this problem went under the radar for so long. I think this is a fundamentally flawed approach. Ideally we would tune everything by eye. But it can't be automated. If we must use metrics, I think we should take a page from the video codec world and check against multiple metrics (at least 3). It's easy to cheat on one metric, but cheating on 3 is much more unlikely. I think with this methodology we could catch regressions before it's too late. So that we don't end up in the situation we are in now.

What is a norm quantity

something to do with statistics https://en.wikipedia.org/wiki/Norm_(mathematics)

it's beyond me 😅 I have no idea what L2, p-norm 3 or any of that stuff actually means. <@794205442175402004> could you please explain this in layman terms?

I know the encoder has been tuned since then, but what if we just changed it back and see what happens...

can absolutely confirm here

I do want to try it at some point. Reverting all the changes and compiling from main to see what happens

you can go ahead and compile from [my libavif fork](https://github.com/gianni-rosato/libavif) if you want, and the build script `ezbuild.sh` will help u out

That's why everyone says to never tune using metrics. See that massive improvement in theora quality on that demo page? Metrics think that's a regression when it's actually a ridiculously massive improvement.

Tuning an image codec by automation without confirming the results with your eye is a really classic example of what not to do

You always, every time, always end up overfitting

Overfitting to a metric that is really naive and loves destructive smears and blobs all over that are impossible to see through

Never ever automate psycho

I get this error compiling on windows: $ ./ezbuild.sh git ✔ cmake ✔ clang X I already have clang installed via Visual Studio, is there another way to do it?

<@167354761270525953> I got this message now:

`[libwebp_anim @ 000001f7cc51c200] Using libwebp for YUV-to-RGB conversion. You may want to consider passing in RGB instead for lossless encoding.`

.sh is for macOS and Linux, .ps1 is for Windows

brush says hello

Basically the 1-norm is just taking the average of the error heat map (so on an image with an easy background, the score will be inflated a lot). The max-norm (inf-norm) is just taking the maximum (so the single worst region will determine the entire score). Norms in between do something in between, e.g. 2-norm is doing root-mean-square on the error heat map, 3-norm is cuberoot-mean-cube, etc. Higher norm asymptotically converges to taking the max.

In butteraugli, what is called p-norm is actually a mix of p, 2p and 4p norms, iirc. So the default 3-norm is a weighted sum of 3, 6 and 12 norms.

That's just a warning, safe to ignore

So the higher the number, the more it tends towards punishing killer samples?

What is Brush?

higher norm means the score is focusing more on "worst-case", lower norm means it's more "average-case"

Actually, we do tune using metrics. The trick is using multiple metrics and understanding why they agree/disagree with each other if their results differ.

PSNR/RMSE is basically the 2-norm of the absolute error, typical SSIM is not that different from MSE but includes some basic modeling of contrast masking

MS-SSIM is SSIM done at different scales (not just 1:1 like SSIM but also 1:2, 1:4, 1:8, 1:16) so it can "see" issues in the low-frequency signal

SSIM / MS-SSIM are usually color-blind, applied only on luma (Y). Sometimes they are also applied on chroma channels but that doesn't really make a lot of sense, since the SSIM formula includes some modeling of the fact that we don't perceive light linearly, so when applied to chroma components it will turn out to make the assumption that chroma errors in the greens are worse than those in the pinks, if applied to CbCr.

ssimulacra2 is basically MS-SSIM but: - SSIM formula corrected so it works for perceptual colorspaces and can be applied to chroma too - applied to XYB instead of just Y, so it's not color-blind - downscales done in linear light and using scales up to 1:32 - both the 1-norm and 4-norm (as opposed to SSIM/MS-SSIM which use just the 1-norm) - additional asymmetric error maps besides the SSIM one: one that measures blurring artifacts (distorted is smooth where original is not) and one that measures ringing/banding/blocking/etc artifacts (original is smooth where distorted is not), also with 1-norm and 4-norm - weighted sum of 108 sub-scores (six scales, three components, three error maps, two norms, `6*3*3*2=108`) where the weights were tuned using a mix of four datasets with subjective scores: CID22, TID2013, KADID-10k and KonFiG

Your last statement on color perception is very interesting. On the subject of metrics, I never did an analysis on metric accuracy for the best implementation of MS-SSIM vs ssimulacra2.

The original SSIM formula includes effectively a gamma 2 adjustment. Then it typically gets applied to Y with the sRGB transfer function, so roughly gamma 2.2. So basically it ends up doing gamma adjustment twice, in a way, assuming humans perceive light with gamma 4.2

Probably this double-correction mistake went unnoticed because the 'real' perceptual function is actually more logarithmic and is better approximated with a higher gamma exponent than what sRGB uses, so gamma 4.2 is not really worse than gamma 2.2

I almost wonder if max norm is fundamentally flawed... Because on max 0.8 is clearly better overall, while p-norm that's meant to average seems to just make the entire image worse instead of only leaving certain spots degraded like you'd expect. And the VarDCT blocks you showed in <#1278292301038227489> seem to just be more granular overall, having smaller blocks with lower qualities each instead of large blocks with more bits. Or maybe that's the issue, now the encoder is overcompensating in large smooth areas when it should be on fine details

<:Stonks:806137886726553651> <:galaxybrain:821831336372338729> best ```py def image_difference(img1, img2): # by DZgas diff = sum(map(lambda x, y: abs(x - y), img1.getdata(), img2.getdata())) return diff ```

We won't know for sure without more testing. I just prefer the "look" max norm favors, even if it's not accurate. I just hate blurring and max norm heavily punishes blurring and favors sharper outputs. I also wonder what would a ssimulacra2 tuned JXL look like...

Well, pixel difference is the best way that there is. But I understand why PSNR and SSIM are used. Because the pixel difference do incredibly terribly slow. Although in 2024 it could probably be used for images now.

I tried butteraugli for AV1. It's unbearably slow. The difference with SSIM is almost invisible to the eye, it is there, but not worth it.

using butteraugli for AV1 is actually like using a placebo preset, but it is added to the current preset

Assuming at it's core, it's 'just' using the difference heatmaps, it wouldn't be too hard to swap to running ssimulacra2 instead for a rough idea of what it looks like

I'm calling it here, my hypothesis is that tuning for ssimulacra2 would fix the color desaturation issues. It really likes color accuracy compared to butteraugli.

Or rather, the heatmap is what changes the quantization and decides on VarDCT blocks (roughly), so changing the heatmap to Ssimulacra2 *should* mean a more accurate distribution... Probably

It'd mean a massive memory hit, so only for a test, but I think I'm right....

Current libjxl is already quite ram hungry 😅

Should we call the fork "ram-eater"?

Throw it in e11 xD

https://github.com/reubeno/brush

Woah

Why not zsh ?😁

So basically none of the metrics take any spectral analysis into account

Missed opportunity to call it Krush

Since bash is basically just a (slow, inefficient) clone of ksh

What does spectral analysis mean in this context?

Low frequency errors are not punished worse than high frequency errors like they should be. And if certain spectra are completely obliterated and wiped away due to smearing and blurring none of the metrics care

High frequency signals have no discernible shape and are easy to see through. Low frequency signals can change the shape and form of what's underneath them and alter the image content

But that's not taken into account at all in any of the metrics, which is why they are so naive and don't care about obliterated detail

It's still a bad idea without confirming with your eyes... all metrics are naive in a similar way right now, in particular all of them think it's great when fine detail in low contrast areas get obliterated.

Tuning without looking will 100% result in wasted time, regressions, and overfitting to a really really bad model

It will result in low contrast areas looking really bizarre and horrible but in a way that get really good scores according to the metrics

Like the regressions that were really noticeable in those TF2 images in that bug report on the libjxl bug tracker

<#1278292301038227489>

It's funny because I think Jyrki was double checking with his eyes... I think it happened for multiple reasons: * He didn't double check every change he made. (I think most commits were unchecked) * He relied solely on 1 metric ~~(not even the best one)~~ * The changes usually caused regressions in other metrics (which were ignored) * I think his dataset was just too small/limited (photographic images only) * Jyrki is just one person

I feel like having a 'jxl community' dataset to test against would prevent future problems. It would contain difficult images from various sources: screenshots, artwork, comics, etc.

You know what? This server should host a biyearly jxl poll, in which we rate if the lossy changes look better in a blind study 😂.

We could setup some community crowd-sourced subjective eval. More ground truth to validate against gives better metrics in the long run.

If I recall, it was mentioned of doing something similar once AIC Part 3 is done

The hard part is designing a scientifically sound double-blind tester web app

And being able to recognize valid test data vs bogus responses

It would be easy to submit nonsense into the crowdsourced data

Some judges just have better eyes or judgement...

And the instructions the judges are given can also skew the results. It should be focused on "which image best represents or is most similar to X" and not "which looks better"

Coming up with a clever methodology to get good information sounds like the most challenging part

Maybe the instructions should be posed as, "which image is different" or "order these images by similarity"

Like a challenge for people to notice the difference between two images. Maybe with a limited amount of time, like 10 or 15 seconds, to determine which image is compressed. With a few trick questions thrown in for sanity checking.

The quality of the data matters a lot and careful methodology has a big impact on what you get

For the spam problem you can possible: require login only take advice if multiple people agree timer (you must look at the image for at least X seconds) maybe ban if detected pressing either only the A or only the B

At least for the first few polls, keep it locally here in the server, I feel like we know what to look for

<:This:805404376658739230>

AIC-3 has a lot of mechanisms to deal with the problem of unreliable participants, when doing Mechanical Turk crowdsourcing we have to reject ~50% of the responses. Besides trap questions, the main mechanism to detect bad participants is to just make the experiment symmetric and ask every question twice, once for A vs B and once for B vs A. Then you can calculate the consistency of the replies. E.g. line clickers who always select the left answer will get very bad consistency,.

https://gregbenzphotography.com/hdr-photos/cats-vs-dogs-hdr-gain-map-test/

images of what it looks like + the actual gain map here: <https://x.com/MishaalRahman/status/1851281393398993159>

This kind of gain map trickery is why they’re not the best system

Agreed. It's a nice gimmick for doing HDR image delivery in a way that gracefully degrades, but it's a mess if you want to use images not just for delivery use cases but also as a source of truth in authoring workflows.

Since the question is: what is this image, actually? Is it a cat? Is it a dog? Is it both?

Is it a bird? Is it a plane? No! It's 2 JPEGs glued together!

'hdr doesnt make files bigger' isnt quite the flex people think when youre actually still using bloated huge jpgs full of compression artifacts - twice larger than an equivalent hdr jxl thats still higher quality

But... It does make it bigger, it's literally 2 files in 1

JXL HDR can actually end up smaller than normal, because of no dithering or banding

Oh that reminds me....

With the kind of gain maps they seem to use in Android cameras, the gain map doesn't add that many bytes: it's 4x downsampled and single-component. That also means that the HDR image suffers from weird artifacts and is quite low quality compared to what you can get with a gain map that is not downsampled and has 3 components — but if you do that then of course you do end up with a gain map that is about as big as the SDR image

I know I was concerned about JPEGs traditionally poor performance in dark areas of images, but I suppose at 4x downsampling it gets smoothed out anyway and is high quality from the camera

I think this is to be expected, Vendors haven't had a lot of hands on time with gainmaps to do any tuning with them. so they are probably following whatever algorithm that google is using upstream

I sympathise with the desire to have control over the tone-mapped version, but it seems to me that one might as well just store two full images instead of this attempt at encoding a transform between the two

Is the only benefit to UltraHDR is that you can still view them on older devices?

pretty much, as far as I understand

correct indeed.

Can’t Google with their infinite industry control push for mass adoption of an actual HDR format (AVIF, JPEG XL) instead of just sticking 2 JPEGs together and calling it HDR

rest of industry can force a paradigm change by integrating jxl into their own apps - those completely bypass browsers and can end serving even more efficient versions of the same content than whats served to browsers

cdns serve apps and games too, not just websites

no because google actually cares about legacy users

Google has realized that if you want to actually transform and push to a new format, the format has to be compatible. We have learned this time and time and time again with various different things.

If you completely break compatibility, people will not adopt it.

The analog to digital conversion showcased this well, monitors that tried to go just digital DVI all failed, DVI dual was made because you need buffer when going from analog to digital. Web services all fallback to formats too.

true, although if the concatenated secondary JPEG requires new processing anyway, might as well make it a JXL or something

The idea is that if you don’t support ultra hdr it gracefully falls back on sdr

indeed, I don't think the format of the gainmap matters a lot, it's probably done to simply processing, but how that works in reality... I doubt it makes much of a difference. they could even use AVIF, but I guess AVIF is slightly too new for supported android devices though

Oh nvm I didn’t understand the argument

I’m guessing just simplification

Couldn’t Google adopt jpeg xt then?

does anyone use jpeg xt?

They could have made people

ehhh, Google is not so all powerful

Ultra HDR is basically doing what JPEG XT is meant for

As in instead of making their own thing they could have pushed for the already existing solution

I guess maybe they wanted an open standard

oh, if jpeg xt is not an open standard, it would have been an instant no for google

I don't know much about the specifics of jpeg xt so I cant comment on that front, how is authoring for it, how it works, I know nada

Jpeg XT has two ways of doing gain maps, one is royalty free, the other has some Dolby patents on it iirc

Anyway it is irrelevant now, the wheel has been reinvented already

There are many ways to deal with legacy, graceful degradation is one way but it's not the method I prefer since it tends to lead to mostly degradation and little grace.

You could also do jpegli in HLG or some variant of HLG and then you can do actual HDR in a single jpeg where the degradations are nicer imo: applications that don't do HDR will show some reasonable-ish SDR image, applications that do support HDR but decode jpeg to 8-bit get some banding, applications that get it right get a good image, at the cost of spending more bytes than with modern codecs.

Or you could just use content negotiation and/or media queries and send an SDR jpeg to legacy user agents and a HDR jxl to user agents that support it.

content negotiation works in many cases, but when it comes to something like generic file sharing it's not really possible.

True. I guess you can just offer a choice to share with "maximum compatibility" (to borrow the Photoshop save dialog terminology) or not.

doesn't that still require ICC profile support to look reasonable in SDR? assuming the use of bt.2020 primaries in that case, i don't really see an advantage over doing it the way libjxl does with the tone mapping built into the icc profile (while using PQ for the actual HDR image)

You could do a variant of HLG with sRGB or P3 primaries so the misinterpretation as sRGB is not too bad

In any case if the application does not do correct color management then colors will generally not be correct.

i guess that's where ultra hdr shines, as it really "just works" in anything that can display SDR images

I guess my position is that gain maps can be a useful approach for interop in delivery and interchange (though I think there are better options), but it's a bad idea as an authoring format, including making it a capture format. You are basically sacrificing fidelity of the HDR image for the sake of interop, which I don't think is a good idea for a capture format.

yeah, that works quite well

also, there might be a case to be made that with HLG, a tone-mapping ICC LUT has “less work” to do

(haven’t ‘formally’ verified to what extent that is true, but that seems intuitively plausible)

or how iPhone does it

https://github.com/strukturag/libheif/releases/tag/v1.19.0

finally opened this issue lol https://github.com/MegaByte/jpegultrascan/issues/9

What are some tools or image viewers for Windows that can support playing lossless animated AVIF files? What are some Android apps that support playing them on mobile devices?

Genuine question, why do you keep insisting on using lossless AVIF? What does AVIF offer that animated WebP doesn't?

maybe higher bit depths?

also why used Animated AVIF when you can use... AV1

It's just AV1 in a convenient wrapper.

Well, convenient for the user.

wouldn't a WebM make more sense for something like a video?

It's just that lossless AVIF sucks so bad I struggle to understand why you would even bother using it. Animated WebP or JXL would be sooooo much more efficient

I can just mux the entire Shrek movie encoded in AV1 into an AVIF and make this come true

Make an ultra loose patches encoder and have the same frame of Shrek for the entire movie

I mean, I do agree that avif is a shitty container, but seperating "image" from "video" from a user perspective is important

too bad mki doesn't exist, we have mkv and mka, T.T

dont forget mks for subtitles!

since mkv supports storing random data streams as attachments, I propose the new file archive format, MKD! It uses the Matroska container structure but stores all the files using data/attachment streams

Lossless AVIF is much smaller

Compared to what?

Compared to WebP? no way

inb4 It's not actually **lossless** AVIF

inb4 someone starts spitting benchmarks

>me when I svt-av1

What is your source, and what programs are you converting it to an animated AVIF?

Can av1 even do lossless video?

only the reference encoder, libaom can

I meant when using libaom-av1

Are you converting from a video or from a sequence of images?

does it matter? [Hmm](https://cdn.discordapp.com/emojis/1113499891314991275.webp?size=48&quality=lossless&name=Hmm)

converting from a video

absolutely. most recommend to go back to the original animation/video of the highest quality as source, because then a *video* codec gives the best results. that however is really a workaround valid for all codecs and doesnt speak of any merit of avif as replacement for gif, just an inefficient substitute

if your source is an animated **gif**, webp and jxl (libjxl) give better results (lossless with smaller size animation than avif)

some animated gifs can still be better as AVIFs I have found

also apng

Optimized APNG to JXL is the best you can get without writing your own tools

On a vaguely related note, has there ever been a GIF encoder that uses blue noise dithering instead of FS?

I ended up reading this, which is the basis for Blender's Blue Noise sampler, along with it's use in the game INSIDE to eliminate banding https://psychopath.io/post/2022_07_24_owen_scrambling_based_dithered_blue_noise_sampling

So it's obvious it can be relatively fast with sample textures or lookup tables, yet I can't find anything that uses it for GIFs instead of Floys Steinberg that seemingly everyone on earth uses

show me an example

I don't have any off hand but if I come across one I'll share

And by better, you mean smaller file sizes, right?

I have some examples, all lossless of course. ``` 34M 123898350_ugoira1920x1080 - Frieren.apng 20M 123898350_ugoira1920x1080 - Frieren_s2.avif 26M 123898350_ugoira1920x1080 - Frieren_e10.jxl 29M 123898350_ugoira1920x1080 - Frieren.webp ---- 98M ZhEb2N3.gif 84M ZhEb2N3.gif.e10.jxl 78M ZhEb2N3.apng 77M ZhEb2N3_opt.apng 78M ZhEb2N3.apng 90M ZhEb2N3.apng.e7.jxl 76M ZhEb2N3.apng.e10.jxl 81M ZhEb2N3.webp 261M ZhEb2N3_s0_crf0.avif ---- 28K 1253399939300458618.apng 232K 1253399939300458618.avif 32K 1253399939300458618.gif 28K 1253399939300458618.gif.apng 36K 1253399939300458618.gif.jxl 28K 1253399939300458618.gif.webp 36K 1253399939300458618.jxl 28K 1253399939300458618.webp 1,7M 1434205067_1434202320284.apng 6,7M 1434205067_1434202320284.avif 2,0M 1434205067_1434202320284.gif 1,9M 1434205067_1434202320284.jxl 1,6M 1434205067_1434202320284.webp 1,5M 1434863403_cats-playing-pong-with-laser-pointer.gif.pagespeed.ce.5wpl8OkWnNe1JovLmnRn.apng 5,5M 1434863403_cats-playing-pong-with-laser-pointer.gif.pagespeed.ce.5wpl8OkWnNe1JovLmnRn.avif 1,7M 1434863403_cats-playing-pong-with-laser-pointer.gif.pagespeed.ce.5wpl8OkWnNe1JovLmnRn.gif 1,8M 1434863403_cats-playing-pong-with-laser-pointer.gif.pagespeed.ce.5wpl8OkWnNe1JovLmnRn.jxl 1,5M 1434863403_cats-playing-pong-with-laser-pointer.gif.pagespeed.ce.5wpl8OkWnNe1JovLmnRn.webp 2,4M 1434987698_asho.apng 11M 1434987698_asho.avif 2,1M 1434987698_asho.gif 2,9M 1434987698_asho.jxl 2,4M 1434987698_asho.webp 1,7M 1434993945_tumblr_museefhGA91qdlh1io1_r1_400.apng 6,6M 1434993945_tumblr_museefhGA91qdlh1io1_r1_400.avif 1,9M 1434993945_tumblr_museefhGA91qdlh1io1_r1_400.gif 2,2M 1434993945_tumblr_museefhGA91qdlh1io1_r1_400.jxl 1,7M 1434993945_tumblr_museefhGA91qdlh1io1_r1_400.webp ``` WebP is often a lot smaller than AVIF, with AVIF only sometimes being smaller

No idea where the three duplicates at the bottom are from

Most of the JXL in this example are derived of the GIF instead of APNG, so they are quite unoptimized

Are the WEBP, JXL, and AVIF files in these examples lossless?

> all lossless of course.

Yes, the lossless files are lossless

Let me get a video and convert it to lossless AVIF to see what happens

How about these? https://www.mediafire.com/file/oi7pv6hbjcpsap9/Test_Files_and_Videos.7z/file

Just get the mkv files and ignore the rest.

And keep the resolution at 1920x1080

ooohhhhh, the input videos are lossy as well, that explains a lot. There is a subtle grain throughout the frames making it hard for animated image formats like webp/png/jxl that rely on interframe optimization.

so that begs the question, why even bother keeping it lossless if the input is already lossy?

In fact I bet a light denoising algorithm would make it compress wayyyyyyyy better

That's what we tried to ask on the 4 Reddit posts they made, now replaced with 2 more Reddit posts, one just linking to the other and asking for comments