JPEG XL

I see

It's just bad practice to turn a lossy input into a lossless one. `Test1.mkv` was 4.2 MB and a lossless AVIF equivalent was around 75 MB. That's like having a quality 80 jpeg and saving as a png or something.

Let's just say I can't upload my videos publicly, and can only do so if they are images.

Alright then, I recommend adding a little bit of denoising and lossy compression, 75+ MB for a 5 second animation is HUGE. Not to mention my computer struggled to decode & play the animated AVIF, it was constantly stuttering/hitching.

Here is a simple example, a high quality lossy avif. It looks visually indistinguishable to me. And the best part? It's only 1.8 MB

And it doesn't stutter or hitch like the lossless versions either!

My ffmpeg input:`ffmpeg -i Test1.mkv -c:v libaom-av1 -cpu-used 4 -crf 18 Test1.avif`

But now do you see my point now on why I chose AVIF?

yes, because the input is already video (and you can't upload that video), so in this case AVIF would actually be the best format to use

Still though, please use lossy compression, that's what AV1 codec excels at. Using lossless with AVIF is like trying to use a car as a boat, it can work... albeit very poorly. 😅

I mean, it helps that AVIF is a video :P

That's the main reason yeah

AVIF is really good at lossy compression, I have no problem with it. When you said lossless AVIF that was a red flag to me. Even with a video input choosing AVIF lossless over lossy is a bad idea imo.

Do you mean Apple live photos?

They mean like animated AVIF but HEVC

Hello.

https://aomedia-review.googlesource.com/c/aom/+/194662

😮 me jumpscare

Could libjxl plagiarise, ahem I mean *take inspiration from* some of the improvements from AV1/AVIF?

No? Most of the improvements came from JXL. We just worked a lot on them. If you find anything that we do that is better, just take it and shove it into libjxl.

the major ideas are in libjxl already, but some of the 'knobs' could be adjusted to be closer PSY's in the low to mid-high fidelity range

it's just a matter of someone having time to sit down and tune this or that feature

Genetic algorithm?

code is efficient already. any talk of quality regarding libjxl tends to be related to libjxl (more specifically, the jxl authors' release of libjxl) prioritizing real visual quality rather than need 3rdparty builds with different priorities and tuning

Libjxl low-medium quality tune version that auto activates at d>10?

probably something closer to more optimal transform size selection

personally i think efforts should be adaptive to resolution so smaller images are smaller and larger images drop 1 effort in order to maintain encoding performance, with e7 the default clearly marked and promoted as *recommended* for any purpose that is not fast capturing/screenshoting (e1-3)

That could make sense

somebody posted some images where 0.8 performs better than 0.11, and it was partly due to a different selection of transform sizes

Like larger images don’t get as many butterfly iterations

Butter-something thing

Can never remember the spelling

when it doubt, just call it 🧈

but it's butteraugli

Butter that encoding! 🛬

I do kinda wish the Var in VarDCT was used under effort 5, since a lot of the time 3 or 4 is recommended/used instead so it's just 8x8 blocks anyway

Then again, they tend to all be the same size at high quality anyway

JPEG XL: 8x8 DCT and the Weighted predictor

Do you know if it's possible to add a comment tag to a PNG file?

Kinda

I wanted to try the latest version of jpegli but it seems the Windows builds are missing the executables. https://github.com/google/jpegli/actions/runs/11703639102

I suppose the jpegli version bundled with libjxl is old. Are there newer Windows executables for jpegli available?

I think it's time to fire all the developers. they are all too lazy to make build

<:Google:806629068803932281> 👍

How did you do that? It’s not natively possible on Windows.

Third party tools

Tweakpng

Thanks

I wish more things supported the `svgz` extension

like lots of things know it's an svg but ignore the fact that it is GZipped

so this happens

To me it looks like some error, neither `jpegli-x64-windows` nor `jpegli-x86-windows` includes jpegli binaries.

hm... maybe https://artifacts.lucaversari.it/libjxl/libjxl/latest/ jxl-x64-windows-static.zip

After jpegli was moved to the separate repository, I'm not sure if the copy included in libjxl is being updated.

An interesting question. I think it's better if you ask this in <#848189884614705192>

Encoding of the original GIF file in 2024 (for example, for discord) *I searched all kinds of sites that compress GIFs, all kinds of forums with programs, with keys for ffmpeg encoding... even one piece of shit on the output* My compression algorithm from video to gif using terminal FFMPEG in 3 steps ``` ffmpeg -i VIDEO.mp4 -vf "scale=480:270:flags=spline+full_chroma_inp,mpdecimate=hi=64*50:lo=64*5:frac=1,nlmeans=1:5:5:5:5,hqdn3d=luma_spatial=3:chroma_spatial=2:chroma_tmp=20:luma_tmp=15,nlmeans=1:5:5:5:5,mpdecimate" temp.apng ``` ``` ffmpeg -i temp.apng -vf "palettegen" palette.png ``` ``` ffmpeg -i temp.apng -i palette.png -filter_complex "paletteuse=dither=none" out.gif ``` Explanation of the primary processing: gif can have any frame rate on each frame, but for some reason no one uses it anywhere at all. For the same reason, there is no FPS setting anywhere in my parameters. It is dynamic, completely dependent on the processing of the input video. —mpdecimate filter removes frames that are very similar to each other, without encoding any gaps. `The strength of skipping similar frames is adjusted by the parameter lo=64*5 (64*4 skips less, 64*6 skips more)` — nlmeans filter smooths out low-frequency spatial noise, which causes GIFs to make noise, losing information no matter what. `In the line 1:5:5:5:5 - the first digit indicates the suppression force, the rest indicate the length in pixels for noise analysis. To speed up coding, you can set 1:3:3:3:3` —HQDN3D filter smooths out temporal noise when gradients move from one to the other.` The power is adjustable chroma_tmp=power (color) luma_tmp=power (brightness)`

gifski is a great tool based on ffmpeg

Doesn't take video input though

it does

isnt the whole point of gifski to take videos?

thought so

Rather, it's disabled by default so there's no binaries for it https://github.com/ImageOptim/gifski?tab=readme-ov-file#with-built-in-video-support

Ah, I did not expereince that due to installing it from my repos

also flexiGIF is quite good (but slow) for GIF optimization

very very slow indeed

I'm having regrets

but I don't understand why it is so slow bc it is advertized as LZW/GIF optimization (and in GIF it should only do LZW), and GIF optimization is like 10+ times slower than for a `.z` file

how does it compare to gifsicle?

It's basically an LZW brute forcer, that's why it's unbearably slow

gifsicle can usually get within 99.5% of the size while being orders of magnitudes faster

Also fun fact, flexiGIF doesn't do palette sorting, so the optimal workflow is to run the GIF through gifsicle *then* through flexiFGIF

And then after 20 years you have the perfectly optimized GIF!

https://tenor.com/view/skeleton-falling-gif-27355771

As far as I'm aware flexiGIF is purely an LZW optimizer, trying to squeeze a few more bits out of the existing data. It does however have the defaults set to be a brute force solution too...

`'flexi.gif' is 18270 bytes smaller than 'Test.gif' (621919 vs 640189 bytes) => you saved 2.854%. Finished after 2651.14 seconds.`

```wintime -- cjxl -d 0 Test.gif Test.jxl -e 10 JPEG XL encoder v0.11.0 0185fcd [AVX2,SSE2] Encoding [Modular, lossless, effort: 10] Compressed to 512.7 kB (0.736 bpp/frame). 365 x 159, 0.002 MP/s [0.00, 0.00], , 1 reps, 16 threads. PageFaultCount: 289985 PeakWorkingSetSize: 42.15 MiB QuotaPeakPagedPoolUsage: 33.38 KiB QuotaPeakNonPagedPoolUsage: 7.828 KiB PeakPagefileUsage: 40.56 MiB Creation time 2024/11/11 00:47:30.790 Exit time 2024/11/11 00:48:05.561 Wall time: 0 days, 00:00:34.770 (34.77 seconds) User time: 0 days, 00:00:01.140 (1.14 seconds) Kernel time: 0 days, 00:00:34.640 (34.64 seconds) wintime -- cjxl -d 0 Test.apng Test.jxl -e 10 JPEG XL encoder v0.11.0 0185fcd [AVX2,SSE2] Encoding [Modular, lossless, effort: 10] Compressed to 505.2 kB (0.725 bpp/frame). 365 x 159, 0.002 MP/s [0.00, 0.00], , 1 reps, 16 threads. PageFaultCount: 330741 PeakWorkingSetSize: 66.79 MiB QuotaPeakPagedPoolUsage: 33.38 KiB QuotaPeakNonPagedPoolUsage: 7.961 KiB PeakPagefileUsage: 64.43 MiB Creation time 2024/11/11 00:49:28.310 Exit time 2024/11/11 00:50:02.331 Wall time: 0 days, 00:00:34.021 (34.02 seconds) User time: 0 days, 00:00:01.000 (1.00 seconds) Kernel time: 0 days, 00:00:33.859 (33.86 seconds)``` That's... Interesting... The GIF is run though gifsicle and flexiGIF, but apparently `gif2apng` found something smaller that JXL could take advantage of with some extra memory

Neat, a new toy. I will try that on gifs using local palettes. Those failed conversion to apng.

Probably it doesn't make much difference but when converting gif to jxl, I would add `--keep_invisible=0`

hehe I really searched a lot for different optimizers for quite many formats <:KekDog:805390049033191445>

what if you optimize your apng ? `apngopt`

Gif with local palettes convert to 24 bpp apng so the apng, even after optimization, is about the same size or bigger than the original gif optimized with gifsicle.

Alright, so the first step is `gifsicle -O3` and then `flexigif`. Do you have any recommended options?

I remove "metadata" of GIFs with `gifsicle -o out.gif -O3 --no-comments --no-extensions --no-app-extensions --no-names input.gif` for flexiGIF typically use a simple `flexiGIF -p`, (tbh I didn't do heavy testing, and on the forum thread there are many benchmarks), if you want faster (you usually want, otherwise fucking slow), just increase `-a=xxx` value only gifsicle *then* flexiGIF seems useful also if you want better compression with flexiGIF, use this 2-step-lookahead patch: https://encode.su/threads/3008-flexiGIF-lossless-GIF-LZW-optimization?p=82323&viewfull=1#post82323

Do the patches apply to gif too or only to Z?

Some of my gifs were probably lossy compressed. I will try flexigif on those as well.

yeah, just look at that table <:KekDog:805390049033191445> <https://encode.su/threads/3008-flexiGIF-lossless-GIF-LZW-optimization?p=68853&viewfull=1#post68853>

gif2apng is part of the apngopt toolset, so it already is

ohhhh <:FeelsReadingMan:808827102278451241>

I actually did gif -> jxl, then jxl -> apng and then apngopt, so apngopt could start fresh from unoptimized 32 bpp file.

I went down a rabbit hole and now I'm finding out MV-HEVC finally exists and has for a while https://developer.apple.com/documentation/avfoundation/media_reading_and_writing/converting_side-by-side_3d_video_to_multiview_hevc_and_spatial_video

Always wondered when someone would take advantage of stereoscopic similarities

Is it normal that flexigif processes a single 1000x1000 pixels frame for several hours?

Although I tried a different gif of similar size and it went much faster.

Yes

It took 50 minutes for a 5 second long 240x320 GIF

Seems weird the first frame of the animation is not yet finished after 16 hours.

It’s the zopfli -999999 of GIFs

Sure, but I tested other gifs and a single frame usually takes under an hour.

One thing about flexigif is it's not multithreadiing, so I can run multiple instances with different files simultaneously and see the differences immediately.

Great, a few gifs went well but now I have 5 gifs stuck on the first frame. Maybe I'll try without `-p` 😄

you should try `-p -a=2048` or so perhaps?

which should be 2048 times faster (ig)

it's funny to see the early history of FLIF: https://github.com/sipa/JIF/commits/master/

I will try it. I don't feel like spending 24 hours on each frame.

I haven't tried the patches but I suppose those would be even slower.

yeah the two-step-lookahead is slower

How did you come to `-a=2048`? Is it some magic value?

I just like powers of 2

`-a=x --alignment=x blocks starts at multiples of x (default is -a=1 => best compression but may be slow)`

but again I'm perhaps saying shit as I didn't do heavy testings for GIFs, almost only for `.z` files

If I understand correctly, this will split the image to segments of x pixels. Wouldn't it make more sense to use a width of the image as x?

mmmh aren't different GIF frames different blocks ?

Yes, each frame is processed separately.

oh yeah I misread your last message so yeah I guess x=width would be the best, but who knows [⠀](https://cdn.discordapp.com/emojis/586168843781668876.webp?size=48&quality=lossless&name=AkkoShrug)

On the other hand, optimized frames may be different width.

right, there are frame that are just differences from the previous one

I will try just without any options at all and see what happens.

default is `-a=1` which is fucking slow <:KekDog:805390049033191445>

I can see that 😄 Anyhow, it could do multithreading pretty easily just by processing multiple frames at once.

A single thread on my Ryzen 5 1600 won't cut it.

In comparison, `cjxl -d 0 -e 11` is actually quite manageable. I found just one image, where cjxl fell apart, consuming the entire system memory, resulting in the system being extremely slow and cjxl just sitting there doing seemingly no progress.

we need animated jpeg with alpha <:megapog:816773962884972565> || 👉 <:JPEG_XL:805860709039865937> ||

Animated JNG?

MNG can do that iirc

is MNG generalised APNG?

Yeah it's the more complicated thing that includes JNG, it never really gained traction though

personally still think JNG should have superseded JFIF

JPEG 1 file formats are a mess

JFIF was first so it became the de facto standard even though it has clear limitations

JPEG 2000 is based on BMFF right?

The committee made SPIFF and tried to push that, but it failed to get any traction and was recently finally retired

Camera vendors meanwhile made Exif

(which is not just metadata but also a JPEG file format)

Or ISOBMFF is based on what was done for J2K, not sure which was first there

I though BMFF was based on QuickTime MOV / MPEG MP4

The basic box structure (4 byte length, 4 byte identifier, payload) is used by many things, PNG chunks are pretty much the same thing (except they also have a CRC and can't be nested).

Many pieces of software still produce illegal Exif/JFIF hybrids.

I think lossy WebP if fine for animation, still provides better color fidelity than gif.

Yeah

I mean, for those short loops made from video.

100%

GIF is kinda bad

And there's lossless WebP for actual animation.

speaking of gifs anyways, is there any actually good way to consistently convert gif to jxl losslessly without bad file sizes?

Problem 1 is gif is already ridiculously low quality

yeah using a lossy encoding isn't really on the table [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&name=av1_dogelol)

It very depends on the specific case.

well the key issue is setting something automated, so it kinda needs to work even in the worst case, or at least, not break badly enough to be actively largely detrimental

some kind of ML-based undithering system?

If the gif was compressed with gifsicle in lossy mode, that's it. No other lossless codec could improve it.

[Hmm](https://cdn.discordapp.com/emojis/1113499891314991275.webp?size=48&name=Hmm)

My problem of converting gif to jxl is it's not easy to convert it back to gif.

Especially if local palettes were used in the gif.

GIF reconstruction data when

Apng is seriously crippled it doesn't support local palettes.

"Seriously crippled" is an exaggeration

It's not that bad

I don't really care about that myself, any gif I have I don't really care about going back, not for my use anyways

I only encode with reconstruction data because im too lazy to figure out how to not do so, and still consistently save space lol

For gifs I usually just convert it to an Apng then an animated jxl

is that actually beneficial? [Hmm](https://cdn.discordapp.com/emojis/1113499891314991275.webp?size=48&name=Hmm)

Yes

Apngopt or gif2apng has additional inter-frame optimizations that jxl can take advantage of

I typically see an improvement in my testing

guess ill try that, I plan on doing a setup where just all of my images, baring a few exceptions, get converted to jxl since literally everything I use supports jxl now

Apng is terribly inefficient compressing gifs which used local palettes. Coversion to 24 bpp looses the advantage of better compression algorithm.

WebP doesn't case about palettes and works well but it's tricky to convert animated WebP to anything else.

The difference isn't that big, only like 5-10%.

Can’t JXL then do palettes as well

Not optimal but not terrible

Yeah jxl can do palettes

Compared to what apng can do with only a global palette, the difference is significant.

hmm, not bad, ill try the gif thing I guess ```ps 69.96 MiB png-jxl 70.55 MiB gif-jxl 88.45 MiB gif 143.29 MiB png 611.07 MiB jpg-jxl 777.94 MiB jpg ```

> Apngopt or apng2gif on average which one is better?

They use the same algorithms I think

Made by the same people

icic, also assuming I *don't* care about reconstruction info, is there anything that is realistically better then `cjxl -e 9 -d 0 ` that is worth adding?

Besides discarding metadata I think you're good

I know `allow_jpeg_reconstruction` exists but I don't know if it's realistically any good

If I do archiving, I care about some way to restore the original format. Most of my gifs are actually hand drawn animation.

Sorry it's gif2apng, not apng2gif

i dont think GIF or PNG reconstruction is a thing

Typo 😭

well, ill run this through and see how it goes

I'm not that strict. In most cases, I don't care about palette order and such, it just has to be pixel perfect. djxl is sufficient for conversion to apng, then it can be run through apngopt.

But I'm also still using gif because according to my testing, gif with local palettes is often more efficient than the equivalent apng.

```ps 610.93 MiB jpg-jxl-no 611.07 MiB jpg-jxl 777.94 MiB jpg ```

well that wasnt really worth the re-encode lol

The reconstruction metadata is usually very small

Like < 1 kB

I would suggest just keeping it, it’s not worth explicitly removing it and you never know you might want to convert it back some time

Yeah you can gain more if you try different efforts and cjxl versions.

well it just be how it be, everything can't be free

Older cjxl versions seem to do better job compressing jpegs. But it's a thing I'm going to look into more deeply.

Also not sure how but higher efforts still benefit jpeg transcoding

Argh regressions!

well I saved enough space to make it worth it anyways

I admit I was exaggerating a little that apng is terribly inefficient. Gif with local palettes and apng containing the same pixel data are about the same file size. So it's not worth to convert such gif to apng, because you gain nothing, but using apng is not a disaster. However, if I would like to create an optimized file, I would have to go with gif.

That's kind of an edge case where gif is *only* optimized when there are local palettes, which are quite rare. Most gifs in the wild are global palettes, so re-encoding to apng would be worth it imo

what am I saying, just use jxl for better savings 😂 <:JXL:805850130203934781>

I'm not sure which animation software were the artist's using but I have several gifs with local palettes.

I would use jxl if I could.

But yeah, if there's only global palette, apng works great, unless it's lossy lzw, then even webp or jxl can't do anything with it.

It's highly content dependent, not to mention I was doing a lot of my testing on 0.8. Maybe 0.11 made lossless gif encoding a lot better. 😅

Probably better off just going straight from gif -> gifsicle -> jxl

yeah probably, I am just making some scripts for "inplace" transcoding so the more simple the better anyways

ofc inplace is not really true since I am changing the file extensions, but I am deleting the source files when done

I should really convert my local archive of jpegs to jxl

literally no downside idk why I haven't done it sooner

yeah, best part is, you don't even need to hash them [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&name=av1_dogelol)

after images, I need to develop some kind of script that will tell me if it's worth transcoding a video to av1 lmao, dunno how best to go about this yet tho

That's a tall order for a simple script, I don't know if even it's even possible.

you can get a "good estimation" using the basics like bitrate for the file size

I see, like resolution divided by bitrate? It super content dependent, but at least you have an upper threshold for super high quality videos worth re-encoding

yeah, it would be a "sortiing" scheme into "likely good candidate" "possibly good candidate" "possibly bad candidate" "likely bad candidate"

for instance, any h264 or h265 video with "BDMV" like bitrates are almost always going to be good candidates for av1, so thats a safe assumption

you're basically making a video quality metric estimator at this point 😭

yeah

[av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&name=av1_dogelol)

at the very least, it will make it a lot easier to sort through which videos are even worth checking in the first place

true, it acts like a sieve, filtering out bad candidates

`--brotli_effort=11` compresses the metadata and reconstruction data a bit more, for the GIFs/APNG `--keep_invisible=0` might help, not sure

I've had some fun with Telegram exports and then converting all the images in-place to JXL, opens fine in Waterfox since it's just HTML, but naturally the files are unreadable with the wrong extensions otherwise

`--keep_invisible=0` seems to be trivial, it just zeroes out the fully transparent pixels. ECT for example, can take more advantage of fully transparent pixels in png. It will fill them with different colors so the bytes become zero after filtering.

That's what I assumed cjxl did

It doesn't make any difference if I use --keep_invisible=0 on a png where fully transparent pixels are already black.

But it's useful for pngs, that were not optimized beforehand. Photoshop, for example fills transparent pixels with some random colors.

Don't even match the png filtering.

In gif, the transparent eras are solid color, so blackening them won't make much difference but it will tidy them up.

Anyhow, it seems the global palette in gif is optional, I come across a gif, where every frame had it's own palette and the global palette was completely missing. I wonder what the decoder is supposed to do if there's no palette at all. I suppose most decoders would render undefined colors as black.

It seems that gif without global palette is considered bad practice. The first thing gifsicle did is moving palette from the first frame and make it the global palette.

yeah im just gonna keep the gifs, it is smaller, but not really smaller enough to sacrafice the proper animation support ``` 338.50 MiB total vs 235.66 MiB total ```

also I encounted some gifs that magick reports mismatching hash, wonder why that happens

Would you mind sharing some of them?

if I come across them again, I already punted them back to my phone

Alright, thanks

Testing what flexigif does with lossy LZW. Doesn't look very promising.

In a way, both are based on IFF: https://en.m.wikipedia.org/wiki/Interchange_File_Format

I think technically such files are still valid [one of the two], just not valid as both Exif and JFIF since those two formats both require their APP marker to be the very first one after the SOI marker.

JFIF seems to be completely unnecessary now. I just remove APP0 markers and the jpegs work without issues.

Does this diagram look OK? It's of course a bit of an oversimplification. Am I being fair to the various formats? Did I forget something?

i would personally put HEIC a bit lower, like maybe cutting in on the top of JPEG

I put it that high up since it's basically only being used as a proprietary capture format, no really suitable for interchange.

I'm actually using BMP a lot.

Perhaps add `Windows bitmap` as application specific.

Windows bitmap is the native format of the most popular OS used on PC and it's specific to Windows.

im not gonna lie. even with the description at the bottom, it's hard to actually understand what the graphic is showcasing

Hm, right, I didn't include the uncompressed formats like BMP and PPM. I guess they're similar to TIFF.

<@794205442175402004> can you send the raw file? i want to try something with it

https://docs.google.com/drawings/d/13dR6dwFMw5l8BKQ53B_4Mi9LaTT0EWN44-ErHL--BRM/edit?usp=sharing

If you have suggestions on how to make it more clear / less confusing, or if you can pinpoint what about it is hard to understand, that would help.

for instance, it's not exactly clear to me why HEIC is above JXL, TIFF, OpenEXR should be sitting too at the top too since but HEIC is still above it it seems like HEIC would be "before" said formats in an authoring workflow, which is really only true in some specific instances, of which similar instances are true with TIFF, and likely in the future JXL

also the graphic makes it look like HEIC isn't used for delivery

you don't really take a photo in EXR

no, but you can use it for creation, specifically texture creation stuff, EXR makes a great "save file" for that stuff since it has dedicated channel alpha, linear, no clamping etc

also many cameras do save in a TIFF file format

i don't feel like 3d modelling really fits in with this

it does explicitly say creation

and does explicitly have 3d modeling in there

I haven’t seen many non-raw TIFFs saved by cameras

is top to bottom meant to be like the creative workflow?

like you start at the top and go down?

Is it? The main way I see it being used is as a capture format. Safari does support it now (just like it does/did support J2K), so I guess it could be a delivery format too, but given the patent situation I consider it not really having a chance of becoming sufficiently interoperable to be considered anything other than a proprietary capture format.

i have NEVER seen a HEIC being served on the web excluding downloading iPhone originals

More or less, though the idea of interchange is that you also go up again from the middle.

yeah, most images shared among iphones are heic, and I know that my samsung phone captures in default, and shares HEIC with my mothers pixel too when we send images back and forth

when you say "DCP" do you mean digital cinema package? that is very much a delivery format

web is only one specific thing though.

correction: i have never seen a HEIC being used in any situation outside phone photography

Canon cameras can produce PQ HEICs

argh, fine

I think other brands produce HLG HEICs but off the top of my head, I don’t remember which ones

maybe Sony and Panasonic

i have never seen a HEIC being used in any situation outside photo capture

while true, I don't think this graphic comes across as accurate if it excludes phone usage considering how prevalent phones are

I see DCP more as an interchange format, of course it is "delivery" to theaters but I mean delivery more in an end-user/consumer/web sense. Maybe that should be clarified in the diagram.

from what I understand, some people then connect their camera to a TV via HDMI and display those HEICs

I agree that it sounds rather niche, though

i think its current location is fine as after the camera takes the photo it will save it as either a HEIC or a DNG (or a JPEG)

also, crucially, the ways that Apple and Canon do HDR-in-HEIC are not mutually compatible

that doesn't mean it's not for delivery though, or else jpeg would need to be moved as well.

or well I think it should rather be extended

https://www.dpreview.com/articles/8980170510/how-hdr-tvs-could-change-your-photography-forever > HEIF/HEIC is a broad standard, and the files from Canon and Apple are not cross-compatible with one another

WHAT

[av1_kekw](https://cdn.discordapp.com/emojis/758892021191934033.webp?size=48&name=av1_kekw)

ok what

\/hj

maybe add avif at the bottom for lower quality

oh right, the graphic also looks like it's intended to be capture on the left, with creation on the right, which is in conflict with the raster < -- > vector

In general Im not sure this graphic format is entirely suitable for what the graphic is trying to convey, and maybe splitting it into 2 related graphics may be better

3 dimensional?

authoring (creating) <-> delivery (showing) capture (real life) <-> creation (cgi, illustration) vector (pixels) <-> raster (geometry)

I was thinking maybe one of those triangle things, or 3 circle venn diagram would work.

what would the points on the triangle be though?

one sec, lemme see if I can remeber how to convery this

is it worth specifing capture vs creation?

i think raster vs vector is probably fine

illustration could go in the middle

I can see that. TIFF can supersede other uncompressed image formats.

However, Windows bitmap is used in capture, creation, intermediate, delivery and display. Pretty much any array of pixels in Windows API is a bitmap.

windows

Yeah I would change this to a three point ven diagram methinks, or even just a two point grid plot graphic. but they are much less visually appealing. But the issue with this original graphics is that it does draw lines where IMO they probably be shouldn't many of these formats like PNG,JPEG,HEIC, and i'm sure jxl soon, are used in capturing software, and also used for delivery. if the top part of the graphic isn't intended to imply a capture to creation scale, I think it how it presents the info should be changed. I think breaking up the "application specific authoring formats" block to allow things like JXL, TIFF, etc should be allowed to scale up.

how about this?

I do like this one more for sure

what is light red

It may also be worth changing AVIF to HEIF (heic, avif, vvc)

though the duplicate nature of heic may be a bit confusing

istg are they making a VVC based HEIF

i bet my life savings that will be dead on arrival except maybe apple

yeah, libheif can already save vvc heif

is VVC actually going anywhere?

yes

To me JPEG XL would just be covering the entire spectrum, apart from SVG, ect for vectors. You can already use it in the camera, Krita was one of the first to support it so digital painting definitely is...

and except PDF / PostScript

although you can add JPEG XL images to SVG / PDF / PostScript files

the horizontal axis is intended to be going from photo / sensor data on the left to artificial / generated on the right, where the left is raster obtained from a sensor, the middle is still raster but not captured by a sensor (digital art, illustrations, screenshots, rendered text, rendered 3D scenes, etc) and the right is pure vector graphics. With a lot of mixed stuff as well, like PDF which can contain photos and text, or GIS stuff like satelite images with road maps and whatever other vector stuff.

It would probably be clearer if it was indicated as 3 stops Left: Photo Middle: Illustration/Mixed Right: Vector

I suddenly remembered that splines get resampled instead of scaled on decode. Or at least used to

?

The JPEG should make a SVG competitor so the entire diagram is just JPEG

Okay apparently I just have very selective memory https://discord.com/channels/794206087879852103/804324493420920833/885888987317817354

not sure how I can add that to the chart in a nice way

tweaked it a bit: didn't really make sense to give j2k a narrower scope than jpeg, also moved XS/HTJ2K more to the left and made DjVu match PDF (just more towards raster)

Can you make avif stretch all the way to j2k?

Also what makes J2K / AVIF /WebP less of a raster format than JPEG

avif and webp are more suitable for non-photographic images than jpeg is, e.g. they both have palette modes, and when doing lossy their artifacts on illustrations generally look more acceptable than those of jpeg

how about this?

J2K also does have some stuff meant to deal better with non-photo, in particular JPEG 2000 Part 17: Extensions for coding of discontinuous media

heic/heif/hevc also encodes vector lines perfectly, like avif. and even better than jpeg xl

raster vector lines yes

JPEG XL vector box when

yeah in terms of artifacts, heic and avif are kind of comparable. The thing is I don't think heic is really used by anyone for illustrations, even if in principle it could.

avif — webp — gif — vector bad line

in general, it's all a waste of time. I would like to make a graph like: vector or flat picture/real photo complexity/simplicity

I don't think that in its current form anyone will be pleased to look at it

that middle region where GIF and DjVu are is not yet vector, it's still raster, just low-color / line drawing / illustration type of graphics

it might be worth redoing. vp9 died a long time ago, aomenc is too un-threaded, svt-av1 is better in all cases

complexity/simplicity is an interesting additional dimension, though it also has various interpretations: do you mean feature complexity? amount of coding tools? computation complexity? the complexity of the image content?

TIFF and PPM are both low-complexity in terms of encode/decode speed (they're uncompressed, or nearly so), but TIFF is very complex in features while PPM is extremely simple.

I think for the average user, only the decoding speed is important.

you can adjust the encoding speed according to the situation. but you won't be able to open a 16k x 16k jpeg xl on your smartphone for $80

dc frame go brrr

av1 cannot be used for resolutions higher than 720p in fact. and considering the progress. This will never happen. This is the last video codec in the history of modern civilization.

<:monkaMega:809252622900789269>

av2 propoganda

to be honest, hevc 4k causes problems, but av1 causes even more problems at 1080p. An interesting fact is that the entire level system was recently removed from svtav1. They can't match.

for delivery, yes, but at the capture/creation side, you do want fast encode too. Can't have a screenshot tool or the "Save" button in an editor freeze your computer for a minute...

most newer formats have some sort of speed tuning, that could affect it

well use jpeg q100 4:4:4 — this is the fastest way to save an image. there is no png and no webp. bmp and png(without compression) are large in size

it even annoys me that there is no alternative, because it is faster than jpeg xl e1

Can’t we make jpeg xl e1 just jpeg q100 + jpeg lossless transcoding ?

by the way. I'm not sure if jpeg is faster than jpeg xl e1 modular

I have never tested the fjxl project.

<:JPEG_XL:805860709039865937>

I saw that it was better than qoi, but I did not compare it with jpeg

Meanwhile crunching a single picture for an entire day.

I'm not in hurry.

so far, libjxl cannot decode a 100k x 100k image. without eat more than 1 GB of memory. It's all boring

I'm pretty sure that jpeg encoding is not faster than libjxl e1 lossless. Decoding is something else though, libjxl decode of modular is not that fast, even though we do have some specialization for the e1 case. But the decode of e1 is inherently a bit slower than the encode. There's some room for improvement in libjxl though — e.g. avoiding the conversion to float32 would probably help.

Even with chunked decoding? (pixel callback, not using the full buffer api)

I'll run the tests soon. as soon as I find a suitable huge image. now I use baseline jpeg, which takes up less memory than jpeg xl

I asked an interesting question in <#848189884614705192>

reading like 256x256 chunk and writing it to the display buffer. either in the output image, as progressive encode png, yes

I know windows at least can use subsampled chroma directly to reduce memory usage

what. No. I'm sure Windows can not work like this

https://learn.microsoft.com/en-us/windows/win32/wic/jpeg-ycbcr-support

funny... but I use 4:4:4 so there is no effect.

I also use XnView and maybe it doesn't support this technology...

I'm not aware of any image viewer that does. Likely too specialised or new for them to make a special case for JPEG, or simply don't know

IrfanView is using just BMP internally, it finally added a full alpha support.

Windows can read JPEG and PNG for some time but bitmap header had to be attached to the images.

But it makes sense to add a native YCbCr support for images. Video cards could do YUV-RGB conversion for decades, but it was only used for video.

They added it, but it tinges my images brown when enabled for some reason

Works fine for me, although the settings may be little confusing.

I wanted to try and compare https://github.com/openapv/openapv which seems to be a prores alternative to jxl, but I couldn't get it working, it would be an interesting comparison

Seems to make JXL images with alpha brown for me unless set to "Apply transparent color to background"

does this make sense?

I am trying to get a bit of an overview on the container stuff, because I think I got some things a bit wrong in the past

in particular, we often say the jxl container is based on ISOBMF, but in reality we're much more like the j2k file format

that is, we use the box structure and we do have the `ftyp` box as an identifier, and when possible we'll reuse existing box definitions like `Exif` and `xml `, but that's about it

we don't implement all of the stuff that is in ISOBMFF, stuff like `idat` and `iloc` and `iinf` and `moov` and `mdat` and all of that stuff

so it's not quite correct to say that the jxl container format is "based on" ISOBMFF, it's rather just using the same box structure and you also have to use the `ftyp` box to identify a jxl file. There's no normative reference to ISOBMFF in the jxl file format spec.

Does it lossy or lossless or both?

looks good to me, though maybe it would be nice to hint that the "image file format" is still a container depending how beginner the target audience is

well the way I see it you have pure container specs (like MIAF), pure codestream specs (like AV1), and then you have image formats (like AVIF) which are a combination of container and payload

HEIF container + HEVC payload codec = HEIC image format

Ignore transparency fails to color manage, with lossy stretching the color image and lossless keeping it's shape Apply transparent color correctly color manages and has no issues Keep alpha is the same as ignore, with bitdepth not increasing to 32bit Apply color, Lossy JXL, Lossless JXL

Have you updated the plugins too?

Irfanview 4.70 and JPEG_XL 4.68, so yes

Opened a JXL image in IfanView and saved as PNG. Converted the same JXL to PNG using djxl. Both PNGs contain the same pixel data. My images are in sRGB, though.

But there's something strange going on with your examples, it looks like red and blue channels are swapped.

An image with transparency, and changing the transparency settings in Irfanview?

Changing the transparency setting changes only the appearance of the background. The more concerning fact is it looks like red and blue channels are being swapped in your image.

Only with the *keep alpha* setting I get the same image as using djxl.

In ISOBMFF how do you know if a box is nested or not?

If the payload of a box is another box then it's nested 🙂

How can you tell if the box should be read as individual values or as a list of other boxes?

If you understand the box type, you know how to interpret the payload. Otherwise you just ignore the box.

GIF has to be pronounced with a hard G otherwise you might confuse it with [JIF (JPEG Interchange Format)](https://en.wikipedia.org/wiki/JPEG#JPEG_files)

That's actually a good point.

Though I don't think many people are aware of JIF. Even JFIF is not very commonly used, it's more common to assume that a JPEG is a JPEG _file_, and when it isn't (like when embedded in a PDF), to call it a "JPEG codestream" or something like that.

In fact, originally FLIF was called JIF (though I wanted to call it JPIF), for Jon's Image Format (or Jon and Pieter), because I didn't know about the JPEG JIF, and I thought the confusion with GIF would be fun 🙂

The JPG file extension was adopted for all JPEG images probably because it has three letters. Back then, file extensions with more than three letter were often being truncated.

blame DOS and 8.3

I also remember `.htm` pages on the web

interestingly, though, it’s been a while since we’ve restricted ourselves to 8 characters for the part before the extension

Even better, names like `mainpa~1.htm`

Short file names are still available in modern Windows and sometimes they're useful, for example in programs that doesn't support unicode file names.

Still to day, I'm not fan of file extensions with more than three letters, especially if the shortened extension would conflict with another file format.

one thing I’ve been wondering is whether there would be concrete drawbacks to using `.c++`

instead of trying to come up with workarounds like `.cpp` and `.cc`

(or worse of all, `.C`)

other annoyances include `.m` for Objective-C and MATLAB, and `.pl` for Prolog and for Perl

Probably a minor annoyance of `.c++` would be that it needs to be urlencoded to `c%2b%2b` when requested it via HTTP

Also probably needs escaping in regexes and shell scripts etc

I don’t personally have a problem with .cpp

.m makes no sense though

It sounds like something for metal shaders

apparently, it comes from “messages”

since message-passing is central to the language

That would be like calling rust files .bc for borrow checked lol

Or .ms for memory safely

There used to be a dozen of image formats using `.pic` extension.

I don't like `.webp` extension. Looks kinda ugly and doesn't have a three letter form. Also, how is WebP supposed to be pronounced?

“Web Pee” I think

By shortening `.avif`, you get `.avi` `.hei` still kinda works, though.

AVIF could probably be AIF, for Av1 Image Format

Apple just gives up and doesn’t even try with their .pages and .keynote files (well, technically folders (well, technically both..?))

I suppose those files won't be usable outside Apple system anyway.

There were even file formats using only numbers in the file extension.

AIF is already an extension for audio

Ah I forgot of it, I have only known of it as aiff

Limiting extension to three letters will bottleneck us at some point if the abbreviations are used to have meaning. But mostly just a remnant of the 8.3 days i think

A1I for AV1 Image would be usable, but it looks kind of stupid, ngl

I think AVIF / WebP is fine as if your system still supports only 3 letter extensions, get a better system

Why does that read like an obscure extension for some obscure video format used in some MS-DOS games?

AVIF is just HEIF using AV1 compression. MKV doesn't need a special file extension for every compression type.

Although in the case of AVIF the distinction is reasonable.

mki when plox

I actually tried that and found out FFV1 is pretty good.

cat.thegreadcodecjpegxlinthisfile

The file name size is still limited to 255 characters <:Stonks:806137886726553651>

not quite, AFAIK https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation#enable-long-paths-in-windows-10-version-1607-and-later

I've been using that flag for a few years and never had an issue

There was severe path length limitation when burning files to CD.

On linux the kernel sadly hard limits to 255 characters for filenames. path length is 255 characters per level

python offers this feature during installation

unrelated, but: https://github.com/LMP88959/Digital-Subband-Video-2/pull/9

asymmetric transform coding using novel subband filter.

asymmetric as in slower encoding, faster decoding

only 1 paper was written about this concept back in 1990 and i havent seen anyone else expand upon it

but now i've implemented my own version of it and added it to the DSV2 video codec

No one complains about .flac having 4 letters because we aren't in the 80's anymore

`.flc` (`.fla` is already taken by Flash)

`.f.lac`

Hey all! libaom has recently gained a new `--tune=ssimulacra2` mode, available on the `main` branch. It's essentially SVT-AV1-PSY's "still picture" tune heuristics and tweaks ported into libaom, developed to improve subjective image quality and consistency, while also increasing SSIMULACRA 2 scores. If you have a project that requires using AVIF, or are just curious about the new tune, give it a try!

(can confirm it is really good)

> subjective image quality and consistency, What does *consistency* mean here? Is it likely to spend more bytes on images that compress well and less bytes on images that compress poorly, given one quality value? (this is my no. 1 problem with AVIF, or with any encoder…). Is there any discussions/docs/writeups/blogposts about this that would spare me testing it blindly?

*Consistency* here means that, at a given quality value, the libaom will spend more bytes on images that are rated lower subjectively (or have lower SSIMULACRA2 scores), and vice versa. The end result is the ability of libaom to hit a quality level more accurately, either within an image (especially if they feature a mix of high-contrast and low-contrast features, or camera noise/film grain), or across images. The libaom changes are hot and fresh, so no formal docs have been written up yet, but the commit development chain can be followed up here: https://aomedia-review.googlesource.com/q/owner:juliobbv@gmail.com. Additionally, almost all of the changes were ported from SVT-AV1-PSY's tune 4, of which we do have some comparisons and documentation: https://svt-av1-psy.com/avif/comparisons/index.html.

Thanks so much! Sounds exciting, will have a read.

Excellent, thank you for reading me!

The most extreme case I've encountered so far between tunes is http://0x0.st/X3On.png. This 36 MP image contains heavy luma and chroma noise. A competent lossy encoder can still get something decent-looking at 2 MB. The SSIM tune (in 4:4:4 chroma subsampling mode) wipes out luma noise almost completely (while overshooting size by 8+%), but the new SSIMULACRA 2 tune retains it much more satisfactorily.

(fun fact: the new tune enables <@179701849576833024>'s qm-psnr distance metric, as it was found to be beneficial for SSIMULACRA 2 by 3-5%, and reduces the occurrence of ugly flat blocks)

Exactly this [here](https://aomedia-review.googlesource.com/c/aom/+/194303)! Since I can only fire one number for quality at thousands of images, I’m finding that, at the same dimensions, those of 6.1KB can easily be 20kb and those of 200kb could easily be 120kb. And the whole thing would look better. And weigh less in total. * nos. totes invented, just to illustrate the point Been looking for it since, like, [forever](https://github.com/GoogleChromeLabs/squoosh/issues/270#issue-380211392).

I was honestly surprised it seemed to do nothing

it might've been the combination of the new optimized QM level picking formula + multi-scale nature of SSIMULACRA 2 actually detecting the psychovisual benefit 🤔

the original QM level formula in libaom's code made zero sense to me

but anyway, it's a perf review time freebie 😛

I don't think it was ever meant to make sense, fwiw

(it might be that I replaced it, in which case it _was_ meant to make sense xD)

yeah, I looked at the commit history and the original formula has pretty much stayed the same ever since its inception in VP10

IIRC the idea was just for it to produce all different QM values to test the en/decoder correctness *shrug*

Now if only they were monkeying with libjxl instead of libaom/svtav1

A format with much more potential, the "next gen alien technology from the future" really...

Not even being facetious there, the format was designed to have very long, long term potential gains deep into the future...

I'd def be down to work on libjxl, the toolset looks fun

I know that transform size selection for mid to low bitrates can be improved

Yes, also there is stuff you can do in block selection that libjxl doesn't even consider, such as using non-naturally aligned blocks (say a 32x32 block starting at coords that are not a multiple of 32) and using blocks larger than 64x64.

🤔 https://gitlab.com/AOMediaCodec/SVT-AV1/-/merge_requests/2336

oh, interesting! so can you have something like, an 8x8, 32x32, 16x16, and 8x8 block along the width axis?

Yes, as long as you don't cross the boundaries of the 256x256 groups

I remember Pascal was talking about the idea in the context of webp2, I think he called it "floating blocks", and we could do that in jxl too without any extra signaling cost or complications for decoding, so we did (even if we weren't using it)

a fun thing this allows you to do is to find an actually-optimal (according to whatever metric you care for) decomposition in 16x8, 8x8 or 8x16 blocks fairly quickly, although I never tried to implement that

(well, you still have to determine all the costs, but once you have those you should be able to find the optimal decomposition quickly)

yeah, you'd have increased chances to align block edges to favorable image features

Lossy modular/squeeze mode, which hasn't even been extensively tested or tuned, already performs just as well or better than default dct lossy mode, so maybe there's even greater potential and simplicity down that route...?

100%, it just takes dev hours

Make -m the default 😂

doing efficient unsqueeze is far from trivial, maybe there's room for improvement in decode speed of lossy modular but it won't be easy. Currently lossy modular is about 3 times as slow to decode as lossy vardct...

Isn’t squeeze really bad for cache performance?

no, it's mostly just hard to do multithreading

well, depends also on how you actually unsqueeze

as it is done today in libjxl, I don't think it's particularly good for cache

if anyone can make stuff insanely fast it's veluca

the simd wizard