JPEG XL

The fast palette detection code from e1 could be generalized a bit and used at the slower settings too

The big difference between e2/3 and e4+ is that e2/3 use fixed MA trees while e4+ uses MA tree learning.

Probably doing either some simpler kind of MA tree learning, or trying a few fixed trees and picking the best could give better speed/density trade-offs

so it's `e4+: try both ClampedGradient and Weighted predictor, try global palette` and `MA tree learning`

Yes. e1 is no tree (technically it's a single-node tree), e2/3 is a fixed tree, and at e4 the tree learning algorithm starts getting used (with the number of decision node properties considered growing as effort grows)

Also: e1 is huffman, e2+ is ANS. For fast encoding, huffman is better...

then why is this `e4+: try both ClampedGradient and Weighted predictor, try global palette` and not `Select and Weighted predictor` (best nonphoto + best photo) because ClampedGradient is faster?

ClampedGradient is also quite good for nonphoto

Select is better for pixel art, I think

(you're making me want to try a fjxl-with-select xD)

Select is similar to png's Paeth

(actually, choosing the best of the two should be relatively fast, no?)

AvgW+N should also be quite good for photo, I think in png it is often the best one for photo

(weighed is better but also much more complicated)

maybe I should just quickly hack together a non-fast fjxl with select, clampedgradient, avgw+n and maybe some more and see if I can find a pareto optimal set of choices

(although I suspect clampedgradient > avgw+n)

what does cwebp -z 0 use? https://pbs.twimg.com/media/FHHLNbMWQAIzXAU?format=jpg&name=large

so on a random rle-friendly nonphoto image select does almost exactly as well as clampedgradient

does anybody have images that they'd think select would be useful for?

<@794205442175402004> I guess

Any pixel art, I would expect. Especially if it has been NN upsampled

some big pixel art?

That fox has weird edges

yeah, and bad alpha cutting

I'm trying the first one of those, it's maybe a 1% difference xD

fox: 0.330bpp clampedgradient -> 0.317bpp select

so that's a bit more

maybe for fast modes it doesn't have much effect <:FeelsSadMan:808221433243107338>

-e 2 is 0.211, -e 4 is 0.1

I wonder what tree it is learning...

ah, the tree is not it, it's just doing good lz77

lz77 is also different for these modes?

e4 does lz77, e1 doesn't

I mean e2+ also does it make sense to use simpler png-like filters, or can they only be faster (and choosing the best one will be slower), but not better?

I think e2 and e3 also don't lz77

clampedgradient is not significantly slower than png filters so...

All predictors are cheap except Weighed

AvgAll is maybe a bit more work than the rest of the simple ones

on this e1 is about 2x better than e2 because good for rle?

though, something weird cjxl latest build with this pr <https://github.com/libjxl/libjxl/pull/2024>

something different for processing palette images for cjxl and fast_lossless? ```png:IHDR.bit-depth-orig: 4 png:IHDR.bit_depth: 4 png:IHDR.color-type-orig: 3 png:IHDR.color_type: 3 (Indexed) png:IHDR.interlace_method: 0 (Not interlaced) png:IHDR.width,height: 1920, 960 png:PLTE.number_colors: 15```

how are you calling cjxl?

it's possible that fjxl read the palette and preserved it with a palette transform and cjxl -e 1 is not

as usual `cjxl -q 100 -e 1 ...`

maybe it's libpng vs lodepng? don't remember what cjxl uses by default

yeah, cjxl uses libpng and fjxl uses lodepng

and yeah, similar results for all other palette pngs

when converted to ppm the fjxl results become worse, similar to -e 1, so needs some palette checks, because this is a very big compression loss, 2-2.5x worse on average for -e 4 the sizes are almost similar for ppm and png

Is the fjxl one actually lossless? Maybe it got confused by the bit_depth 4....

yes, it's lossless

mhhh weird

Do fjxl and cjxl -e 1 use the same fast-lossless effort?

they ought to

Ah, I think palette detection depends on whether input is given as RGB vs RGBA

so, cjxl accepts everything as RGBA?

No, fjxl does, I think.

only if input is PNG

https://github.com/libjxl/libjxl/blob/main/lib/jxl/enc_fast_lossless.cc#L3478

It only does palette detection when input is 8-bit rgba (so it can just use uint32 per pixel)

I think it's needed for all inputs, because the difference is pretty big image from the qoi set

and if the same image as ppm

Yeah palette can make a pretty big difference...

perhaps cjxl e2 would also be better with this palette detection

https://twitter.com/juliangough/status/1611031552472793089?t=JfdOz4XNlqSXGaVKc_mjSQ&s=19 I am very happy that Chrome didn't use this strategy to block jxl. There is basically no way to do anything when lawyers say something ambiguous or say nothing at all, that can put anything in "legal limbo" where nothing at all can be done.

That's how these 'transfer of copyright' contracts often work. They have to put some number there. If you sign, they likely won't even bother to actually pay that 1 dollar (or pound or whatever).

to be accurate, the actual commission was 20k EUR, but this didn't actually cover the copyright assignment (as they never fully signed the contract)

so legally speaking, Gough can't retract one's piece from Minecraft (as the commission was paid) but can put the work into the public domain

which was what Gough has done

the overwhelming majority in HN thought this end poem is only a very small part of the game, which is true (I never saw it by myself, for the record, because I never got into the ender dragon battle...), but that's not the point

no matter whether Gough is an attention seeker or not (in some sense, one is), the situation is quite disheartening

That moment when e 10 fails after a few hours <:Cheems:884736660707901470>

ehhhh

No idea what caused the crash though. The image was only 1920x1080 and I have 32GB RAM, so doubt it didn't have enough RAM

After an hour it was only using 4GB - 5GB

it can't be free because then the contract is not legally enforceable, so it's essentially a placeholder value

👍

https://youtu.be/oba_Y9fNeFg

<:KekDog:805390049033191445>

native decoder or libjxl?

libjxl i'm pretty sure

also why does it say JPEG-XL "decoding" they let you encode jxl files

I think so too, otherwise they would probably say native

related to the fractional frames discussion https://youtu.be/RDHgSbmJvjs

Are video decoders just really fast? A quick and dirty djxl reports 110 MP/s on my laptop (8 core Ryzen 7 PRO 6850U), but a 4K60 video would require 498 MP/s to decode. And an 8K60 requires nearly 2000 MP/s. Am I missing something?

Well hw decode is of course very fast, but also for sw decode: it helps that they typically use yuv420 (so only half the pixel data compared to 444), and that quality is typically quite low so there isn't a lot of entropy decoding needed (compared to doing intra only at high quality)

Yes: that 4k60 video is 4:2:0 <:YEP:808828808127971399>

That makes sense and also partially explains why video based codecs get faster at lower qualities.

Also jpeg is faster at lower quality. Basically the transforms have the same cost regardless of quality, but entropy coding cost is more or less proportional to compressed filesize.

In jxl the effect is less pronounced, I think because 1) higher quality uses more small blocks, which is slightly cheaper than big blocks, 2) lower quality uses more epf iterations, which is slower to decode, 3) entropy decode in jxl is less dependant on quality thanks to hybriduint, which uses more raw bits at higher quality.

I don't think the effect of 3 is particularly pronounced

Of 2 OTOH absolutely yes

In fact, I believe if you disable restoration filters then JXL also becomes significantly faster at lower qualities

Video codec if you watch the video in vertical you get ugly distorted video

Especially using AV1 and open source codecs

EPF is decoder-only though, right? Encode side it doesn't affect speed...

Correct

ooh, thank you. I will have to give this a try

libjxl-based

native decoder is something on Lynne's todo list but she's still busy and hasn't gotten around to it

you'd have to blame jb ig, since encode via libjxl is also supported

also JPEG-XL, not JPEG XL, needs some sort of FAQ, with correct and incorrect naming/spelling

If ffmpeg makes a native decoder I'm sure everyone will start using it instead of libjxl, like how everyone uses dav1d

And I'm sure it will have such ludicrously blazing fast performance that "hardware support" will sound even more like a joke than the running gag it already is

only if it will be better, ffmpeg has its own encoders/decoders which are worse than external ones and which are only used when needed less extra dependencies or for some other reason when external ones have not been linked dav1d is some independent open source decoder, but it is not an ffmpeg decoder, though supported and sponsored by VideoLAN (who are now leading ffmpeg development?)

videolan are not leading FFmpeg development and usually the internal decoders in FFmpeg are faster than the reference implementation, especially cause FFmpeg has the fastest DCT code on the planet right now

typically FFmpeg relies on external libraries to do encoding, but usually uses internal decoders if present

yeah, I didn't mean all of them are worse, but that doesn't automatically make them better, especially encoders for non-exotic formats for VideoLAN this is mostly a thought and a question, because judging from the presentation Jean-Baptiste Kempf seems to be making a lot of quite important decisions for ffmpeg development

Internal decoders aren't always faster than external ones. For example, a recent development in libvpx-vp9 decoding has made it faster than ffvp9 in 8-bit decode.

I hadn't heard about this, huh

also internal hevc decoder, but openhevc was faster (although now they have merged, if I am not confused), but ffh264 is still faster, for opus not sure, but the encoder is bad and etc. maybe integrate fpnge as a fast ffmpeg png encoder? <:FeelsAmazingMan:808826295768449054> then it would be more widely used does ffmpeg even have an internal png encoder/decoder?

yes it has an internal decoder and encoder for png and for apng

opus encoder is experimental, they recommend you use the libopus wrapper, and you can't use teh internal one without explicitly turning on experimental flag

If there's another encoder/decoder for jxl that is doing something better/faster than libjxl, we'll likely want to port that to libjxl 🙂

FFmpeg is LGPLv2 for most of its internal stuff

which could possibly be an issue

hmm, yeah, or maybe dual licensing

I thought that ffmpeg uses libpng and libjpeg-turbo or something like that because everyone uses them, although I haven't really looked into this because I don't use ffmpeg for image manipulation more than as an intermediate for importing/exporting <:Thonk:805904896879493180>

In that case we would either need the author of that fragment of code to also contribute it to libjxl, or we need to reimplement the same thing with new code.

Since libjxl has a BSD license which is more permissive than LGPL...

it would depend on libavutil/tx for its DCT/FFT/etc. code, which is faster than libjxl's DCT code, and it would be written in pure C.

also LGPL

so it would be difficult to port over directly

also FFmpeg tends not to support features like progressive decode (including for things like JPEG that can do it)

it works on the frame-level for those sorts of things

and won't have any kind of render pipeline

so porting the code over without doing render pipeline would be possibly quite difficult

Yes, I agree

If libavutil has a faster dct, we may want to use something like that in libjxl(-tiny)...

I suppose it is faster because it's handwritten simd using integer arithmetic only?

Yes.

we've been thinking of making integer-only decoding paths for a while, but that's significant work

like being written from scratch in C and integrated into libavcodec?

Aren't there any BSD licensed libraries that implement integer only dct?

That would surprise me, but who knows

Everyone tends to have a slightly different definition of integer dct...

It's funky how recently I wanted to add QOI inside the libjxl decoder. And already now libjxl has fjxl which is faster and stronger than QOI

fjxl is cool

maybe djxl could use fpnge to write the decoded output

That would be a good idea. I'm already seeing people measure jxl decode time including the output writing, which is of course just bad benchmarking but with fpnge we could avoid it having too much impact.

(just yesterday I saw someone claim jxl decodes slower than j2k, which was essentially because djxl is slower at encoding a png than the j2k decoder is at writing a bmp)

yeah, that would be cool because djxl does not need a full png encoder like libpng, but fpnge at least needs animation support and jpegli isn't already used for encoding/decoding? not for speed though, but for quality also is jpegli a completely separate codec for now or does jpegli+libjpxl together have a smaller binary size than using separately?

had a quick build and gotta say fjxl is indeed amazing~

> It’s worth noting video images require only eight bits of digital information compared to audio which requires twenty-four bits for full fidelity. <:Thonk:805904896879493180> a very strange claim

8 per channel you mean?

also audio only requires about 16 bits for good dynamic range.

to be clear, that quote is not from me

that it’s 8 per component is indeed one of the problems with it

also, it’s at least 10 for decent dynamic range even with non-linear encoding, and ideally with millions of samples per frame, of which there should be at the very least ~20 per second or ideally even more

it would be pretty cool if images were composed of adequate precision of sampling of all the frequencies actually involved in making colors appear as they are instead of results of 3 types of sensors per pixel but that's not really needed for us as receivers, though would probably aid non-human receivers with different cone configurations

i suspect image format supporting such stuff would be only useful for astronomy purposes where they capture much wider range of frequencies and need to analyze it

https://www.tandfonline.com/doi/abs/10.1080/00387010.2021.1931788?journalCode=lstl20 wow it's actually a thing

sigh, the ISOBMF spec disappeared from the page of publicly available specs, luckily you don't need it for jxl (we don't reference it normatively, we just copied the description of the box format and informatively say it's the same as in ISOBMF, since it's simple enough)

there's still a copy here: https://mpeg.chiariglione.org/standards/mpeg-4/iso-base-media-file-format/text-isoiec-14496-12-5th-edition but that's not the most recent version

really annoying how at ISO sometimes you can somehow get something to be publicly available, but then a new edition gets made and they remove the old one without making the new one publicly available too

the full ISOBMF spec is over 200 pages, twice as long as the jxl spec, but we only really use the box structure itself, which is just half a page to describe, not all the stuff they have in ISOBMF to represent and reference multiple streams and mux them together

I think that's actually better for everyone else, last time we had people thinking JPEG XL is possibly patent encumbered because ISOBMFF is, we only concern about the physical structure itself here so this hopefully reduces an overall misconception

I don't think ISOBMFF is patent-encumbered since generally 'containers' are not assumed to be patentable (though Nokia might disagree with that since they do claim patents on HEIF) and since it is quite old (the first version of mp4 is from 2001 and it was based on QuickTime which dates back to 1991), though of course there are still new editions being made so who knows what the patent status of that is. Just parsing the box format should be fine though — if there would be anything patent-encumbered, it would have to be related to the payload of boxes that have been defined more recently.

I agree so much, but sometimes misconceptions can be only solved by irrational measures...

I'm getting weird behavior with ffmpeg and yt-dlp (YouTube) downloaded videos exporting as libjxl (as I mentioned in <#806898911091753051>): I downloaded Big Buck Bunny at 4K with AV1 encoding via yt-dlp and extracted one frame like so: PNG: `ffmpeg -ss 03:28.367 -i bbb-yt.mp4 -frames:v 1 03-bbbyt-ffmpeg.png` JXL: `ffmpeg -ss 03:28.367 -i bbb-yt.mp4 -frames:v 1 -distance 0 -effort 3 03-bbbyt-ffmpeg.jxl` When I look at the extracted frames, however, the JXL looks washed out compared to the PNG, even after decoding with djxl, which I assume reads colors properly:

(It's a bit hard to see at small sizes, you have to open the original file)

Weirdly, imagemagick says that the two images are identical... what?

For reference, here's the encoded JXL file from ffmpeg:

I can't replicate this using avifenc-encoded AVIF files, nor by converting PNG images to JXL via ffmpeg.

Okay, on Chrome they look identical, but on Firefox and Gwenview they are not.

I think something's going wrong with the ICC color profiles somewhere...

The PNGs look identical, but the JXL still looks washed out...

I'm so confused.

Oh, now the profile from JXL and PNG from djxl have a more reasonable trc/gamma: rec709. While the PNG from ffmpeg didn't have ICC profile attached, and usually assumed as sRGB.

And it seems like if you strip the ICC profile from JXL and PNG[djxl], results in a same pixel values as PNG[ffmpeg] one. Thus maybe why imagemagick says that two images are identical.

Maybe something funky is going on with ffmpeg's libjxl integration.

Also, what do you happen to use to inspect ICC profiles?

Encoding the PNG[ffmpeg] file via cjxl results in an ICC profile of RGB_D65_SRG_Per_SRG, according to `magick -identify`. For the JXL[ffmpeg] and PNG[djxl], magick reports an ICC of RGB_D65_SRG_Rel_709.

for jxl, the `jxlinfo` is enough to give profile info if enum values is used rather than ICC: ```JPEG XL image, 3840x2160, (possibly) lossless, 8-bit RGB Color space: RGB, D65, sRGB primaries, 709 transfer function, rendering intent: Relative```

Okay, I've narrowed it down to this: - FFmpeg recognizes AV1 YouTube videos as `av1 (Main), yuv420p(tv, bt709)`. - When you encode a video in that format to JXL, ffmpeg reports encoding in `rgb24(pc, gbr/bt709/bt709, progressive)`. - jxlinfo reports: `Color space: RGB, D65, sRGB primaries, 709 transfer function, rendering intent: Relative` (note the 709 transfer function) - magick says ICC is: `RGB_D65_SRG_Rel_709` - FFmpeg recognizes yuv420 8-bit avifenc output as `av1 (Main), yuv420p(pc, smpte170m/bt709/iec61966-2-1)`. - When you encode that to JXL, ffmpeg reports `rgb24(pc, gbr/bt709/iec61966-2-1, progressive)`. - jxlinfo reports `Color space: RGB, D65, sRGB primaries, sRGB transfer function, rendering intent: Relative` - magick says ICC is: `RGB_D65_SRG_Rel_SRG`

In short, AV1 YouTube encodes show this: Source: `av1 (Main), yuv420p(tv, bt709)` JXL: `gb24(pc, gbr/bt709/bt709, progressive)` The output JXL then has a 709 transfer function.

If you encode other files, you just get an sRGB transfer function.

Seems that ffmpeg is, for some inexplicable reason, *sometimes* encoding a JXL file as sRGB, but with Rec.709 transfer functions??

what if you compare it to the avif

because video is video, i'd say keeping the bt709 trc is correct

video is video... can it even be compared when bt 1886 is a thing

aight i had to do some reading on Rec 709 and things, and yeah, you're probably right about that, the 709 transfer function should be correct

but, what do you mean by this?

how video in sdr is displayed always *depends*

seems like Chrome and Firefox handle ICC profiles differently, as well as mpv

for mpv, it's displaying both the keyframe copied AVIF and the JXL with BT.1886

Chrome shows the AVIF similarly to mpv, so it might be using the same

But for Firefox, it shows the AVIF as more washed out

Here's a keyframe copied `-noaccurate_seek -ss [time] -c:v copy` AVIF

Okay, that's annoying, Discord shows an AVIF preview when uploading but doesn't show anything on embed.

It seems that the JXL is being rendered "correctly" using the inverse 709 transfer function, instead of BT.1886.

IIRC firefox (nightly) doesn't color manage JXL yet, and rendered as default sRGB instead.

just discord being discord

One of the JPEG projects will be discussing a document called "Proposed updates to the proposed draft call for proposals"

Committees are fun

looks like a range issue

FFmpeg reports a warning if you try to pass a partial range input to its libjxl encoder but it will encode it, but assume it's a full range

update: it looks like FFmpeg isn't changing the pixels and just using BT.709

which is correct

looks like a bug with ffmpeg's png encoder. it's taking BT.709 video and not tagging it at all, which is causing most viewers to assume it's sRGB colors, which it is not.

the `03-11-bbbyt.png` file doesn't have any color tags

if you inspect the PNG, it has a IHDR, pHYs, and IDAT with no other chunks

(other than IEND)

which is causing viewers to assume it's sRGB, when it shouldn't be

you could fix this, possibly, by inserting a scale filter `-vf scale=t=iec61966-2-1`

Sorry, it seems like I somehow stripped some metadata while optimizing the PNG to get it under 8MB. Here's a version that actually has the metadata, with a gAMA and cHRM tag: ``` Gamma: 0.50994 Chromaticity: red primary: (0.64,0.33) green primary: (0.3,0.6) blue primary: (0.15,0.06) white point: (0.3127,0.329) png:cHRM: chunk was found (see Chromaticity, above) png:gAMA: gamma=0.50994 (See Gamma, above) ```

it's not possible to represent BT.709 in PNG without an ICC profile

gAMA approximation of BT.709 would produce the slight difference you see

To move some discussion away from <#1021189485960114198>, how should we deal with JXL images with a 709 transfer function (as those exported by FFmpeg)? If you just use the inverse OETF to display the image, it will look "washed out". If you use the EOTF in BT.1886 to display the image, it will look "correct", but since libjxl doesn't target displays, this can mess up calculations. So, should *image viewers* implement a special case for 709-tf images and decode them with BT.1886? Should *libjxl* handle this, since AFAIU libjxl tries to handle color management for you? Or should everyone just use the inverse 709 OETF, like the situation right now? To a normal user who knows nothing about this, I don't think it's optimal to see a "washed out" JXL from FFmpeg. Or at least, it should by default show a JXL that looks "the same" as the video frame when played in a video player.

AFAIK, video players decode BT.709 video with BT.1886 transfer funcs since that's what's recommended. For images, I'm not sure how common images are in 709 colorspace.

bt1886 only works properly if youre profiled and managed

keep in mind that the tagged transfer function is a suggestion if the image is XYB encoded

so the image is actually linear light, and can be decoded to sRGB or any other color space, ignoring the tag

so we deal with them ideally the same way we deal with anything else

> To a normal user who knows nothing about this, I don't think it's optimal to see a "washed out" JXL from FFmpeg. > Or at least, it should by default show a JXL that looks "the same" as the video frame when played in a video player. it does actually, if you use a video player that's properly color managed, like mpv

but again, this is not an FFmpeg issue, since it just passes 709-encoded pixels to libjxl and tells libjxl that they're 709-encoded.

and AFAIK libjxl encode just lets LittleCMS handle all its color happenings

So is the issue that some video players are just not displaying the colors correctly?

Not video, but image players. MPV does happen to show the JXL via BT.1886, and FFmpeg decodes it as such too.

In Chrome, for example, opening an ffmpeg-encoded JXL would show different results than opening an ffmpeg-encoded PNG.

I think they're displaying the JXL colors correctly, though. mpv and ffmpeg are just making a special "case" for 709 and display it via BT.1886 instead of the inverse 709, if I understand correctly, though I may be wrong.

Oh. Feels like a similar issue as the one with ImageMagick (until recently) somehow confusing sRGB with gamma 2.2, i.e. when producing a png it would write a gAMA chunk without an sRGB chunk, and when reading that it would pretend like that's the same thing as sRGB.

ah, did they finally fix this thing? https://github.com/ImageMagick/ImageMagick/issues/4375

strictly speaking, there is now a cICP chunk that allows this (https://w3c.github.io/PNG-spec/#cICP-chunk), but it will likely be a while before most applications implement it

(although Chrome does!)

(and libjxl too!)

<@604964375924834314> oooo I should add this to FFmpeg

update: send a patch to the FFmpeg mailing list to support cICP <:poggies:853085814032302122>

<@853026420792360980> sorry about the ping, but I think for gamma 2.2 and 2.8, you need to do 1 / 2.2 and 1/ 2.8 instead of 2.2 and 2.8 (see https://github.com/libjxl/libjxl/blob/3b026cebd995c9e98b68fc295cac83613924eb38/lib/extras/dec/apng.cc#L186) Otherwise, gamma 2.2 and 2.8 encoding is broken:

thanks for the report, will investigate

turns out, FFmpeg doesn't support sRGB on decode either

so I had to add that support too <:KEKW:643601031040729099>

it writes sRGB chunks on encode but doesn't understand them on decode

I can reproduce this, using this image:

It contains a gAMA chunk equal to 0.45455 but no sRGB, iCCP, or cHRM chunk

this produces gAMA.jxl that's tagged as Gamma=0.45455

and decoding this with ffmpeg fails to assign gamma22

and falls back on sRGB

which would not be a problem in this case because it's XYB

but if it wasn't XYB it would cry

but yes, I can reproduce this, and I should brew up and send a patch shortly

urgh super annoying firefox mobile still doesnt support icc

would love to be able to use xyb jpegs

Works for me

doesnt for me on android 12

i have to download firefox beta or nightly so i can go into about:config and manually enable colour management to get it to work

I guess that's why

I only use nightly

srgb input, xyb jpeg

strange that mode is 0 on mobile by default but is 2 by default on desktop

https://bugzilla.mozilla.org/show_bug.cgi?id=1412350

theres a bug for it thats just been untouched for 3 years except a bot changing the severity

Firefox team is dead <:sadge:1060773081007784006>

> Mozilla Corporation, the for-profit side of the overall Mozilla organization, generated $585 million from its search partnerships, subscriptions and ad revenue in 2021 — up 25% from the year before <:Stonks:806137886726553651> not enough to pay some devs

anyone here w a bugzilla account or a firefox contributor who could give this a poke?

I blame the metaverse thing they have been doing

I think Mozilla does have Mozilla Connect for those kind of discussions, and in fact it seems JPEG XL was well received there, so there is indeed no reason to poke them at the bug tracker.

it seems to me that Chrome has no similar venue for discussion.

You could say "If there are no objections to this patch, please merge."

Bug trackers are meant for testing and patching

They are not forums for chit chat, so if you want to say something, it should be patches and/or test results

update: sent a bug fix to the mailing list

https://servo.org/blog/2023/01/16/servo-2023/

```We would like to share some exciting news about the Servo project. This year, thanks to new external funding, a team of developers will be actively working on Servo. The first task is to reactivate the project and the community around it, so we can attract new collaborators and sponsors for the project. The focus for 2023 is to improve the situation of the layout system in Servo, with the initial goal of getting basic CSS2 layout working. Given the renewed activity in the project, we will keep you posted with more updates throughout the year. Stay tuned!```

AFAIK it's from Igalia https://en.wikipedia.org/wiki/Igalia

which existence itself is amazing to me

No thanks to mozilla

Remember when they actually used to have software developers on their payroll?

They fired all of them so they can give themselves more money :)

spain actually has a number of famous worker co-ops, like mandragon

a friend would like to convert a passport photo into a PDF file with reasonable file size, what’s a good encoding approach that one can implement with free software?

is `pdfjam passport.jpg -o passport.pdf` close to the best one can do with reasonable ease?

That should be fine. PDF more or less stores JPEGs as is, so the size should be the size of the image plus a small amount of overhead.

that was my impression as well

but I was wondering whether perhaps trying some JPEG2000 would make sense, or something like that

``` pdfjam ERROR: no PDF/JPG/PNG file found at test.jp2 ``` not this way, then, I guess

ah https://github.com/josch/img2pdf

colorspace is wrong in the jpeg2000 (image too dark) and quality is worse than with jpeg at that filesize target

I’m clearly doing something wrong

I guess jpeg will do

I'm considering to buy a phone with a thermal camera. I could use it to make some test images for jxl files with a kThermal channel. Any suggestions on which one to get?

Do many phones come with one? I thought it tended to be an addon that you plug into the USB port.

I have a Cat S61, I can make those test images if you want

i know at least the latest one (s62 pro) is advertised with a feature that overlays regular & thermal imaging, but no clue how its stored

How does it store the thermal data? Do you have a way to extract it so we can make some jxl bitstreams with thermal channels?

(i also still may want such a phone to find cold bridges in my home, but that's another thing)

http://git.videolan.org/?p=ffmpeg.git;a=commit;h=9d5e66942c5bae578926e29efebec348199798df

Wasn’t thermal, but I’ve seen 2MB+ depth images being stored in JPEGs XMP as binary blobs.

Speaking of extra thermal/depth channel, is there any JXL sample images for that?

Not afaik, though shouldn't be too hard to produce one using the encode api. But it would be nice to have some based on what real thermal cameras produce.

(also it would be cool to have foss viewers that can actually visualize such extra channels)

hmm, trying to fabricate one

and cICP support in FFmpeg's PNG encoder/decoder is now merged :)

great!

Practical tooling facilitates adoption.

I uh-- may went a bit too far with extras <:KekDog:805390049033191445>

Extra channels can be used to store all sorts of data. Pixel data are usually unsigned but arbitrary data can be signed as well. How does JXL handle that?

it can store signed data but it can only signal either uint or float types to be the nominal sample type.

when the nominal sample type is uint8, that doesn't mean that the actual data is all in range 0..255 — it's perfectly possible to store out-of-range values

you should expect them to get clamped to the nominal range though

I suppose in the current decoder api you would get clamped values when requesting the data in a uint pixel format, and unclamped ones when requesting the data as floats

i asked ChatGPT to write a rap about JPEG XL, with lots of rhymes ``` (Verse 1) JPEG XL, the new image king With compression that'll make your heart sing Smaller file size, better quality too This format's got it all, what more can you do? (Chorus) JPEG XL, JPEG XL The future of image tech, no time to dwell JPEG XL, JPEG XL Upgrade now, and take your photography to the next level (Verse 2) From photography to graphic design JPEG XL's got your back, it's one of a kind Wider color range, better options too This format's got it all, nothing to do (Chorus) JPEG XL, JPEG XL The future of image tech, no time to dwell JPEG XL, JPEG XL Upgrade now, and take your photography to the next level (Bridge) Mobile support, it's a breeze With JPEG XL, your pictures will never freeze Faster loading times, smoother scrolling too JPEG XL, the format you never knew you knew (Chorus) JPEG XL, JPEG XL The future of image tech, no time to dwell JPEG XL, JPEG XL Upgrade now, and take your photography to the next level (Outro) So don't be left behind, join the trend Upgrade to JPEG XL, and watch your images ascend Smaller files, better quality, what more can you ask? JPEG XL, the future of image technology is here at last.```

beautiful

This is amazing, I love it! 😄

I think AVIF would be a perfect fit for AI generated images. 😉

<@853026420792360980> does jxl use multithreading to encode screenshots in mpv (encoding feels slower than through cjxl)? I also did some investigation for png screenshots with `screenshot-high-bit-depth=yes` for 8 bpc video ``` png:cHRM: chunk was found (see Chromaticity, above) png:gAMA: gamma=0.45455 (See Gamma, above) png:IHDR.bit-depth-orig: 16 png:IHDR.bit_depth: 16 png:IHDR.color-type-orig: 6 png:IHDR.color_type: 6 (RGBA) png:IHDR.interlace_method: 0 (Not interlaced) png:IHDR.width,height: 1920, 1080 png:pHYs: x_res=0, y_res=1, units=0 png:sRGB: intent=1 (Relative Intent)``` after ect optimization ``` png:cHRM: chunk was found (see Chromaticity, above) png:gAMA: gamma=0.45455 (See Gamma, above) png:IHDR.bit-depth-orig: 16 png:IHDR.bit_depth: 16 png:IHDR.color-type-orig: 2 png:IHDR.color_type: 2 (Truecolor) png:IHDR.interlace_method: 0 (Not interlaced) png:IHDR.width,height: 1920, 1080 png:pHYs: x_res=0, y_res=1, units=0 png:sRGB: intent=1 (Relative Intent)``` is RGBA really needed?

after some experiments found that for losssless 8-bpc screenshots it's worth using either e1 for fast encoding or e4 for better compression, e3 is surprisingly not good even for movie/live content, and other efforts are too slow or very unstable in compression level same for ffmpeg, but not only jxl e1 is better and faster than internal png with default settings, but even fpnge have much better compression for video screenshots, even if not counting much higher encoding speed <@794205442175402004> I wonder if jxl would accept 10-bit (maybe through some intermediate buffers) and yuv420 directly, might that give some extra compression (and maybe a bit more speed)?

jxl *can* accept 10 bit, but not yuv420

I think we may be able to make it accept yuv420 but don't really see a reason to

fpnge being better than internal png is *very* surprising though

10-bit can be useful for 10-bit video for screenshots (it's very common, even without hdr) because 16 bpc screenshots are very bloated not only for png, but also for jxl yuv420 is also very common for video (and lossy image formats), although I don't know how much it will improve something

jxl can only do JPEG-style yuv420: full range, interstitial siting of subsampled chroma

in video codecs, it's typically tv-range and a weird siting where horizontally samples are cosited (at the positions of luma samples) while vertically they're interstitial.

for screenshots of video, does this get integrated into the movie player so it has access to the yuv420 data? because what's on the screen will be rgb data, and probably rescaled to fit the screen too

avoiding the rescaling is probably the main thing you'll want to do — if it's upscaling, you'll encode too many pixels, if it's downscaling you're losing information

frames from some random youtube trailer ffmpeg (2263180) fpnge (1611574) just for comparison (for other movies, animations and various content fpnge was also better)

most likely, but mpv uses ffmpeg at least if ffmpeg can do it, so can many other players and tools

and as I understand it ffmpeg cannot directly accept anything between 8 or 16 bit for jxl https://discord.com/channels/794206087879852103/794206170445119489/1065821078120824954

mpv uses FFmpeg to encode screenshots, which defaults to autothreads

as for alpha channel, that's because of the frame buffer. try setting screenshot-sw=yes and see what happens

either way, that might change when haasn overhauls the screenshot code

I'm planning on changing the default to e4

16bpc is only bloated for lossless. unfortunately this is a limitation of libjxl

mpv screenshots are pre-scaling

unless you push Ctrl+s, then it screenshots the actual window

wdym?

I mean libjxl is limited to accepting 16-bit buffers or 8-bit buffers

you can't pass a 10-bit RGB buffer to libjxl

it can take 16-bit buffers and treat it as 10-bit though

(ignoring the 6 lsb, basically)

how does that work? 1023 doesn't divide 65535

65535 factors into 3, 5, 17, and 257

1023 factors into 3, 11, 31

not sure if we already have a way to pass data using only the lower bits (so you're using uint16 but the range is 0..1023)

basically I guess that's how you'd do it

you'd have to left-pad 10-bit and tell libjxl it's 10-bit with the JxlBasicInfo

but can also right-pad, take the 10-bit data, do `* 65535 / 1023`, and send it as uint16 with bitdepth 10

how does libjxl know which one it is?

ah you can do either

https://github.com/libjxl/libjxl/blob/main/lib/include/jxl/types.h#L113

oh, good

JXL_BIT_DEPTH_FROM_PIXEL_FORMAT means uint16 buffers have range 0..65535 regardless of bitdepth, so things are lsb-padded

JXL_BIT_DEPTH_FROM_CODESTREAM means uint16 buffers have range 0..(2^bitdepth)-1, so things are msb-padded

and then you can even do JXL_BIT_DEPTH_CUSTOM, to e.g. decode a 12-bit image to uint16 with a 10-bit range (0..1023)

hm

looks like FFmpeg doesn't define a pixel format which is 10-bit RGB packed into 16

it does define 10-bit *planar* formats

which just makes me feel like libjxl should accept planar input

how does the 10-bit planar of ffmpeg work? no padding except at the very end of the plane?

i mean, does it do 4 samples in 5 bytes? or just 2 bytes per sample?

I don't know

I could check

also if using ffmpeg to make screenshots/thumbnails: jxl `-vf thumbnail -distance 0 -effort 1 thumb%04d.jxl` png `-vf thumbnail -compression_level 100 thumb%04d.png` and for jxl there are some warnings (but not for png, webp, bmp, tiff, jpg): 10-bit bt709 av1 ```[libjxl @ 00000150256f52c0] Unknown transfer function, assuming IEC61966-2-1/sRGB. Colors may be wrong. [libjxl @ 00000150256f52c0] Unknown primaries, assuming BT.709/sRGB. Colors may be wrong.``` 10-bit bt709 hevc ```[libjxl @ 0000024ed1d7dfc0] Unknown transfer function, assuming IEC61966-2-1/sRGB. Colors may be wrong. [libjxl @ 0000024ed1d7dfc0] Unknown primaries, assuming BT.709/sRGB. Colors may be wrong.``` even if ffmpeg accepts it correctly, i think it would be nice to handle the most common color primaries and video transfers for jxl (or the same as for png) but it seems that ffmpeg creates 8 bpc jxl and png for 10-bit video, which is bad <:PepeSad:815718285877444619>

<:FeelsReadingMan:808827102278451241> https://trac.ffmpeg.org/ticket/7978

how to use xyb-jpeg encoder?

is it included in cjxl 0.8?

C:\Users\Use\Documents\basando benchmark_xl --input bhby.png --output_dir . --codec=jpeg:enc-jpegli:q90 --save_compressed

thanks fab

how do you confirm encoded jpg is xyb?

Xyb is not

Then you should add xyb

How?

where?

yeh. windows build moment

check `magick identify -verbose` for an ICC profile i think

`benchmark_xl --input=x --codec=jpeg:enc-jpegli:q90,jpeg:enc-jpegli:q90:xyb --save_compressed --output_dir=.` https://discord.com/channels/794206087879852103/803645746661425173/1067878025716305930

Many image editing programs are pretty stupid and they are encoding fully opaque images as RGBA PNGs. I think libjxl could provide a way to detect and automatically choose the appropriate pixel format. Developers of those editing programs are apparently unable to do it so the encoder could do it for them.

hmm

https://youtu.be/YW26YMe8iUQ?t=3m56s

quite high quality

Hm, `cjxl` v0.8.0 can only beat an `ect -9 -strip` PNG r/place by a little bit, even at `-e 9`: ``` 1.1M final_place.e7.jxl 916K final_place.e9.jxl 956K final_place.png ```

a very unique picture, but interesting nonetheless

as mentioned before by Hitori, `cwebp -z 9` beats both JXL and PNG by a good amount: `888K final_place.z9.web`

yeah that's a rather particular image

I can beat webp but it requires some funky options

are there recommended options for screenshots with text (e.g. terminal)? I noticed lossless webp is usually better than jxl for such images

`-I 0 -P 0 -e 9 -g 3` is a way to make jxl more png/webp-like, i.e. it will learn no MA tree but do lz77 search instead.

Usually that's worse but for things like screenshots and pixel art it can be better

Can confirm, in special cases, this is worth trying.

Though for screenshots in particular, patches might be more effective than lz77 to get rid of duplication like repeated letters. But then again our current patch heuristics catch only some percentage of those.

(when doing the -I 0 thing, it's probably better to disable patches altogether because it kind of interferes with the lz77)

Did I mention that there's room for improvement in various parts of lossless jxl, and even e10 is quite far from exhaustive/optimal?

It's just such an extremely expressive bitstream that it might take AI heuristics to really get close to optimal lossless jxl encoding

I can imagine that deep search for patches would be pretty time consuming. Solution might be a way to give a set of patches, like a font, to the encoder, so it knows what to look for.

Yeah if the application that drew the text (or other repetitive elements, like icons or profile pictures in a chat or whatever) could tell some stuff to the encoder, it could be made more effective

Who knows libjxl 3.0 will have an api for that 🙂

<@274048677851430913> I went to https://kampidh.saklistudio.com/ and noticed that jxl files are shown to browsers that support jpeg-xl however I went and compared a couple of the images and noticed that in all the ones I tested the jxls ended up lower quality then the jpegs

JPEG:

JXL:

JPEG:

JXL:

aaaaa i'm not sure if the image editor i'm using is copying the layers correctly into discord

I should have checked first (edit: I just checked and they are correct)

I shall update those images in that comm page, those are made on earlier build of krita which doesn't use xyb~

Plus that jpeg is at q100 if I remember correctly..

speaking of q100 does krita support doing q100 lossy vardct jxl(s) or does putting the quality slider to 100 set it to be lossless? (I don't use krita so I wouldn't know)

current krita (5.1.5) doesn't have that quality slider, only exporting to default -d 1 with original profile set to true. the nightly / upcoming 5.2.0 will have that q slider and xyb optimization

(I might have to update those comm images as well since those refs are old haha)

I've noticed they aren't listed in your artwork gallery

yup

any specific reason? I can see the one under special tier not being listed since it's a collab however the rest seem fine

I see, good to know.

nope, just still focused to put personal artworks first on the main gallery page~

and done updating those images~ some still use old ones but reencoded properly

just checked and not only are they higher quality they are also smaller in file size 👍 (only was able to compare with the tier 0 image since I didn't download the tier 2 image for testing and all the other ones are changed to different images)

actually I do see a problem and that is that the colors between the jpeg and jpeg-xl are now out of sync

oh?

~~maybe because of jxl one have 16 bit depth?~~ scratch that, its 8 bit

JPEG and old JXL (old JXL was sent for this example but the colors are the same between the JPEG and old JXL it's just the quality is lower as previously discussed):

new JXL:

ohh, the old one kinda had an edge sharpening there

Interesting, the jpeg one has differences as well compared to the old one.. pretty sure I'm using lanczos as well for resizing back there (or it was resized somewhere else)

A lot depends on what colorspace you resize in

Resizing in linear space is 'correct', but it looks brighter than resizing in nonlinear space and most people are used to nonlinear space resizing (since that's what browsers and many viewers do)

gotta say the 0.8.0 improvement on vardct with anime went quite nice as well

On anime, codecs with directional prediction (like webp and avif) have an advantage. Though that's mostly only really true in the lower quality range, where angles can get rounded to the nearest supported direction.

https://pony.social/@cadey/109813678329119959

creative use of video compression artifacts

https://twitter.com/palsnurseoutfit/status/1622541981737177096

what it ect

https://www.color.org/images/browsertest.jpg

lol, neat.

interesting... this also does the gray-white test which chrome still fails

seems discord thumbnails dont do icc, but if the image is small enough it doesnt generate a thumbnail

is this an XYB jpeg?

I think so: ```console $ exiftool -icc_profile:"ProfileCreator" -icc_profile:"ProfileDescription" IMG_20230208_104324_041.jpeg Profile Creator : Unknown (jxl ) Profile Description : XYB_Per ```

makes sense

(scaled it up to make discord generate a thumbnail)

Hey, is there any good resource about how JPEG XL does entropy coding? In particular I'm interested in how MA trees and context clustering with histograms work in jxl. I tried to look through the iso spec but I found it quite cumbersome. How does jxl come up with distributions it uses for ANS (I've read about predictors, but how does it translate into actual distribution)?

Distributions are easy: just count. For ANS the encoder needs to collect all tokens first anyway before it can write bitstream (since the encode side of ANS is LIFO), so the distributions used are just the histograms of symbols that occur in the image (though similar distributions do get clustered to reduce signaling cost, so after clustering it won't be exact anymore, but it doesn't need to be exact)

The hard part is coming up with an MA tree that ends up being a good ctx model. FLIF and libjxl do that in two different ways, and both ways are basically a greedy algorithm that grows a tree one decision node at a time, which means it likely only finds some local optimum but not a global one. E.g. if the optimal thing is to split a property range into three, say distribution A is good when N < 50 and when N > 150, while distribution B is good when N is between 50 and 150, then likely neither flif nor libjxl will discover the appropriate MA tree, since in a single binary decision node you can't make a very good split.

So what do use for tokens, DCT coefficients or just channel values, and how do you bundle them (histogram for each DCT freq)? Also what role do context play in encoding? As I understand it each context comes with predictor, which you use to get residues. Do you count histograms after getting residues, or how do they relate to histograms?

The entropy coded symbols are residues.

That makes perfect sense, thanks!

I wonder how well JXL can compress this image...

What the hell? Apparently waterfox has gfx.color_management.enablev4 set to false

that's the firefox default and waterfox has not changed it

only the newest versions of firefox have it on

and waterfox is based on firefox esr

which is a bit behind

I heard that v4 profiles are very ancient

Wonder why it was ever off by default to begin with

oh wait, the question answers itself again. Mozilla

firefox 3 introduced support and firefox 108 turns it on by default

current firefox esr is based on version 102

so waterfox is on firefox 102 right now

lmao it's been around since version 3 and they waited until 108(!) to turn it on for some reason

Why are they so terrible... Not enough manpower to flick a switch to the on position?

it was slightly messed up in firefox 3 so they fixed it in firefox 3.5

and I think the reason they didn't flip it on was because they where worried about security or something

not sure what made them flip it on recently but i'm glad they did

found the but report it's from 14 years and was only closed 4 months ago for firefox 108 https://bugzilla.mozilla.org/show_bug.cgi?id=488800

oh wow did they turn it on because xyb jpegs where not working?

this is what got them to turn on support i'm pretty sure

lmao

safari for windows 🧓

In lossless mode, it should compress quite well.

yeah we need to improve e2 - e10 lossless in terms of speed and compression

``` cjxl Mozilla.png -d 0 M.jxl -e 9 -g 3 --patches 0 JPEG XL encoder v0.9.0 1ff5061f [SSE4,SSSE3,Unknown] ./lib/extras/dec/color_hints.cc:54: No color_space/icc_pathname given, assuming sRGB Read 786x539 image, 72542 bytes, 153.7 MP/s Encoding [Container | Modular, lossless, effort: 9 | 14-byte Exif], Compressed to 19380 bytes including container (0.366 bpp). 786 x 539, 0.25 MP/s [0.25, 0.25], 1 reps, 4 threads. ```

https://littlecms.com/img/blog/stress.jpg

lmao... Firefox still does not support v4

If you set enablev4=true, all it does is cause v2 profiles to be correct

not v4

How stupid

Otherwise it doesn't even do v2 profiles right lmao

What's funnier is I think it's literally the only browser that has this problem

Every other browser works, even vivaldi

> Fixefox folks argue color management is a thoughput killer, so in order to accelerate things, V4 is de-activated by default and has to be enabled by using an arcane combination. Ok, no problem, I did the gfx.color_management.enableV4=true trick, and tried again. Nope, it still does not work. Using default settings it fails on the 15-years-old test we used to check V2 compatibility. Now on V4 with all bells and whistles activated, Firefox keeps ignoring profiles otherwise fully V4 compliant and valid. It is fast, sure, blazingly fast. But color management doesn’t work. Worst, it works a little bit. That means, you are not sure if the color you see is correct. Which is indeed a very bad situation for color pros. I think it would be better to deactivate completely color correction and specialize this browser on tasks where speed is the main requirement.

I can convert the v4 test into a jxl but cjxl gives me an error if I try to convert the other 2 v2 tests

`JxlEncoderAddJPEGFrame() failed.` `JxlEncoderSetICCProfile() failed.` if `-j 0`

Oh, interesting. Can you open an issue for that? Or get a more informative error from a debug build?

> even Vivaldi Is vivaldi's color management somehow different from chromium's?

nope vivaldi is pretty much just chromium but with it's own ui

That's what I thought

Is there already a jxl issue for vivaldi?

I don't think there is a issue filed for adding back in jxl support for vivaldi

I don't think vivaldi even has a public issue tracker at all. They just have a forum and someone already made a forum post.

I don't have access to a debug build rn.

yeah I'm pretty sure vivaldi's issue tracker is private. also what forum post do you see?

https://forum.vivaldi.net/topic/61006/support-for-jxl-image-format/

oh that thread....

The image they are referring to is a losslessly-transcoded JPEG

that thread is very old and the 2 people in it don't know really anything about jxl and they think it's larger then jpeg and webp

The reason why it's so large is because the camera's quality settings and quantization tables

Since it's an original from a camera

Also they think Ghent is the same as "Gwent," the Witcher card game

or Welsh county of Gwent perhaps

while I was fighting with AOM in 2 days, I mastered MSYS2 and now I can build Windows Jpeg XL builds myself <:Windows:806135372298977342> <:JXL:805850130203934781>

although the received cjxl size 90 megabytes, and requires mingw libraries in same folder. I think it's not that important.

that doesn't seem right, why is it so large?

for example, this static build of cjxl v0.8.1 i compiled yesterday is less than 8MB

oops, this is an O3 march=native build, right

what CPU are you running, so I can maybe attempt to make a build for you?

90 megabytes sounds like it has debug symbols

you can change `CMAKE_BUILD_TYPE` to `Release` in `CMakeCache.txt` directly and then rebuild