JPEG XL

Ok now with animated set and all frames set to "page end" VipsDisp doesn't display it properly any more. Like shows only parts of the image, my guess is that it cycles through frames at max speed. Which isn't the fault of frame stitcher of course

But it's interesting to see that VipsDisp decodes the frames "on demand" when switching frames. Like it doesn't just decompress all the pages on load

Oh wow there are actually public domain comics. Check out THE HANGMAN #1 <https://comicbookplus.com/?dlid=86397> where Bob Dickering becomes the hangman to avenge his brothers death while taking two boy buddies with him on his war on crime adventure!

*To protect people's sanity, this comic book has been locked away in a multi page JXL file that nobody can read. But will a mad scientist unleash this forbidden book from the 1942 on the world?!*

actually right now it decodes the whole thing on load decode-on-page-load is on the todo list

Huh I could have sworn I saw individual tiles popping up. Or does VipsDisp load the file into a tiled image?

Could also be the UI system uploading / swapping image tiles to the GPU mem

Chunked decoding is a thing yes

Some image viewers support jumping one frame using keybinds, so those work fine for these kind of JXL images xd

I think for a multi page document reader on desktop you'd want 4 different fullscreen modes: Select / Autodetect: - Comic book: Single/Double/Wide page, zoom 100% screen height - Webtoon: Single page vertical scroll, Zoom ~800 pixel width or ~25% screen width - PDF / scanned document: Single page vertical scroll or like Comic Book, zoom 100% screen width - Manga: right to left (inverse page order?)

Could pages be imported as layer groups perhaps?

vipsdisp image display is based on pyramids of tiles .... it generates a sparse pyramid of 256x256 tiles, loads them on your GPU as textures, then scales and composites them to the screen it needs to do this to do smooth zooming on huge images (eg. 500k x 500k pixels) tile generation is decoupled from image display, so on page flip it starts generating the tiles and the screen only updates some time later as threads in the worker pool complete so if page flip is faster than pyramid generation, you can see tiles from one frame in the next

I'd like to load tiles from jxl on demand (it does this for other formats, like tiled tiff), but the jxl chunk interface isn't a great fit for libvips, so for now it just decompresses the whole thing to a temp file and makes the pyramid from that

Thanks for the info! VIPS handling gigapixel images is pretty cool

Vips is the gigachad image library

I read that as "war crime adventure."

I've been playing too much metal gear...

I'll gonna put that to a to-do, for now I've proposed a fix to load as a normal animation frames instead

Is JXL frame stitcher available on non Windows?

Not currently, maybe gonna try to fiddle around with WSL again for linux deployment like what I did with jxl batch converter

Is it open source? I could possible make something at lease CLI for Unix

Don’t know JXL api though

Yep, the frame stitcher source code is on that same repo as the link above. Could be a great help if someone can compile that on Unix systems xD

Dumb question I’m guessing, but I __can__ fork it right?

Of course ^^

I’m terrible with UI but I could probably get a CLI version compiling

Although not right now, end of year exams

the biggest libvips image i know of is 1,600 gigapixels heh https://www.epfl.ch/labs/emplus/projects/diagram/

Damn, just outside of JXL spec haha

Outside Level 10. Not outside the spec. Just doesn't conform to any current profile/level.

480k x 5m pixels from memory

spec allows up to about 1g x 1g pixels, though Level 10 limits it to about 1100 GPx

Apparently Microsoft's WIC codec for JXL in Win11 v24H2 is still incomplete, the developer of Paint.NET posted on the Paint.NET Discord that it just throws an exception when you try to use it.

Right, yeah. I was thinking back to our "Earth in a file" idea with 1m per pixel resolution

> Typical image file sizes for 8 bit colour ranged from between 1.3 TB and 1.5 TB (per scroll?) . This is simply doubled for the 16 bit versions given that the files are saved as uncompressed RGB values. > 3805340 x 425000 pixels, 1.62 terapixels

So roughly 4m x 400k

The bottom of this article has file formats and processing https://www.epfl.ch/labs/emplus/projects/murten-panorama-digital-twin-scanning-project-the-making-of/

Yeah, they added it to the background selection but it does nothing since there's no code to run, just registry entries

I am a little surprised the would add it to the selection if it isn't yet supported. But it could be like their WebP and AVIF WIC codecs where they plan to use an addon MS store package that has to be installed for it to work.

Yeah, that's what we've seen evidence for

is the actual image available as it would be funny to see how long it would take to encode

~~with hydrium~~

hydrium would end up taking 4 seconds

maybe, being single threaded is not great for its perf vs libjxl

but I'm not going to implement threading because most low-memory environments where it shines aren't going to be multithreaded anyway

is hydrium for lossy or lossless?

also how would it work for something like that really big panorama, as wouldn't it have to store the pixels that were incoming?

or do you have to pass pixels in group by group?

it's fixed quality, currently at around d=1

and no the library accepts pixels one tile at a time

tile = 2048x2048 group?

choice between 256x256 up to 2048x2048

power of two

if the input is from a PNG then generally you'll need to buffer between 256 and 2048 scanlines of that PNG file

since PNG goes in scanline order

so scanline caching is up to the end user

ye, though if you want to maintain super duper ultra low memory you decode the PNG multiple times

wouldn't that kill cpu?

yes

libhydrium accepts tiles, and the default frontend which decodes PNGs (the library doesn't) decodes enough scanlines to make a group at a time

so if you want 256x256 tiles you need to decode 256 scanlines before those tiles are available

ah I found the discussion where the software guy on the project talks about generating the final output image https://github.com/libvips/libvips/discussions/3939

though he doesn't give a time to generate the deepzoom image

Huh, a museum in London. Makes sense when you think about it and what VIPS is best at. I'm actually heading up to the city Saturday to meet some friends haha

oh, you're uk? yes, I was at the NG 1990 - 2005, I did all their imaguing research

Yeah, I'm down near Brighton but travel up every 3 weeks for a social event

Here's a [sample playground link](https://wasm-vips.kleisauke.nl/playground/?modules=jxl&deflate=dVJNa9wwEP0rw1KwAxtpU-hlm81hQw-BQqG09LBeGtmelZXKGiHJcUPIf-9Y3q7poRhkzcd782ZGh5WUcHC765sj9KhchICuxQCDS8ZC6hDYBjrla0vN0KNLlWPUQ680QqQhNLiFLiUft1LGRIH9QhNpi8qbKBrqZUujs6TaKEasex_oCZskKGjJtpdm4oqyVa4xTv-slePvvbAUo8WYQb5yFhPLGPraKda2g2fjo8gyxMVdFjM4Q-bGijV82GzW8Fo5gA6N7tJ28lTu7epj5XIzP4JJmHsMGAfLdQgU1JbqyjXkIjs2XNFjOFHoWSYKR2M54ecwDWk_nE48ud2iUYwT7TeaI2Uhnn7bYsGkm_9QzglkkSegy8d7ZXkVBP-wsYN-wbtXJrlmcW_QG2tNREbylB8XmpTb4EoOR9jztTxcxB55KpBePC-wyDuQk0LIc1mw379-ZjifogmoEn6pp-2xXU7RJdf0mvP-PpJz8ieLk1Uyv86981_E0HDmmZt9F4zGdAbsXx7asmClfkjFlVDe80O874xtS2aYiFbr1W1rnsG0u2o1J1aru1vJvrsc5eP4Bw) now that wasm-vips v0.0.11 was released (which uses libvips 8.16).

the stable branch of libvips now has multipage jxl support https://github.com/libvips/libvips/pull/4231

and that comic views correctly in vipsdisp

the fast progress of vips is amazing

vips doesn't support HDR transfer curves for any image format yet right?

I was just looking at the code. They have a streaming implementation for input, which is understandable given that app's use case. Most other libjxl users (including my own Paint.NET plugin) just load the entire JXL image into memory for processing. It is also one of the few implementations I have seen with animation/multi-page support.

It's my favorite general purpose image library, a great replacement for imagemagick

although imagemagick/graphicsmagick is still used a lot too, so it would be great if there was collaboration there.

Oh, cool! You just released an update to the jxl plugin!

thanks :)

I think the most important thing when it comes to adoption to a new format is convenient and available tools that can utilize it so it's great how vips and paint.net have good compatibility thanks to great people donating their skill and time. Thanks tremendously

So hang on. JXL supports multi page multilayer animations ?

surprisingly yes

I can't wait for default Jpeg XL support in Paint.net.

each page can be animated separately

Can pages reference previous pages?

How does it work if pages themselves are internally implemented as frames?

I think each frame counts as a frame that page special frame with time code

it's up to image viewer application, it could treat a frame with duration `0xffff_ffff` as the end of single animation and turn it into a page

which means it DOESNT work

because if there's no standard then there's no POiNT

durations are defined in the standard, and `0xffff_ffff` magic code is also in the standard

then it shouldnt be up to the app

if it's up to the app then it's a broken standard

As in the app can request to enable pagination

Some apps don’t want to / don’t need to / can’t handle pages

then all may as well not have it

Oh God I hope this doesn't become another TIFF situation

see: random features in every other format

they may as well not exist

Just cause not everything will use a feature doesn’t mean it’s not useful for those which do support said feature

nuh uh

TIFF PTSD

as I remember, an image has TPS, keyframes have their duration as unit of ticks, and if duration is `0xffff_ffff` then it's not an animated keyframe but a page in multipage image. how to interpret no-`0xffff_ffff` image and all-`0xffff_ffff` image is obvious, but how about mixed ones? I agree that it's not so clear, though the spec doesn't reject such images as invalid. then how should apps react to those images?

Paint.NET's main developer (Rick Brewster a.k.a. rickbrew) recently said on the Paint.NET Discord that he wants to add JPEG XL support after 5.1 releases. Quoting that [post](<https://discord.com/channels/751565809972936735/1010984760782377160/1300910354766757959>): > After 5.1 stable releases I want to get JPEG XL support in the main app. Since the WIC codec that comes with Win11 v24H2 doesn’t actually work (just throws ComponentInitializeException), that leads me to think that out-of-processing libjxl could be a good direction to go in if it still exit()s

> Since the WIC codec that comes with Win11 v24H2 what?

there is an included wic?

Yeah. But it doesn't work yet, hence why he is looking at bunding my Paint.NET JPEG XL plugin.

neat I guess, kinda

Sadly Microsoft has a history of shipping half-baked WIC codecs through their store. IIRC their WIC AVIF codec can't even open all of the test images they contributed to the AVIF test image repository because it lacks transparency support. 🤣

yeeee

The spec needs to clarify this with specific guidance. Should the animation loop?

And yes pages can theoretically reference previous pages since they are just frames, but someone actually needs to write an encoder that is smart enough to detect redundancy between frames and do that, which might not ever happen

I guess the animation cannot loop, because the last frame doesn't have actual duration

Yeah. I haven't read the spec, but I would have thought that mixing animations and multi-page images would be invalid. Using an animated frame duration of 0xffff_ffff to mean multi-page is a clever trick, but it makes the format harder to use versus having a basic_info field that specifies if a multi-frame image is an animation or not. Adding layer support is on my long-term feature Wishlist for the Paint.NET JPEG XL plugin, but I still need to reject animated images because my plugin does not have the UI for configuring animations.

PDN 5.1 plans to have color management in some form right? depending on what that exactly entails (I haven't looked at the beta(s)) I assume your JXL plugin would have to be updated since iirc it currently converts JXLs to sRGB on decode

Yes , 5.1 will have color management. Color management is one of several areas of the plugin's code that I will have to look into. I already have a bug where the plugin will tell Paint.NET use the embedded color profile without checking if it is RGB. Paint.NET doesn't currently support HBD, it only operates in 8-bits-per-channel. So I think the XYB to sRGB code may only need to tag the loaded document with a sRGB profile.

I haven't looked at a lot of the new 5.1 APIs, but I will after it is released.

HDR is the slightly mysterious part, for me anyway and getting chunked load/save working sigh

HDR is mysterious, I wrote an AVIF plugin for Photoshop and I still don't understand it. Are you writing your own encoder?

imo color management itself is very mysterious, I've written non-trivial amount of color management code (including HDR stuff) for jxl-oxide and I don't quite still get it 😅

Yeah. Color management is one of those complex things that are best left to specialized libraries like Little CMS. My AVIF Photoshop plugin has a hack to detect a few color profiles that can be represented as CICP values, and I also synthesize ICC profiles from the CICP values when loading as Photoshop doen't understand the CICP data.

one day I will make a rust color management library, the issue is my RSI prevents any serious coding sessions.

tbf, I'm pretty sure people intentionally make it as verbose and hard to look into as possible

well, I mean unless you pay for paywalled specs. looking at you sRGB

you will buy spectrophotometer and calibrate

I maintain libvips ... it has HDR (scRGB etc., so float channels with values that can be outside 0-1) and CM (with little CMS), I just need to link it to libjxl correctly

HDR is significantly easier to deal with then SDR because sRGB is such a mess

hlg and Pq are very explicit

I think ICC 4.4 profiles can do it, but there are odd cases, like ICC profiles for SDR with CICP data hidden in comments for HDR

lcms refuse to add support for things like that (reasonably) so loaders have to do it

or that's my understanding right now (probably wrong)

ICC profiles can have a CICP tag that is informational. I forget which ICC 4.X version added it, but I have used that feature.

I remember the first time I opened the spec for iccmax, I gave up on ICC that very moment

that's ICC.2 iirc

There is a header field for that. Animations can play a fixed number of times or loop infinitely.

This is off topic, but what are the requirements for a developer role? Contributing to libjxl?

Only question is what to do with looping if there are multiple pages that each contain an animation. I guess this is not clear: should each page show its animation looping, or does looping just mean that after the last page the next page is the first page again? (The latter would be my interpretation of the spec, but it should probably be made explicit)

It's just ad hoc, devs of other implementations of jxl and of other image libraries can also have that role. Would you like to nominate someone?

<https://svalboard.com/> might be for you

I know troy sobotka complained that only ICCMax was suitable for HDR images because some tags were unsuitable

yeah, when I get some money, I plan on looking into them very indepth

``` So for starters, ICCs shouldn’t be used for HDR as they are literally domain bound. It would require ICCMax extensions to do it properly. ... If one is serious about fixing it, the wiser approach would be to use the non-domain bound ICCMax tags. And in the CMS, which would ideally be written from scratch for display work, leverage the appropriate ICCMax tags. At least if I were tasked with designing an appropriate path, that’s where I would certainly start. curv and para tags are explicitly domain bound in ICC, and scaling or other hacks do not seem prudent. And it would help unfuck the glaring fuckup in V4 regarding achromatic axis. ```

ergonomics are not real (im on the computer for 18 hours a day and have 0 issues)

insert, <god I wish that were me> here

I was wondering whether I could get it assigned to myself. I maintain the Paint.NET JPEG XL plugin (which uses libjxl).

ICC 4.4 adds a full float path with parametric curves and out of range values, so you can do HDR with plain ICC profiles or that's what mm (lcms maintainer) says I think

that would likely help for sure then. I know this was back in like 2020 so it's refering to 4.3

all I know, is that the ICC spec sucks to go through, I pity people working on it lol

of course only other icc 4.4 programs will be able to consume your profiles sigh /shoots self

of which there are 0 because icc 2 is the best

I can't think of really great use cases for animated multi pages. Like animated comics or presentations, but then you could also create some ad-hoc format to bundle webm files and an app to play it

ICC needs to be retired, I hope we find a better way soon, oh well, jxl metadata is good

I have to work with DICOM as well, so the ICC spec seems OK to me hehe

(the DICOM spec is a 2,800 page PDF)

v2 or icc.2 because I would rather play dodgeball with a wreckingball then use icc.2

v2 is the best

according to argyll

I think libvips will do multipage jxl or animated JXL, but not both in a mix

yeah, v2 is woefully inadequate. but at least it's doesn't make you want to play free fall trying to implement it

I think that's probably safe lol

Not surprising. DICOM is a medical imaging standard, so I would guess it picked up a lot of different format extensions over the years. But 2,800 pages is crazy for a single PDF, I would have thought it would be split into different sections that are each their own PDF.

that's when you add the whole lot up, there are lot of more digestible versions but it's a *very* big standard, and somewhat TIFF-like, in that it's easy to write a DICOM, but the format is so flexible it's very hard (impossible?) to read an arbitrary DICOM someone might give you

I don't really get why you would design a format the TIFF way, that is, maximizing encoder convenience by requiring no normalization at all. Big endian or little endian? White is maxval or black is maxval? Etc etc. TIFF just allows all possible combinations so an encoder can basically just do a memcpy of whatever internal buffer format they have, but a decoder has to be able to deal with anything...

yes, TIFF is like markup, it's descriptive not proscriptive

i doubt if anyone has ever implemented a TIFF reader than can do all TIFF files

that's kinda what happened with jpegxl no?

Is there a command line tool to "pack" multiple jxl files into one multi-page jxl file? And then unpack them again? That should at least be possible theoretically right?

How do you mean?

Not sure if an unpack would actually be that useful except for jpeg

there's just too much with the format

nobody knows what to do with pages and animation

lossless lossy patches

modular vardct splines

Possible, yes, but tricky in the general case. There are some constraints too, e.g. you cannot mix XYB and non-XYB jxl since that's a global property, not a frame property.

The difference is that in jxl all decoder complexity is motivated: it either brings more functionality or (potential for) better compression. In TIFF much of the decoder complexity is only there to make life easy for encoders.

for me, I just care if oiiotool can decode it lol

it's quite clearly defined in the standard

not according to the recent chat here

It is defined we just all forgor

Not in 18181-1 at least, this is the entirety that mentions pages > If duration has the value 0xFFFFFFFF, the decoder presents the next frame as the next page in a multi-page image.

Ah you mean pages *and* animation in the **same** image

Personally, I'd assume you treat each page like it's own image/animation, with the user moving forward and back and looping applying to the animations. Since it's far more useful than just going back to the first page, which you can already do by reloading, viewer settings, TOC, ect

Otherwise you'll end up with animations ending before you've even read the context around it

That's exactly why the spec should clarify

Problem is the last page has a time code of 0xffffffff so how long is the last frame last

You could ignore the last frame and loop the rest, and use a dummy empty frame to mark page boundaries, I guess.

Having a dummy frame doesn’t seem particularly elegant though

Also that would clash with how non animated pages work

As they have 1 frame so it would just get ignored

Multiple pages with looping animations on the pages is a bit of a specific thing, though I guess it could make sense for something like a slide deck

Actually it could probably be worded like “the frame with 0xffffffff is processed as if it had a time code of 0”

Idea; maybe it could be used as some kind of alternate frame for if the system doesn’t want to handle animated pages

Like if the system supports animated pages, play frames 1 to N-1, if not then play frame N

so it would not be a wasted frame

That's also what I was thinking but it's still something the spec ideally provides guidance and clarification on

A dummy noop frame only costs a few bytes to signal

Worst case scenario you could use a tiff container :)

Yes. For now I am not aware of applications that can show multi-page animated jxl in a sensible way, but if there are some, we should have a good convention for how to do it and put it in the next edition of the spec.

I just feel like that “dummy” frame should be able to do something

Well, you could name it the page number, ect

Also it’s interprets different to how non animated pages work Non animated pages: 0xffffffff means that frame is a page Animated pages: 0xffffffff means to skip this frame and next frame is a new page

For animated it could inherit the previous frame duration, but that is a bit limiting

Actually it could just be “a frame with length of 0xffffffff is processed as if it has a length of 0”

That would solve the interpretation issue

As for non animated pages 0 is expected and for animated it means skip

Wouldn’t you ideally want the page name to be stored in the first frame of the page?

I kind of like the idea of using the last frame of an animated page as an optional alternative page for non-animated display, e.g. when printing it.

(or when showing it on a website when 'prefers-reduced-motion' is enabled)

> If duration has the value 0xFFFFFFFF, the decoder presents the next frame as the next page in a multi-page image. Animated pages will use an optionally empty frame to mark the end of a sequence, interpreted as duration 0 by the decoder. If animation is not supported, the frame with duration 0xFFFFFFFF will be decoded instead. ?

Only thing is a page with 2 frames would technically be a 1 frame animation and would show up as differently with and without animation enabled, although that could be helpful in some situations, like as you say with prefer reduced motion

"Animated page" needs to be defined. I guess it's any page containing frames of nonzero duration.

Yeah, just a rough draft

> If the duration has the value 0xFFFFFFFF, the decoder presents the next frame as the next page in a multi-page image. Any page with frames with nonzero duration shall treat the frame with duration 0xFFFFFFFF as an “alternate frame”, which is to be decoded when animation is disabled.

How’s this

I’m not great with specification-ese

> If the duration has the value 0xFFFFFFFF, the decoder presents the next frame as the next page in a multi-page image. > If `metadata.have_animation` is `true`, any page with nonzero duration frames shall treat the frame with duration 0xFFFFFFFF as a “static page”, which is to only be decoded when animation is disabled. I think this better explains what it's purpose is, although maybe it should be the last nonzero frame instead since it's already been decoded at this point

I would think the table of contents would be best

What series an empty frame?

I don’t think that is really necessary as how much space does it take to say “make one big patch from the previous frame”

Let me tweak it

Each page can have its own separate canvas size/dimensions right?

Yes , which seem to be allowed to be zero for some reason..!

A lot of Dvorak layout users claim the improved layout ergonomics helps with their RSI

Changed it a bit. Maybe just best to keep it simple and not mention specifically using a patch or a blend mode. Worst case it would just be the last frame of an animation, best case the software allows setting a static page image

Right... I *think* that works?

Have_animation is always true for multi-page, otherwise there are no frame durations...

Ah right, was on mobile until now so reading the spec was... Less than ideal...

> If the duration has the value 0xFFFFFFFF, the decoder presents the next frame as the next page in a multi-page image. In case the current page contains an animation, that is, frames with a non-zero duration have occurred after the previous page boundary (or since the start of the codestream, if it is the first page), then this frame is ignored until the animation has been repeated as many times as implied by `animation.num_loops` (i.e. it is ignored forever in case of an infinitely looping animation). Animation loops restart at the beginning of the current page and the loop count is reset at each page boundary. Applications that want to render a multi-page image without animations (e.g. for printing or for displaying page thumbnails) render each page as it is just before the page boundary, i.e. including the contents of the frame with duration 0xFFFFFFFF.

how about something like that?

Sounds good to me

Sounds pretty clever, I like it.

ah great a version 3 of the spec -_-

If we were doing that we could add UltraHDR transcoding :P

LGTM 🚀

UltraHDR is a pointless technology no one asked for...

Yet another half baked format

Doesn't jpeg xt also have gain maps in a backwards compatible way?

So in other words it's just reinventing the wheel?

problem with JXT is they don't really like ISO formats, something something paywalled spec something something

So they concatenate two ISO/IEC 10918 bitstreams because ISO/IEC 18477 is an ISO format?

Also open source encoders are available

And they have money

They like paying for things. It makes them feel important

Especially when it's really official sounding and pretentious looking like ISO

It makes them feel soooo legitimate and professional and like a real boy

Like the fairy godmother coming down from heaven to grant their wish to be real

Oh won't you sign up your name? We'd like to see you're acceptable, presentable, oh respectable, a vegetable... And watch what you say! They'll be calling you a radical! A liberal! Oh, fanatical, criminal! Take take take it yeah! (sax solo)

Can you remove adaptive / --photon_noise_iso from a jxl again for further editing / conversion / compression?

it’s theoretically possible, but I don’t think the current tools allow it

Wouldn't be hard to add a decode option to skip noise, if we don't already have that.

Thanks, I found an issue for "jxltran" that already suggests this: <https://github.com/libjxl/libjxl/issues/871>

maybe some kind of advanced JPEG XL editor which lets you control all the intricate details

jxltran --allow_expert_options

Allow expert options really ought to be a compile time option 😂

Not runtime

Also if the original image has photon noise, technically you would be degrading the fidelity of the image by removing/omitting it

I would not recommend it. Whatever codec you're re transcoding to should be smart enough to detect it and replace it with synthesized noise

Well, that depends how accurate the noise was to start with

And you shouldn't do lossy to lossy anyways...

The noise is just as much a part of the image content as color or any other image feature

Which is why I wish lossless photon noise was better supported, two versions of the image for a few hundred bytes

What? Lossless photon noise?

Yeah

isn't noise inheriently random data ?

why would you need *it* of all things to be lossless

that's just burning bits to store effectively RANDOM.ORG data in your files

JXL noise is seeded, but either way I'm talking about the image data underneath

Currently it applies the photon noise in XYB instead of RGB for lossless encoding

I've seen more than enough artwork which is uploaded as "Lossless" on a Patreon only to have ISO 3200 noise splattered over the top making it a 50MB file for what used to be a fairly smooth shaded image

I don't know what you mean by lossless photon noise support

Photon noise on a lossless JXL

ahh

then its not lossless though

i guess you mean some kind of parameter that lets you set the photon noise and then it can be removed later as an artistic choice?

> Two versions of the image for a few hundred bytes

iirc photon noise is a per frame thing..?

you could have a 2 frame image

What would the point be of having a lossy feature put into a lossless file

Not even that, just change the colorspace with the image space to avoid the current red-blue noise it outputs

one frame without noise and the other frame being ``` All { Predictor: {Set 0} } Patch { frame: 0, width: image_width, height: image_height, x: 0, y: 0 } PhotonNoise { iso: 3200 } ``` ?

Not having a second lossless file that's 20x the filesize

That is another way to do it that's been described before

So it's a file that has the noise removed and replaced by synth noise?

lossless *and* we removed the noise!

A file that has synth noise instead of a noise filter applied in Photoshop for example

that would be ˜nice˜

how tf did that symbol happen

Is that a spec bug or a libjxl bug? Sounds like a libjxl bug

You have already traveled miles away from any semblance of lossless after getting those heavy duty filters involved

If I hallucinate hard enough, all files are lossless really <:galaxybrain:821831336372338729>

It's just a matter of perception

\> encode image to quality 1 jpeg \> re-encode image to png \> publish as "lossless" \> profit

> Using the additive noise AR, AG, AB and the strength SR and SG, the X, Y and B samples of each input pixel are modulated as specified by the following code, in which base_correlation_x and base_correlation_b are defined in I.2.3. Hmmm

so it only works for XYB?

no way to do RGB?

Don't blame me, blame the artists that insist noise is a vital component of their digital artwork :P

what do you think of my amazing music!

It is a bit annoying that you cannot mix XYB and RGB in a single image, but that would have complicated things too much. For the art-with-noise thing, you'll have to use d0.1 or something to get the nice noise.

Very beautiful song. All those frequencies are in perfect harmony

a very accessible foray into microtonality and polymetre as well

(or use lossless AVIF)

_ducks_

if you want really fancy artistic noise, you can also consider making some jxl art noise patterns and alpha blending or patch blending it over your image 🙂

updated 3rd-party jxl addon for Paint.NET integrates libjxl 0.11.0 in place of 0.8.2 previously PDN author wants jxl out of the box so updated working support in the addon helps get there

Photon noise looks pretty nice to break up the monotony of some images or shading. Similar to paper texture in a book.

So you might have a (near) lossless master image with photon noise that you archive and then want to create a web version from it. But when you recompress it with cjxl it becomes bigger! So you'd want cjxl to detect and remove synth photon noise, then reapply the same parameters when re-encoding.

That was my thought, not sure how many people actually use it though

There is a similar issue with AV1 too

For video it helps when you reduce the bitrate too much and things become too smooth, to add "faux detail" so the video doesn't look so synthetic

For pictures I imagine the same, also it's like better dithering

I mean say what you want about jpg artifacts, but the do create new detail!

Sometimes you never miss something until you loose it 😄

You made me stumble across this while looking for something else https://docs.amd.com/r/en-US/Vitis_Libraries/codec/guide_L2/kernels/jxlEnc.html

Seems to be an FPGA implementation

Oh nice it's a compute implementation not CPU or hardware

Ah

Looked into it more and no, this is a WIP FPGA implementation for silicon, unless I'm reading it wrong

Oh sorry, I assumed because of "_compute". But hardware makes more sense

What does the non-xyb noise look like

Is it a bug if a non xyb file has photon noise?

no

it just doesn't look that nice, since it gets applied to RGB instead of XYB so it makes much less sense perceptually

It mostly appears teal, but in brigher areas there's some red too

`--photon_noise_iso=64000` with just `-d 0` to make it RGB with lossless. Interestingly the encode speed also slows down massively

Weirdly it also allows an extra 0 on the value when JPEG transcoding

it's called photon noise, but it doesn't oeprate correctly on physical light data? 😭

It was designed for XYB, that is it assumes the second component is luma and that changing it does not change the color. This is not true for the G component of RGB.

Would it work well with permuted YCoCg? 😄

Better, I guess. But modular RCTs are undone before applying noise, sadly.

its both

> The AMD Vitis™ software platform is a development environment for developing designs that includes FPGA fabric, Arm® processor subsystems, and AI Engines.

well in theory, if it's designed for fpga, assume it will only be good on fpga

vitis specifically is, and I quote > Vitis Codec Library is an open-sourced library written in HLS C/C++ for the acceleration of image processing. It aims to provides reference designs for image codec algorithms that fit the Xilinx Alveo Series acceleration cards. The APIs in Vitis Codec Library have been classified into two layers, namely L1/L2. Each targets to serve different audience.

So is this going to be fixed in the decoder? It seems you'd just need to apply/covert the noise layer differently when lossless decoding

Interestingly the right looks more like digital sensor noise

I wish the film grain in av1 would look like photon noise. Afaik in av1 it's only 1 pixel large, so higher resolution changes the perception of it

I know film grain is different phenomenon, but imho photon noise is what you really want because it's a quantum phenomenon of the sensor size. Afaik our visual cortex constantly filters photon noise

film grain is more just like adding a special effect it ends up feeling like

I suspect it's somehow pleasurable to our visual cortex because we're always looking for detail, so detailed images have their own beauty. And filtering detail from noise is "fun" because we're so good at it

Sir Dejay, photon noise does exist in AV1 stuff. We've been using it for years as another form of different looking grain synthesis.

https://aomedia.googlesource.com/aom/+/refs/heads/main/examples/photon_noise_table.c

JPEG XL would be perfect for PDF/A

Huh maybe I got it wrong. I assumed the photon noise in jxl can be lower frequency or the "photon grain" can be larger than one pixel. Afaik if you take a photo with a 1" sensor, the frequency of the photon noise should be fixed no matter what resolution you scan. So the noise frequency should stay the same between 2160p and 1080p, which isn't the case for av1. But I think it isn't the case for jxl either

isn't it controllable ?

Well now I don't know any more. Theoretically the frequency or "size" of the photon noise should be dependent on focus length and sensor size, not on the resolution. Maybe I fooled myself thinking it looks different in jxl 😄

is photon noise based on IP address or by focal length?

frequency depends on resolution rather than focal length, AFAICT

also did i just say IP address lol i mean ISO

in the libjxl/aom implementation, ISO is just used as a proxy for the focal plane exposure that the image was probably made from

in principle, ISO should be more or less “10 lx·s ÷ ‘focal exposure of a mid-tone’ ”, and that will be true of the processed image generated by the camera (if you hold ISO and exposure fixed, the resulting image will look dark or bright on that basis; if you keep ISO fixed but exposure floating, it will try to target an exposure that makes the image look good; and if you keep the exposure fixed and ISO floating, it will roughly match the ISO to that exposure, such that the output image looks good)

but if you then take the raw image and develop it independently, the correspondence might break unless you recompute an appropriate ISO value (which a recent amendment to ISO 12232:2019 allows)

https://tenor.com/view/jfk-clone-high-i-like-your-funny-words-magic-man-jack-black-gif-18659433

I don’t really do photography

All I know is light + camera = image

https://www.iso.org/standard/79168.html > The ERS can be used when > - a camera records a processed file, either scene‐referred or output‐referred, or > - a camera raw file is processed. > > Additionally, in the raw file processing case, the ERS to be recorded in the file can be displayed by the raw processing software, to allow it to be set to a specific value by the user.

I do wonder if cjxl could check for an ISO EXIF tag to match photon noise too, but without the adaptive noise learning it probably wouldn't match (not actually sure if that code still works, seems to just use a lot of time and memory to do nothing)

one thing that would be hard to estimate the impact of is noise reduction (the code simulates the noise one would get without it, but it would be rare for there not to be any)

(also, there’s still that unknown factor by which the libjxl version is off)

posted on Reddit: https://www.obviousidea.com/announce_light_image_resizer_7-1_native_jxl_support/

https://github.com/Aleksoid1978/MPC-BE/releases/tag/1.8.1

> Added support for JPEG XL image files for the logo and album art (requires WIC decoder installed). Can't tell if they mean the broken Windows 11 one or this https://github.com/saschanaz/jxl-winthumb

I think it might just check for any WIC codec that registers itself for ".jxl"

commit is here: https://github.com/Aleksoid1978/MPC-BE/commit/d50af6fd4ccc18922b7abb200dc2c5c37fccf806

so what browsers for android current support jxl [Hmm](https://cdn.discordapp.com/emojis/1113499891314991275.webp?size=48&quality=lossless&name=Hmm)

probably Thorium and Firefox Nightly (maybe?)

could try regular firefox with a jxl extension

thorium works, relationship with cromite ended

Waterfox mobile doesn't

its not gone from cromite, it will restore it at some point and iinm is in debug builds. upstream chromium enforced a new parameter cromite author is afaik still researching how it affects his changes

existing or WIP, the integrations always need eyeballs and testing to confirm things work as supposed to - even early adopters lack understanding of many concepts introduced in jxl compared to the old formats and any help there can get wips and test branches reach stable and sooner

good cause thorium crashing for me now lol

Waterfox still crashes the tab when CMYK jxl is accessed.

lol

could be this issue https://github.com/Alex313031/Thorium-MacOS/issues/67#issuecomment-2448952447

https://www.waterfox.net/docs/releases/6.5.1/ > Updated the JPEG-XL library to the latest version. This should stop various websites that serve .jxl files from crashing in the tab they’re loaded.

that -

image hosting script Picsur freshly added jxl as a conversion target with sharable links (for 0.5.7 or next release). Its pretty much an imgur clone with autoconversion

https://github.com/CaramelFur/Picsur/commit/5b1f0a59bcd9e3a849ec51e92064cbc036d91a19

*Definately* doesn't have transcoding support, but nice nonetheless

Definitely

Definitely

Definitely

Defiantly

Deafeningly

Definitively

Distractingly

`def init(): ely()`

Include gain maps to jxl, great idea 🤦‍♂️

i hope this is hdr to sdr gain maps and not the other way round

HDR -> SDR makes a little bit of sense, something something artistic control, SDR to HDR gain maps only exists for backwards compatibility, which isn't a concern for JPEG XL

Seems no, not the only way

can you export one and send it

It was real hdr, no gain mapped

And this one is with "maximize compatibility' checked, twice in size, so probably gain map is there

Still no direct jxl support in Photoshop 2025, only via Camera Raw 😦

``` box: type: "jhgm" size: 3807293, contents size: 3807285 Gain map (jhgm) box: version = 0, 3807136-byte gain map, 141-byte metadata ```

not sure which way round that goes

I am guessing it's HDR -> SDR, as the `jxlp` box is the same size in both files

No, it is real hdr, iiuc

as in the jxl codestream is true HDR, and the gain map is to convert the HDR to SDR

It is odd that gainmap is called “maximise compatibility”

Where I would hope any jpeg xl enabled system knows how to gain map

hdr srgb is wild

srgb in general is.. not the ideal in 2024, tbh

It would be nice if they gave an option as to which way a gain map goes, but granted is HDR -> SDR really a "gain map" [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&name=av1_dogelol)

so I guess in this case, they do mean sRGB is the gamut, what transfer does it use? pq?

I hate that we still use srgb to refer to gamut tho

Sorry, I'm total noob in HDR, it never interested me because lack of monitors. For me gamut wider than srgb is much more important

It’s more just an artistic thing

the gamut of sRGB is *fine* it's not good, it;s not bad, it does the job for the most content consumption, the real issues with sRGB is the transfer, the transfer is a mess

Is sRGB transfer that one with the weird hump for CRT compatibility or something?

My hobby is nature photography, and it is more than easy to be limited by sRGB gamut. Actually sky is the first candidate to be clipped, second are flowers. Also I have an ~AdobeRGB monitor, and trust me, it is a day and night compared to usual sRGB monitors

well the real use of it is processing

it's not even that big of a problem

you just need to get calibrated

YOU are the problem

Unblocking you was a mistake, thank you for confirmation

Please wait… calibrating left eye

Hey 😦

<:clueless:1186118225046540309>

Probably if anything does it, it would be the Adobe software that created it in the first place. Other than that, I am not aware of any applications of jxl that take gain maps into account.

So "maximize compatibility" should really be called "make it look better on SDR displays in naive viewers that do bad color management / tone mapping, at the cost of making the file twice as large and not rendering in HDR anymore in nearly all applications that do support HDR jxl"

"Minimize HDR"

"Maximize file size"

Minimum quality, maximum filesize, less compatibility

That's the Adobe guarantee

"Include losemap"

What is the bitstream spec for the jhgm box

Continuing from https://discord.com/channels/794206087879852103/805176455658733570/1307041682952556564 <@688076786525143117> try these. I get the same blue image with broken Alpha handling

Copy & pasted directly from IrfanView with all three transparency settings.

It even copies alpha into clipboard.

Transparent background = keep alpha Gray background = apply alpha, I'm using #c0c0c0 background color. Black background = ignore alpha.

Oh wait, do you *have* color management enabled?

I don't use color management.

That'll be why then

CMS in IrfanView is broken in random way 😦 It works with transparency for old formats, but does not for jxl, webp, avif. Btw, it seems that avif always have some sort of alpha channel, even if it is not used, so avif always looks wrong in IV with CMS on

this popped up as a dependent of jxl-oxide https://github.com/google/neuroglancer/pull/653

thats a neat usecase and also a nice example on integration

> Once this merges, we'll start using JXL (lossless at the very least) on an industrial scale. Beautiful

Looks good too, automatic JPEG to JXL transcoding with (apparently) layered files https://github.com/seung-lab/cloud-volume/commit/40840a96d83c91c65141893212b1ba69de0547c2

Why not make it mandatory that the base image is HDR and the gain map is only needed for SDR? Isn't it simple to convert or "invert" the gain map?

You could make sdr base image illegal in the jxl spec and decoders...?

We could, but then recompressing UltraHDR becomes illegal.

Current proposed spec text has a note saying: > NOTE Since JPEG XL supports HDR baseline images, the recommended way for encoders to use gain maps is to encode the HDR image as the baseline image, using the gain map only to define a preferred tone mapping for SDR displays (or HDR displays with insufficient HDR headroom).

Is there not simple math to "invert" the gain map?

Or is it uglier than that

I know the loss of quality already happened when it was encoded as SDR with a low res HDR gain map

Isn't there a way to "invert" it without making it worse, since the loss already happened?

you’d make the SDR worse

since it would now go through two gain maps in total (original gainmap + lossy HDR reencoding + inverse gainmap)

not if you still want to be able to reconstruct the original UltraHDR file. Of course you could just convert the UltraHDR to real HDR and encode just that (with or without an HDR->SDR gain map), which is what I would recommend to do with such images, but if you want a lossless recompression of UltraHDR to be possible, then you need a SDR->HDR gain map where both the main image and the gain map are a recompressed JPEG.

Maybe ultraHDR was a big mistake and should be aborted before people are in too deep... 😂

I would recommend against reversible compression too because of how much space is wasted... or maybe if feasible and practical a pixel-lossless transform if not a bitstream-lossless one

Maybe only allow SDR->HDR gain maps for UltraHDR images

It would still save 20%, and if you require bit-exact reconstruction of the original file it could be a good idea. If you don't need that, then I wouldn't bother with gain maps at all and just encode an HDR image from scratch from reconstructed HDR pixels.

is there a way to convert the HDR->SDR gainmap back to a SDR->HDR gainmap as an extension to `jbrd`?

I feel like including support for both SDR->HDR and HDR->SDR gainmaps would overcomplicate the format

especially because SDR->HDR gainmaps should really never be a thing

Yes, I think it is, but phones are producing these things now and Android is actively pushing more phones to do it, so I am a bit torn between the need to deal with it and the desire to ignore it and hope it will just disappear.

like can you take an SDR image and the SDR->HDR gainmap and convert that to a HDR image and a HDR->SDR gainmap in a way which is reversible?

You cannot just change the gainmap, you then also have to change the main image, which would ruin any chance of reconstructing the original ultrahdr file

or maybe just declare UltraHDR images as not part of the de-facto JPEG spec

however this is a good point

currently how does the JXL gainmap box work? is it a jpeg or another jpegxl codestream?

or is it a seperate format

it's another jxl... which could itself be a recompressed jpeg, so in principle we could define a slight extension to the jbrd procedure (we'd need to define a new box for it to keep things backwards compatible) that reconstructs an UltraHDR by just reconstructing the main image first and then concatenating the reconstructed jpeg from the gain map box.

is it just a 18181-1 or can it be a 18181-2 ?

if its a 18181-2 it could have it's own `jbrd` box

That is funny and tragic at the same time

This is all 18181-2 stuff, the jbrd and gain map box are defined there, part 1 has no boxes at all, just defines the codestream itself.

I just wonder if there is a clever neat mathematical transform that can apply the gainmap in dct space maybe, and invert the gainmap.

Or, maybe some near-lossless, irreversible conversion would be acceptable

I don't think that's possible, since the gain map spec doesn't even define which exact upsampling to use for the gain map, so it's rather horribly underspecified...

Or even "visually-lossless" but with guarantees that it won't be a larger file

i mean is the jxl in the gainmap box allowed to be an entire `18181-2` file or just a `18181-1` codestream

First we thought only codestream, but allowing arbitrary nested 18181-2 file seems a better idea if we want to allow putting a recompressed jpeg there...

Oh my god it's worse than I imagined... well then you need to test how the most common implementation upscales it...

im pretty sure [libultrahdr](https://github.com/google/libultrahdr) is the reference implementation

so funny that they put "a true HDR image format" in their project title, as if they're trying to convince themselves

I think the bare minimum you need to do is a way to convert it irreversibly, but with guarantees that it won't be larger and even guaranteed transparent.

only problem with letting the gainmap be an arbitrary 18181-2 file is what should the decoder do if the gainmap sub-file has metadata boxes or its own gainmap

As long as you can guarantee that then it shouldn't matter if you can't get the original file back

It should die anyways

Really it should die

Which is better? Phone takes UltraHDR images Phone takes true HDR AVIF images

Hdr avif obviously

100%

But apparently it takes sdr heif

anything other than the main image of the gainmap sub-file just gets ignored for the purpose of the semantics of the gain map, but can be used in the reconstruction of the original bitstream — e.g. there will be XMP stuff there in case of UltraHDR

Even hdr heif is sdr

that's option 3, the worst of both worlds

ahh that makes sense

is it literally just two jpegs concatenated, JFIF and everything?

You don't want infinitely nested matroyshka dolls though do you?

my gainmap has a gainmap which has a gainmap ...

what about my usecase?

Why not keep it flat?

Enforce metadata flatness

maybe only allow 1 layer of nesting

Infinite matroshka is hellish nightmare world for decoder devs