JPEG XL

to pad some zeroes

naively, imagine that you have 2 sections total, and s2 takes 4 bytes in the TOC... then you cannot have your TOC+lfglobal section be exactly 17408+2+4 bytes: if you use a lfglobal of 17407, it's 2+4+17407, if you use a lfglobal of 17408, it's 3+4+17408

i.e. size jumps by 2

oh that's a nice idea

this is admittedly a contrieved example

another idea is to use name

instead

and just put null bytes

I think that's more future-proof as I don't have to use up one of the 63 extensions

yeah I was thinking of figuring out what sizes could be problematic but this idea is better

I just have to be a bit careful cause name_len is before a zero_pad_to_byte is invoked

you can also decide to potentially add a couple of bytes of padding to the second section but I am not certain that's better

I think that would require me to shift the second section downward if I need it, wouldn't it?

we could also decide to standardize a padding extension 😛

yep

I think it's probably easier to utilize name if I need to

probably not easier

at least code-wise

maybe

if I was going to use a padding extension, which one should I use? bit 63 or bit 0?

With hindsight we should have done toc entries as U32 without offsets, that would have made things a lot easier

As a padding extension, probably bit 62 would be best. Bit 0 might get an actual meaning some day. Bit 63 is the one we will use if we need more than 64 extensions to signal "more extension bits will come".

Alternatively we could just define bit 0 to mean "no real extension, just padding"

Since it looks like that would be a useful extension 🙂

So the issue with padding LfGlobal is that in a very unlucky case the padding would also make it need a 1-byte longer toc to store the padded size, but then the padding is no longer needed

But can't this issue be circumvented by pre-padding it so it's not exactly at one of those problematic sizes where that could happen?

Is there a way to write no-op toc permutations and do padding that way?

Looks like permutations can be no-ops, they just need to signal `lehmer[i] = 0` for every `i` in the range, and that range has a signaled size `end`. Signaling those zeroes could be done with a fixed prefix code that takes 8 bits per symbol. I think that should allow you to do a variable amount of padding, using a no-op toc permutation of the size you need.

No, color profile support is coming for Paint.net

Mhhhh well that won't work if you have a toc permutation though no?

No, though I guess if you only need to permute hf groups, you can still vary the amount of no-op lf groups, or be creative in how wasteful the prefix code choice is

Oh, another much easier option: use the container format, make a `jxlp` for imageheader+frameheader+toc, then put in a bogus box with room for padding, then another `jxlp` for the actual data.

no idea if isobmf already has a box type for "just ignore, this is padding", if not then I would propose to call it `padd` or something

Not sure how that will help

Ah, ok, gotcha

you can also put a metadata box there (Exif or XMP), they probably don't mind having trailing padding (or in case of xmp, you can just add whitespace to the payload)

I'd prefer to just write the codestream directly

using bit=62 for padding will probably be what I'll do

the `jxlp` option has an overhead of ~60 bytes (if you otherwise wouldn't use the container format), but has the advantage that you can afterwards just concatenate the two `jxlp` boxes and get a clean codestream without extension padding (or whatever other hack to deal with the unknown toc size)

so I would reserve a jxlp box worth of padding, essentially?

which has a header size of 12 bytes

and I'd write the TOC to that box

however, if for some reason I need to grow LF Global slightly, I instead don't write the box header and make it part of the box before it, which extends LF Global by ~12 bytes, is that right?

this gives me a lever of 12 bytes to turn on and off which I think is enough?

1. I figured it out: allocate enough bytes for a max-size TOC. 2. Encode image. 3. Compute actual TOC, except LfGlobal. 4. For each possible size of the LfGlobal TOC entry, you get a range of sizes for LfGlobal. Pick the one that lines up. 5. If none of them line up, that's because increasing the signalled size of LfGlobal can make ToC one byte longer, which decreases the required signalled size. If this happens, insert a jxlp box to decrease the padding by 12 bytes. Then try again, and it should work this time.

<@794205442175402004> is this the idea?

my idea was to do this: write `\0\0\0\0jxlp` [headers] [empty room for max-size toc + 8 bytes] `\0\0\0\0jxlp`[data] then when the actual toc offsets are known, you can write it where it should be and the two sizes of the `jxlp` boxes, and then you're left with a gap of at least 8 bytes, which you can fill with some dummy ignored box type of an appropriate size

so the actual codestream remains 'clean', no fiddling with padding in LfGlobal or extensions or whatever

the padding is done completely using the file format

Does the spec of 18181-2 say to ignore unknown boxes?

It doesn't say anything about that, but that is the only sensible thing a decoder can do, no?

Refusing to decode because there is some unknown box would be a bit much

Perhaps we should make it explicit in the spec that the presence of unknown boxes doesn't invalidate a file, and that they should be ignored

erroring out is also sensible if unknown boxes are UB

but explicitly instructing them to be ignored also works

Erroring out would not be a very future-proof behavior. Showing a warning, maybe. Perhaps we should do something like PNG and let it depend on the case of the letters of the box type

why would this line create an integer divide by zero?

```c (*flushes)->state_flushes[(*flushes)->pos++] = (StateFlush){token_index, value}; ```

does anyone know how this could cause a SIGFPE (integer divide by zero)

Could that be something aborting in a weird way?

valgrind says integer divine by zero, that's why I ask

AddressSanitizer says the same thing

For example: - first letter: capital=not critical, lowercase=critical (so Exif can be ignored since not critical but jxlc has to be understood) - second letter: capital=private (application defined), lowercase=public (part of the standardized spec) - third letter: always lowercase for now - fourth letter: uppercase=safe to copy (image editors can copy this box even if the image itself changed and they do not understand this box) So that's basically exactly like the png chunk naming convention except lowercase and uppercase are swapped (so we have `jxlc` while png has `IDAT` but both are critical public not-safe-to-copy chunks, and we have `Exif` while png has `eXIf` which should actually have been `eXIF` since Exif metadata is not really 'safe to copy': if the image changes, likely some of the Exif metadata should be updated too)

And then we could define a `Padd` box for padding - which is probably a useful thing in some other use cases too when you need to reserve some space (say an authoring workflow where you want to have metadata in the beginning of large files but you want to reserve some room for it so the metadata can be edited without having to rewrite the entire file each time.

We're not quite following that naming convention ourselves since everything except Exif is currently lowercase only, even though things like `xmp ` and `jxli` should probably be something else

So basically then a decoder can ignore boxes that start with a capital letter, but not boxes that start with a lowercase letter.

(and an editor that does anything more than modifying non-critical boxes should strip boxes it doesn't understand if they end with a lowercase letter but it is free to copy them blindly if they begin and end with a capital letter)

So you could make something like `MYbX` ("my box") that is ignorable, privately defined, and safe to copy, and decoders will not complain and editors can keep it around if they want.

The copying rules are probably the most useful bit of that scheme. For the optional vs required boxes, my view is that all required boxes must be understood by spec compliant implementations. If you want to add a new thing that isn't part of the old format, then either you need to have some kind of fallback so older decoders can still produce meaningful output, or you just need to make a new format.

Otherwise we end up with TIFF again, where only a small subset is supported widely, and everything else is vendor specific.

Extensibility is nice on paper, but unless your project is explicitly targeted at other people extending with no illusions of compatibility, then you can't ever use those extensions without breaking existng deployments.

Or rather, the few places where it can work require a single common library thst everyone uses, which can be common for application specific formats, but not so for sharable media.

so hydrium now uses a permuted TOC in order to prevent buffering as much

but that apparently increased the file size pretty dramatically

so I have to figure out why

also, it's still using a lot of memory. like 44 MB, which is... a lot

not really sure why

okay, so now I have an interesting problem

consider the following JXL file

djxl, ffmpeg, and mpv play it / read it fine

without warnings, either.

the GDK Pixbuf plugin fails to load it

as does magick display

this means Eye of Gnome, ImageMagick, and GIMP fail to read it

all are based on libjxl, which is what's confusing

jxlatte can decode it and doesn't whine, but it produces an image with the wrong colors (I suspect that's simply a jxlatte bug though)

why would this be the case, where djxl/ffmpeg/mpv can read and display it, but the other plugins can't?

I'll try with jxl-oxide later today

jxl-oxide decodes it without any errors

interesting, perhaps a bug in the gdk pixbuf plugin?

great, I feel stupid

I totally understand everything here 😊

actually the spec does say "Boxes with an unrecognized type shall be ignored and skipped by the decoder."

this is somewhat concerning, can you open an issue?

https://github.com/libjxl/libjxl/issues/2456

so it is explicit, good

It kinda hit me just now and I haven't found something about this after searching a while: what about blind users? Wouldn't it be awesome to have an image format that has alt text baked in? Like literally in the feature set so that platforms and tools actually take that into consideration when adding support. Else it would be a gimmick.

Technically it could be done by just adding an exif tag or such with the text inside. It's only recently I realised windows can read the bpp of an image if it had the metadata set

I think GPT-4 based screen readers that can just tell you what's in the image will be more likely to succeed than getting everyone to put alt text in their image metadata...

Good idea

I like the idea of human-embedded alt text - there are cases when an author/publisher wants to specify something that not necessarily would be the main thing recognized by an AI

Sure. But also: alt text depends not only on the image content itself, but also on the context.

I can hardly image a case in which context is really needed in the alt text. I know users with visual impairment that constantly tell people to only put in stuff in the alt text that is actually seen and that interpretation should be done in text outside of the alt text. After all, people that aren't visually impaired won't see the alt text but will get the context from what is outside the image + what they see in the image.

I mean, an image of a woman and a car could be all about the woman (e.g. it's a celebrity and the alt text would likely be something with her name in it, and nobody cares about what car model it is) or it could be all about the car, or it could be about both.

Imagine you're on Mastodon & there's a pic of some celeb on their phone. User #1: "check out this celeb spotted in Vienna" `picture of celebrity on their phone in Vienna` User #2, same image: "check out this celebrity using a Google Pixel" `picture of celebrity using a Google Pixel in an urban area` this might also complicate things for AI descriptions. I've posted a couple image comparisons for shooting RAW on my phone on Mastodon, and I write the alt text to try and illustrate why (subjectively) the manually edited RAW photo is better than the processed one straight out of my camera app

my description doesn't necessarily say anything intensely descriptive or specific about the image contents - albeit still while describing them - but tries to show why I prefer the manually edited RAW photo

https://elk.zone/disobey.net/@gianni/110268018488481066

Which pixel?

looks like a 7 non pro

Okay, fair. User #1 might not even know the model of the phone. However, I think if an alt text feature was in an image format, users actually caring to add alt text to an image will _hopefully_ tend to think a bit longer what should go in there to give it a general, extensive description of what is visible. I would lean towards that the alt text might on average be better than alt text on the net, especially social media, in which context matters. Also let's say User #2 took the image from User #1 that put in "picture of celebrity on their phone in Vienna" as the alt text. But User #2 is too lazy to use the alt text function on social media. The platform could then just parse the alt text that User #1 put in initially. I would say alt text with a bit of missing information is still better than no alt text. But yeah, platforms that parse the alt text from the file should still give users the option to set their own alt text in case the parsed one really doesn't fit.

Well, there is also a potential of misuse... That needs to be addressed in the society and can partially be addressed in the way software is designed that will allow saving alt text to the file. At least on social media I have rarely seen alt text misused.

If the context is visual I don't think that is a useful example

how do you describe these to someone who has never seen green 🟩 <:calibrate:930792680282259497>

If the person is colorblind, you could describe it in terms of colors they know. Like, you could say the right one is brighter and more vibrant, whereas the left one is dimmer and darker, with examples using a pair of colors they can see.

but if they dont know?

even with context there are so many assumptions

so imo just give up

it's also not always going to work in language

You can just use the description without the examples. I’d imagine even people who can’t see color are able to associate the ideas of brightness and saturation with certain concept or feelings that those colors might be meant to convey, just because they have heard other people talk about those ideas and concepts. Like, “bright” usually represents “lively”, and I think people who can’t see color would know that, even without fully understanding it, just because that’s how people talk about those colors.

🟤 my fav one is is this brown or dark orange

Though arguably, what you’d call a shade of a color vastly depends on the context the color is on. I remember a video from Tom Scott talking about this (and his hoodie), and there is also that one popular image with the cylinder on the checkerboard floor casting a shadow. Also see https://discord.com/channels/794206087879852103/824000991891554375/1097596416051916830

having descriptions for images is useful, just that enforcing it often doesnt work/help

Ah, I see what you mean. I feel like it’d be nicer to have a way to encourage it more actively, at least. Like, maybe show the input for it by default, with an explanation.

Color codes in worst case. But yeah, if they don't get a realistic concept from colors, they could still get numbers or might develop a understanding of when a term is used by people that see normally. Now I would actually like to get to know a young blind person cause I feel like I could learn a lot from them, lol.

hows JXL been going for the new

release

looking quite good, current git jxl performs slightly better than the previous release (0.8), which was better than the release before that (0.7)

Hi everybody, is it biased to use ssimulacra2 to measure the quality of JPEG XL compression compared to other compressions ?

biased toward what ? john could answer better but if I understand well, ssimulacra2 is optimized to fit faithfully to subjective (human made) assesments

as any automated tool, it is not perfect, jpegxl use butteraugli internally if I remember correctly

Ok thanks, i was wondering if ssimulacra2 was used at any point to optimise jpeg xl, which could favor it in comparisons with other codecs using this measure

libjxl was mostly optimized based on butteraugli, though recently ssimu2 was added to benchmark_xl so I don't know if currently <@532010383041363969> is looking at that or not — I assume he's still using butteraugli max/pnorm for tuning and only maybe looking at ssimu2 afterwards as a sanity check?

internally libjxl uses butteraugli at e8+, at lower effort it doesn't really use any metric (but the heuristics were tuned mostly for bpp*pnorm)

Ok, I’m only using libjxl right now so it should be safe then, thank you 🙂

this is so promising

I optimize on butteraugli, decide on my eyes -- if ssimulacra2 disagrees, I look three times again and attempt to doubt my eyes

I feel bad when ssimulacra2 scores go lower, but don't gate on it (also I don't gate on lowering butteraugli scores when eyes tell that that is the right thing to do)

well, it's been a few months, I think it's time to make comparisons of the last build and my current working build

well, the most important problem has been solved. now pixels from blocks do not touch other blocks in any way

But now I see a completely different kind of problem, which did not exist before, for some reason, colors appear that are not there. and they are of a wave nature -- to be more precise, it looks as if the color does not distribute completely. where it should be, color depressions and color hills appear. Instead of distributing the colors evenly across the block, the color for some reason gathers in one place, and does not reach another place

there are noticeable points, or even "places" in which there is a depression of lack of color, and hills in which there are more colors than necessary

I do not know how noticeable it is, but it has a periodic, next to each depression of color there is a hill of overabundance of color, it would seem that this color should fall like a wave into the depression of lack of color... but this is not happening

I have looked at each picture under a microscope, here is a new list of images that have noticeably worse look on the new version (future 0.9) than on the old one (0.8)

it should be noted that the very first image still looks just awful on the new version. I also noticed that the file size increased by 1.5% with identical parameters

original | old | new

there were also some problems with the RED, but it was not very noticeable, for example, if you switch the images, it will be noticeable...

But I found another way to make it noticeable

when I was making an image selection, in paintdotnet its blue selection filter is applied to the entire image, and then I saw.... I raised the color levels of all channels except RED and saw this original | old | new

for some reason, colors that do not play a special role in the image, greatly spoil it, instead of being smoothed by compression as it works on the old version

all tests on -e 9 -d 1

can you take a look at e7 too? I have the feeling that e9 is actually worse for lossy than e7 at the moment, due to not getting as much attention as default speed...

all the problems are completely identical. on all parameters from 5 to 9. By the way, now I can say that the problem is definitely not in the red channel, because after extracting it, I saw that it was the red color that worked almost perfectly. While all other colors behave terribly in the image where there is an overabundance of red. original | old | new 1. color correction 2. only red channel 3. all channels except red

how'd you plot "all channels except red" in grayscale?

subtract RED , convert to black and white , increase brightness

color drift mostly happens in the area around the edge?

subtract red from what

Just do the grayscale formula, only without the red . ( (0.3 * R) + (0.59 * G) + (0.11 * B) ). You'll need to normalize it so remaining coefficients add up to 1.0.

I mean I know how one *could* do it I'm just asking dzgas how he *did* do it

RGB

.....

RGB is three channels

He said that he replaced red channel with blackness, then grayscaled image, then increased value of every pixel by 33%.

where did he say that

🥴

It seems very clear to me: https://discord.com/channels/794206087879852103/794206170445119489/1107364440321556570 Just take away the red channel from RGB before converting to grayscale.

Is jpegXL better for streaming video over what twitch and youtube are doing?

Just very curious in terms of bandwith

No

It's not a video codec

no, JPEG XL is for still images

Under some definition of “still image”, I’d say. 😅

No, not at all.

jpeg xl supports animations, but each frame is compressed independently of the others. this means it will always have significantly worse compression than a video codec, since video codecs generally only compress the difference between nearby frames.

Does live streaming video use nearby frames for compression?

yes

it can't be used as extensively as in VOD, but it is still used

in VOD you can look ahead at subsequent frames for compression, in streaming you either need to maintain a buffer (and add latency) in order to look ahead, or you can't look ahead at all

it's really not intended for animation, that's basically there for feature parity with GIF tbh

it's still recommended to use a real video codec for video

There are some basic inter-frame things in the jxl bitstream (cropped frames / blending / patches) but there currently isn't any encoder that even tries to use that for compressing animation better.

It's not what it was designed for. For most video use cases I expect it to be a lot worse than actual video codecs even if you would make an encoder that makes more use of the bitstream potential.

For some specific use cases it is probably quite good though. Like very high fidelity video, like in digital cinema, where they currently use intra-only j2k

I noticed that `djpegli` inserts a `cICP` chunk instead of an `sRGB` chunk, even for sRGB output

is there a reason for this?

no fallback sRGB or iCCP either

I vaguely remember noticing that and I thought maybe we had fixed it, but apparently not?

Is PFM format supported by the ./cjxl and ./djxl tools currently? Tried to use the compressor to encode a PFM and got the following error(s) message Only JXL_BIT_DEPTH_FROM_PIXEL_FORMAT is implemented for float types. Invalid input bit depth Where can I change the bit depth setting, which seems to be the issue? I've looked in the encode.h file but it seems that the correct setting JXL_BIT_DEPTH_FROM_PIXEL_FORMAT was already selected.

it's supported in theory but there's a known bug that's been fixed in git main

nvm it hasn't been fixed yet

https://github.com/libjxl/libjxl/pull/2444

<@206628065147748352> I know that you are creating a decoder for Jpeg XL, could you help me structurally disassemble some(any) JPEG XL file in order to schematically draw it by a data block

what do you mean by drawing by a data block? do you want to draw varblock borders, or split frame data itself into groups (LfGlobal, LfGroup, HfGlobal, PassGroup) and draw the result?

like

but to do this using some specific JPEG XL image and take into account the number of bytes per block

i have one specific file

hmm well I guess it's doable with jxl-oxide (`jxl-frame`)

`Toc` has group offset and size info

if you can write Rust, you can use `jxl-oxide` crate, load frames (but not rendering them), then retrieve TOC from frame data and iterate over the group list

I guess libjxl also provides frame TOC

(I haven't used libjxl as an API though)

Toc is?

table of contents, it has how much size each group has

I'm not a programmer and I won't be able to use your software, but it would be nice if you clarified how to get the structural data of a jpeg XL file

maybe

Well, to be honest, so far I do not know what to do

afaik there isn't any ready-made tool that shows TOC information, so it would require at least some programming

<:Thonk:805904896879493180> this is bad news.

decoding TOC needs entropy decoding, and it's complex computational work

~~Perhaps I just need to find out on which byte or after which word this data goes and decode it manually~~

Shouldn't be too hard to make a verbose version of djxl or another decoder that just spits out the offsets

in fact, it is sad that the 8-layer xl was not applicable for its intended purpose due to the use of too strong compression. it was necessary to modify the encoder so that it would create only 1 single layer. because when progressive decoding is included in this way, after loading 40% of the image - the image begins to update on this layer - just line by line in 256x256 blocks

https://github.com/libjxl/libjxl/pull/2491 should fix this

thanks, left some comments

<@794205442175402004> with regard to PR #2444, it looks like it's blocked because of an unresolved conversation I created, but I don't believe I have the permissions to resolve it on github UI

but consider it "resolved" for lack of the perms

ok, merged it

yaaaay

then this can be closed I think <:CatSmile:805382488293244929> https://github.com/libjxl/libjxl/issues/430

thanks!

Well that would be nice. I was honestly really surprised that the JXL tools could not read its own format as input.

considering transcoding lossless-to-lossy is an extremely common and obvious usecase

I've heard some people pushing for JXL support in Blender, which makes a lot of sense. They're mostly looking at it as a replacement for OpenEXR and for rendering/compositing purposes, but I think it could be perfect for use in materials, where you have different texture/material maps for the same object. It would be convenient to store those in one compressed file than using .zips and such. This also could be useful for games for the same reason.

support in blender would be perfect

But, regarding games, would features like the progressive decoding possibly improve texture streaming performance? I'm 100% that a custom decoder would be used in a game engine scenario, but would it maybe make a difference either way?

it seems like just a matter of someone writing a proper implementation since I don't see any reason why the blender foundation would decline if they where handed code on a plate

I have thought about texture streaming as a use however it doesn't seem practical at all due to how GPUs work

it just wouldn't work

and if you tried to force it to work it would be slow and use a bunch of resources along with quite a few other problems

?

How so?

how texture streaming usually works from my understanding is the textures on the disk are already in a format the GPU can understand and also have mipmaps and the texture streaming process itself just involves being smart with how the data is uploaded to the GPU and rendered by the game by loading in lower mip levels and streaming in the rest of the mips until you have the full texture(s)

that fundamentally would not work with image format progressive or incremental decoding and would be very inefficient if you tried to make it work

I do see a use case for textures being stored as JXL however progressive decoding would just not work

So, the issue is the use of a compression format? So, any game that just uses jpgs or pngs is basically having to convert them all to raw pixel data before actually using them and that's why just stream it from a format like JXL just wouldn't cut it?

yep pretty much

Would it be possible to shift that load to the CPU, then?

I mean, it might defeat the point, in terms of performance, but it might be worth looking at.

the problem is you wouldn't be able to do anything useful with the progressive scans of a jpeg or jxl since they all end up being the resolution of the full image meaning they would take up the same size as raw pixels

it would probably be a better idea to just decode the full image then upload it to the GPU once which is what game engines thet use image formats like png and jpeg do

I like the idea of trying to make use of progressive decoding but sadly it's just not practical in this case

Hmmm.... maybe I've misunderstood (or just listened to people who misunderstood), but doesn't this feature also allow the dynamic creation creation of thumbnails? I've seen many people say that using JXL would eliminate the need for generating separate thumbnail files. Is that a different feature?

My thinking was using a different approach to mipmaps and megatextures in basically one go with JXL. (<@245794734788837387>)

https://cloudfour.com/thinks/on-container-queries-responsive-images-and-jpeg-xl/ isn't that related to this

it's the same feature. I may be wrong but from my understanding the progressive loading does not produce images that are a lower resolution just a lower quality and the benefit is you can generate these without reading the whole file

so if you have a 1024x1024 image and you progressively load it then it will still be 1024x1024 the whole time

it will just ramp up in quality as more data is available to decode

this ruins texture streaming because texture streaming works off of a texture having mipmaps inside of it which are just pre-generated lower resolutions of the texture

Hopefully its role in thumbnail generation can be cleared up. But, I get what you mean. Loading in small bits of the file's final "quality" allows for small amounts of data to be sent, but a GPU would still deal with the same number of pixels (unless other wizardry is introduced).

That's what I was hoping to see "replaced." That it'd be easy(er?) to just pull in the same of pixels or LODs you'd need for a texture or material dynamically. (<@245794734788837387>)

"Same?" Not sure what I was trying to type. Number, I think.

the possibility of dynamic thumbnails that people talk about with JXL is just having a program with smart logic to save on bandwidth usage by only loading a percentage of a image's data and then stopping when it has enough data to display a progressive render

say for example there is a website with a image gallery that has a lot of images you probably wouldn't want to load in the full data for each picture so it would only load enough to show a low quality preview then if a user wants to see the full quality image they just click on it and it loads the rest of the data

and a benefit to this is a server would only have to store a single file for each image instead of storing both a thumbnail image and a full image

Ahh. So, it doesn't *decode* to a different size, but just gets *displayed* at one. So, the transforms are happening in the same ways and places as with any image and is not format-speciric.

So, a thumbnail for a 4K or 8K image would still be *technically* 4 or 8K and would just be displayed smallishly in the browser (or app)?

that's my understanding of it however I have been wrong about stuff in the past (like in the past when I thought that lossless AVIF could only be in YUV) so I could be wrong in some form with this however I'm pretty confident that's how it works

Well, don't take this the wrong way, but I kinda hope you're wrong about this. 😅 But, this has been really enlightening. And it does confirm that JXL would be a good format for game development. Maybe someone could invent a "raw pixels" format that allows some kind of dynamic sampling, of there isn't (a good) one, already.

I was getting kinda freaked out when you were talking about texture streaming and saying "that's just not how GPUs work," because I had just recently posted a proposal that involves real-time procedurally generated textures and geometry and I was like, "crap! Will it not work?? 😰" Here it is, if you don't mind me shilling it a bit\: https://forums.armory3d.org/t/endgame-features-procedurally-generated-and-real-time-streamed-geometry-and-textures/5259

If anyone is familiar with any open source image viewers or image libraries that support ROI decoding or any other API that supports cropped/resized decoding, it would be interesting to see how existing APIs implement such features and to add similar functionality to libjxl

It can be both. For 1:8 the data is actually at 1:8 and we have to upsample it — obviously we could also skip that upsampling step. For progressive passes corresponding to 1:2 and 1:4, the easiest thing is to just do the full IDCT where the missing coeffs are zeroes, but you could also implement a partial IDCT that only uses the coeffs that are available and produces a lower res image.

Agreed. I didn't know what "ROI" stood for here, so I searched up "ROI decoding" and got a few papers and a github page regarding region-of-interest decoding for Neuro-imagining scans. \:O (Here's the Github\: https\:&#47;&#47;github.com&#47;neurostuff&#47;NiMARE&#47;blob&#47;main&#47;docs&#47;decoding.rst) But, it sounds like that's what I was thinking of when describing mipmaps and megatextures. Maybe we could ping John Carmack, now that he's free from Zuckerborg's clutches, and ask him. \:D (<@1028567873007927297>)

Even just having lossless JXL replace PNG textures in games would save a lot storage and download time. Scrap Mechanic had a 50MB texture for the UI atlas, it went down to 25KB

uhm, that sounds like absolutely awful PNG compression tbh

No doubt, but at least as a JXL you can't really mess it up

I'd be surprised if -e1 does nearly as good of a job 😛

Theoretically it could be made as an addon too, at least to start with. Eventually included with the other default addons, then implemented natively And as with games, even just having lossles JXL to save more storage would be good. Using layers for different map types is just a very nice bonus

I'll check if I still have the game installed

JXL would make a great replacement for PNG and JPEG, surely. Especially when we have better encoders that can match VP8L, or better lossy encoding (for both photographic and non-photographic).

If it can also be useful in weird edgelord cases, like stereoscopic images or medical images, or other weird uses, then that's a nice bonus I guess but I have come to learn that you can't get too greedy.

It would be nice if cropped/subsampled decode was more standardized and common. It would also be nice if there was a standard way for web browsers to request partial image downloads where appropriate for the screen size.

And then re-use partially-downloaded data when downloading the full image after zooming in

Browsers already do that for video. Request a set of data then as you progress through it, cache the old set and download the next 5 seconds, ect

So I don't have that 50MB UI texture, but I found another 25MB one... I think it's the first time I've seen e3 lossless almost double the size from e1 and e2

that makes sense, it's pretty far from photographic 😛

Well, lossless isn't really made for photographic either, since it uses modular

nah, e3 is pretty much made for photographic content

well, lossless photographic content, but still

Huh, interesting

If someone else has scrap mechanic installed and wants to try then the file is here, I'd rather not install it just to grab a file and delete it 10 seconds later `Data/Gui/Resolutions/3840x2160/Seat/gui_bearing_right.png`

As far as I know main direction of current game development in terms of textures is to eliminate CPU from the path at all. DirectStorage is aimed to do this, and I doubt GPU can decode JXL without CPU help. Correct me if I'm wrong in the last thought, please

Depends if hardware JXL decoders become commonplace, and I'll add DirectStorage is still rarely used if at all

It is used, first game using it came out this january, and I believe it will become much more widespread in few years

Huh, so you're right. Just gotta hope everyone switches to NVME drives and Windows 11 then :P

hardware jxl on desktop will never happen like hardware any image format

at the most, for ai like jpeg

https://www.theverge.com/2023/4/21/23692423/microsoft-directstorage-2-1-hdd-support-game-load-speed-times

Well, now they're just trying to prove me wrong at every turn

Ahh right, I saw about nvJpeg but missed that it was A100 only <https://docs.nvidia.com/cuda/nvjpeg/index.html>

Regading different maps on different layers, does JXL support per-layer compression settings? That could enable more savings without sacrificing desired quality. (<@238552565619359744>)

It supports per channel settings at least, namely for separate alpha quality. So it could probably be done

Worst case, generate them separately then merge into layered

Cool. That could work.

<@532010383041363969> Tell something?

Hello, I'm working on (re) embedding JXL into an image viewer. I was wondering if its possible to build only libjxl_dec/libjxl_threads from a CMake-only interface to emulate the existing behavior with the MSBuild system only including the decoder. Thanks

The MSBuild interface uses `/t:jxl_dec /t:jxl_threads`, and I am unsure how to do that in CMake

seems to be a memory leak in djxl

lemme reproduce it in valgrind and put up a bug report

https://github.com/libjxl/libjxl/issues/2509

Both `h_div8` and `height` depend on `div8`, so they overlap in the bitstream. However, `height` appears to depend on `h_div8`. Is this a mistake?

oh wait I see- if `!div8` then parse height as being that.

I see

It's a shortcut to define height either directly or via h_div8 and a default

yep yep

I'mma keep using this channel because I have a TON of questions.

What does `<bool> and <u(3)>` mean? Is it like a `bit and` but with only one bit?

sorry, I am a very literal person 😅

*when it comes to interpretation

chatgpt says to treat nonzero as true and 0 as false

ah, that's actually very fair, `!ratio` ougth to be `ratio != 0`

<@794205442175402004>

O: w00t w00t

that makes sense, thank you

Also, if it's reading only 5 bits for dimensions that are both divisible by 8, then is the maximum resolution that can be minimally stored this way `32x32`?

no, 256x256

has to be a multiple of 8 🙂

ohhh right

Oops!

No, it's correct as it is, isn't it? Or maybe make it `ratio == 0` to be more explicit

`ratio` decides the width if it's nonzero, so `ratio == 0` is correct, yes

Ah yeah, I got a sign wrong as usual 😛 I meant to just change the style of the comparison not the meaning

it's ok- i understood it

Am writing encoder :3

first attempt so far is... not great

unsure how to organise it

I'd suggest writing a decoder first

it gives you a stronger understanding of the bitstream

but i need encoder

What do you need it for, and why can’t you use an existing one?

my hobby project

and because funsies

Is JXL.js still the best way to decode jxl images with wasm

i give up lol

I was looking on a way to update jxl.js based on the last release of libjxl, niutech did its last update 5 month ago

for lossless JXL is there a reason Squeeze isn't used? iirc it can be lossless. or does it just not really save you anything

``` ==37872== Memcheck, a memory error detector ==37872== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==37872== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info ==37872== Command: ./ffmpeg_g -i ../newton-cradle.jxl -f null - ==37872== ffmpeg version N-110977-g5199c71ca9 Copyright (c) 2000-2023 the FFmpeg developers built with gcc 13.1.1 (GCC) 20230429 configuration: --enable-libjxl --enable-lcms2 --disable-stripping --disable-doc --assert-level=2 --enable-libplacebo libavutil 58. 12.100 / 58. 12.100 libavcodec 60. 16.101 / 60. 16.101 libavformat 60. 6.100 / 60. 6.100 libavdevice 60. 2.100 / 60. 2.100 libavfilter 9. 8.101 / 9. 8.101 libswscale 7. 3.100 / 7. 3.100 libswresample 4. 11.100 / 4. 11.100 Input #0, jpegxl_anim, from '../newton-cradle.jxl': Duration: N/A, bitrate: N/A Stream #0:0: Video: jpegxl, rgba(pc, gbr/bt709/iec61966-2-1), 480x360, 100 tbr, 100 tbn Stream mapping: Stream #0:0 -> #0:0 (jpegxl (libjxl) -> wrapped_avframe (native)) Press [q] to stop, [?] for help Output #0, null, to 'pipe:': Metadata: encoder : Lavf60.6.100 Stream #0:0: Video: wrapped_avframe, rgba(pc, gbr/bt709/iec61966-2-1, progressive), 480x360, q=2-31, 200 kb/s, 100 fps, 100 tbn Metadata: encoder : Lavc60.16.101 wrapped_avframe [out#0/null @ 0x76f28c0] video:17kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown frame= 36 fps=7.6 q=-0.0 Lsize=N/A time=00:00:00.80 bitrate=N/A speed=0.169x ==37872== ==37872== HEAP SUMMARY: ==37872== in use at exit: 0 bytes in 0 blocks ==37872== total heap usage: 67,576 allocs, 67,576 frees, 442,657,629 bytes allocated ==37872== ==37872== All heap blocks were freed -- no leaks are possible ==37872== ==37872== For lists of detected and suppressed errors, rerun with: -s ==37872== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) ```

Merge hopefully very soon :D

it can be lossless, but for nonphoto it tends to be counterproductive and even for photo it rarely gives a compression advantage, at least not one that is worth the slowdown of doing an extra transform

so it's more useful when used in a somewhat lossy way

it reminds me of those hybrid codecs which put a lossy image first

followed by the difference between the OG and the lossy

so it's both progressively loadable as lossy and fully loadable as lossless

but I found that losslessly compressing the residuals basically saved you nothing

sounds kinda like that

in audio, WavPack supports something like that

might be better for audio

DTS does it too iirc

another thing is that the encoder hasn't been tuned at all for lossless squeeze. It's likely making poor choices of predictors etc when doing this

the only real advantage of squeeze is that it gives you progressive previews, but in case of lossless you probably better just add some lossy downscaled preview frame and call it a day

makes sense

is there a way to make libjxl generate a preview frame atm

i.e. are there any tools to do that

I don't think so, actually we should probably do that if only to make sure that it properly works

Would help a lot with the massively slow progressive lossless images currently

anyway, what I think is the main use for Squeeze is things like d0.03 - d0.1 encoding, i.e. just a tiny bit lossy. In that range I think it can give better compression than VarDCT (and probably also better guarantees regarding generation loss and peak errors). Not sure if there are many use cases for that though...

hm, interesting to try comparing

you can try `-d 0` vs `-d 0.05` vs `-d 0.05 -m 1` to see if it makes sense on your images

interesting, testing `distance=0.1` and `effort=9` yields a 557k modular image and a 667k VarDCT image

they might not have the same quality though

ssimulacra2 reports 96.something for both

I checked that

however, effort=7 created a smaller file with VarDCT than effort=9

with similar ssimulacra2 score

interesting

d0.1 is probably where vardct generally already starts to make more sense than lossy modular

lossy modular is still better file sizes with ~96 ssimulacra2

what I find interesting is that VarDCT performs better on e7

than it does on e9

I feel like that kind of nonmonotonicity shouldn't be a thing

should I open an issue about that?

probably because at e8+ it really tries hard to hit that distance target, which is probably wasting lots of bits when the target is that high quality

remember, in vardct mode the image you get is different depending on effort setting

in modular mode, the effort only affects the compression density, not the image itself

yea, I'm aware

butteraugli_main reports 0.187 for e7 and 0.135 for e9

so it's a higher file size but it's more accurately hitting the butteraugli target

however, ssimulacra2 is reporting 96.5ish for both

``` $ cjxl -d 0.1 -e 7 orig.png e7.jxl JPEG XL encoder v0.9.0 e35880c4 [AVX2] Read 1080x720 image, 1281685 bytes, 103.2 MP/s Encoding [VarDCT, d0.100, effort: 7], Compressed to 627161 bytes (6.452 bpp). 1080 x 720, 4.23 MP/s [4.23, 4.23], 1 reps, 4 threads. $ cjxl -d 0.1 -e 9 orig.png e9.jxl JPEG XL encoder v0.9.0 e35880c4 [AVX2] Read 1080x720 image, 1281685 bytes, 100.9 MP/s Encoding [VarDCT, d0.100, effort: 9], Compressed to 682412 bytes (7.021 bpp). 1080 x 720, 0.26 MP/s [0.26, 0.26], 1 reps, 4 threads. $ ll e7.jxl e9.jxl -rw-r--r-- 1 leo leo 613K Jun 2 12:17 e7.jxl -rw-r--r-- 1 leo leo 667K Jun 2 12:17 e9.jxl $ ssimulacra2 orig.png e7.jxl 96.61789607 $ ssimulacra2 orig.png e9.jxl 96.77498696 $ butteraugli_main orig.png e7.jxl 0.1870112866 3-norm: 0.086522 $ butteraugli_main orig.png e9.jxl 0.1348828226 3-norm: 0.072864 ```

ssimulacra2 is less picky than butteraugli maxnorm

at least for high qualities, right?

well butteraugli maxnorm is not a very useful metric at all for low qualities

but yea because libjxl targets butteraugli I prefer to measure the output with an unrelated metric

does look like ssim2 also gives the e9 one a slightly higher score, just a less dramatic difference

I wonder how lossy modular compares for non-photographic content

and very low distances

to VarDCT

I expect lossy delta-palette to be the most useful thing for that, but that still needs work

meanwhile, I gave the lossless palette some attention, seems to be useful for recompressing png8 stuff: https://github.com/libjxl/libjxl/pull/2523

productive day today for me to make lossless jxl slightly better 🙂

just tested a specific sample, I found that Modular outperformed VarDCT in filesize at effort=7 with distance=0.05, it approximately tied at distance=0.1, and lost to VarDCT at distance=0.2. In all three cases, ssimulacra2 reported very similar scores

sometimes I feel productive because I threatened to merge a patch if nobody reviews it on the FFmpeg ML 😅

I'm half-kidding but I sent animated JPEGXL demuxing support to the ML in march

someone touched lossless 👀

that memory leak bug is funny

linux killed it not me

It still performs badly after that PR?

Interesting, git master ssimulacra2 reports "failed to decode image" for a jpeg created by jpegli

but reports a score of 89.something anyway

same with butteraugli_main

https://github.com/libjxl/libjxl/issues/2524

Yeah my bad, that's because it first tries to decode it as a jxl, which fails

It should silently fail instead of verbosely

what I find odd is it uses system libjpeg decode, instead of djpegli to compare

what are the default settings used for these, can't find them

run `cjxl --help`

libjxl defaults atm are effort 7, distance 1 (quality 90), progressive 0, epf -1 (i.e. let it depend on distance), and the rest 0

Thanks

FFmpeg now supports animated JXL <:poggies:853085814032302122>

on decode only atm

Hello there! I am currently attempting to compare JPEGXL to JPEG2000. However, I am facing difficulties in getting the JPEG XL encoder to meet my target bit rates through the use of distance and quality metrics. Even when I set the quality to maximum and distance to the minimum, I am only able to achieve a 0.5 bpp, which is much lower than my desired outcome. Can you suggest any way to directly adjust the bits per pixel?

the where is?

who stole, when did

cjxl.cc removed and tagetsize code just not transferred??

https://github.com/libjxl/libjxl/commit/15a5cc2dc280ff1b1345f4377bd5be00cbb2cd26 wtf

they just didn't rewrite it.

What image and command line are you using?

could just be that jxl is going into lossless mode

you can't target bits per pixel directly. what CLI options are you using? 0.5bpp seems very low, for most distances

can you provide the command-line you're using?

(do note, distance and quality are mutually exclusive, setting both will just use the distance setting)

<@794205442175402004> I submitted a pull reqeust for the website: https://github.com/jxl-community/jxl-community.github.io/pull/14

Merged it, thanks!

If the distance is set to 0 and it's still **lower** than your desired bpp, then it's actually outperforming while being lossless. As the others said, some extra info will help

I'll add, you can do decimals too, for instance -d 0.3 was what I used fairly often when testing

I think I mentioned the target bpp only being in the Benchmark a few weeks ago, didn't realise there was even a filesize target too

OkyDooky\: Since you are on Matrix, you may not be aware of the ping that was in the unbridged #adoption channel.Apple just added JPEG XL support to Safari 17 Beta! (https://developer.apple.com/documentation/safari-release-notes/safari-17-release-notes)

Ooh! Thanks for letting me know! It seems like the tide is turning, bit by bit.

what is the website where we can download binaries of cjxl that are up to date with current git ? I already saw it but don't remember

nice thanks !

I'm only using the -e and -d flags and nothing more. The effort is set to 7. The image is a PNG that has been transformed before being compressed by JPEG_XL. Interestingly, this only happens when compressing the transformed version and not the original raw values. Would it be possible to use an older release to get the target bpp feature, or would that yield a different compression performance?

it just does the same thing as manually changing -d to a value that matches the target bpp/filesize (or gets close enough)

what kind of transform was applied to the PNG?

Also, what distance settings did you use?

If you want to stay on a current version but also use bpp, then benchmark_xl still has the flag, but uses different formatting <https://github.com/libjxl/libjxl/blob/main/doc/benchmarking.md#:~:text=bits>

heads up <@794205442175402004> \: I expect WebkitGTK (and thus GNOME Web / Epiphany) to turn on JXL support soon now that at least one big player (Apple) has turned it on

looks like that tide might finally be turning, let's hope Mozilla plays along

cool, makes sense!

I tested some non-photo content on cjxl v0.9.0 e11e728, result look really great(d 0.5~d 0.3 e 7~e 8), But I notice some tiny artifacts still happen on red color area, other color area is great, only red area have those tiny artifacts, Probably I missing some parameter or forget enable features? I still hope can get more balanced quality for non-photo content. > cjxl -d 0.5 -e 7 --num_threads=12

If I recall DZgas noticed that on their album covers they were testing with

These are my covers, only I did tests on them

it's just that the problem is clearly and obviously here, and I hope the developers saw it - because it's not so difficult to fix it...

I tried using the tools from the v0.6.1 release with the target_bpp flag but still had the same issue with bpp maxing out at 0.5 bpp, even though the target bpp was set higher. I will try to use the benchmark_xl tool and see what happens when I compress it losslessly too.

The images are hyperspectral, first I run a reversible KLT/PCA transform on the spectral axis and then flatten the image into 2D and code them to PNGs. The image samples are 16-bit positive values.

oh, interesting. How many components do you have originally? It could be interesting to see what happens if you don't reduce the number of components, jxl can represent up to 2^12 channels...

(though you probably still want to do some kind of decorrelation transform since libjxl will only try to decorrelate the first 3 components)

applying the default lossy compression on these images is likely not a good idea since that is heavily tuned for visual images

(so it will take your data, interpret it as sRGB and take it from there, applying all kinds of perceptual tricks, but what you probably want to do is just store your data in a lossy way minimizing MSE)

I understand, I guess I need wait next quality improve to fix tiny artifacts issue. 🙂 And I notice libjxl have a new palette PR, maybe this heuristic can apply on lossy mode? some non-photo content apply varDCT mode will inflate file size, if encoder can avoid this situation, that will be very useful, I remember Delta Palette also a good choose, but still very experiment.

why even tho- like, why lol

Also I might try tackling an encoder again.

my buddy just got sold on jxl when I told them that it supports animation like gifs but doesn't have any "video" features

why would that sell them on it

Not turning things into a blobby mess is my first guess

you can do that with a video codec, just don't compress it as much

they want to be able to easily edit frames in an image editor

Oh right, good point. Never considered the fact of having to reference the first frame

Ok? You can just use all intra coding then 🤷‍♂️

not every instance of mp4 will be encoded that way- less work this way

it's not about availability it's about *how* available

Hmm, HDR GIFs...

most people don't know what that means! they just know that many image editing programs make it convenient to edit gifs.

and actually if you're only using intra coding then what's the upside compared to using jxl?

none probably

although iirc JXL does have some inter frame coding features in the spec that could be enabled in the future by a future encoder but aren't in use right now

If I recall it's still 'lossless' in comparison to the video codecs, but just reuses data from previous frames, feel free to correct me though

so does gif

Yeah

lossy JXL intra could replce prores tbh

Hell yeah

That's jpeg2000, right

Mjpeg replacement too, assuming you can sync the audio to it and don't run out of frames

I tried this a while ago by muxing a jxl sequences into mkv, it was quite the while ago, but it worked really well aside from decode speed, but libjxl was pretty immature then

well, I was comparing it to dnxhd

Probably a dedicated encoder and decoder for ultra high quality video usage, lower overhead and increase speed

I should try it again

If you have enough throughput and are willing to tolerate latency, you can use HW encoders to create the stream and decoders to view a downscaled preview.

just tried it again, it's pretty close but the decode speed penalty is pretty bad still the size makes it quite worth it though

though I didn't check the quality properly,

Svt is a faster encoder than jxl and currently is impossible to distinguish an svt and a jxl image

Jxl image looks as dark as svt

In higher zooming 270% and 300%

In high fps

Svt is fast

Though for real gains I think you need at least p3

0.1fps single pass i3 330m

And that sucks

I'm still at 14frames

Though svt doesn't try to sharpen the image at p3 is better than jxl e8 for medium bitrates

Qp24

Qp21 jxl wins but it uses high amount of sharpening and contrast enhancement

Jxl is better for blonde hair

For television pictures at s 7 d 0.52

For magazines at s 9 d 0.736

Or general photos at s7 d 0.493 dots 0 photon noise 84 i 0.832

S 9 d 0.664 is the optimal parameter for jxl

It tries more of what I said

Or you can add I 0.417 and epf 3