JPEG XL

Hey that's exactly what I was looking for actually, I switched PCs and was trying to find that link

sure, but no need to go for more complex things if the simple ones work 🙂 and libjxl doesn't decode to <8 bit anyway...

blue noise definitely looks the best, unless you compare it with error diffusion algorithms

And theoretically it's no more complex than a bayer matrix, you just replace the bayer matrix with a blue noise matrix

https://surma.dev/lab/ditherpunk/lab

https://surma.dev/things/ditherpunk/index.html

oooo

wtf 3 years old, how had i not come across that article before 😅

blue noise looks cool

https://tannerhelland.com/2012/12/28/dithering-eleven-algorithms-source-code.html this article is also good, it focuses almost exclusively on different error diffusion algorithms with nice comparisons

Just simple 4x4 or 8x8 ordered dither is good enough for dithering to 8bit sRGB or P3. I don't see a need to do anything fancier.

Well, it IS ordered dither, it's just a different matrix

and 8x8 is going to be significantly better than 4x4, even for a bayer matrix

and 16x16 or 32x32 is also going to be better but there are diminishing returns of course

and a matrix generated with the "void and cluster" method is going to look better than a bayer matrix

It would be a pretty trivial change to simply modify the matrix values, and possibly even the size of the matrix to make it larger.

It's ordered dither, that looks just as good as error-diffusion dither, without the cost of error diffusion.

Yeah, this is a good article, basically seems like Sierra is the best error-diffusion method.

yeah, it's definitely much less visible when the destination is truecolor or better

i think my raytracer does floyd-steinberg even to go from f32 to int8, just because i didn't want to implement two algorithms

Burkes dithering also seems better than 2-row Sierra

I guess I like 3-row Sierra and Burkes best.

As far as error-diffusion goes

I'm not disputing that, I am just saying that I would be surprised if the difference is perceivable when dithering to truecolor... but also I don't want to be the person that makes this decision and it should be <@532010383041363969> xD

Yeah, dither patterns would probably be hard to notice at that bit depth. probably.

Another big issue with dither is that it often changes the perceived brightness

since the dither algorithm isn't aware of what brightness means

or what a gamma curve is probably

Yea, and dithering in linear light doesn't work, because you don't want to turn 8bit linear light into sRGB

how perceivable it is first depends on transfer curve, primaries, and if the image is later transformed further...

worst case would be linear

how do you mean, doesn't work...

good dithering is most essential there

one aspect is that the amount of linear noise that dithers low values enough once back in sRGB will not be enough for bright values

conversely, if you use enough dither for the bright values, you’ll overdither the dark ones

The dither noise is only there to prevent rounding errors during bit depth reduction. When you then convert to a different transfer curve in high bit depth, it is not neded. If you then reduce bit depth again, you of course have to also dither again.

right, but so this means that dithering in linear light to prepare for 8-bit sRGB is not going to be effective

you usually don't apply dither noise at once for multiple transformed versions of the image, that would be mere cope for a bad processing pipeline

yes, true

the problem is not that you can't dither down to 8-bit data in linear light but you need the precision to be higher when going from linear light to sRGB

since the transfer function is nonlinear

you always do transfer conversion in high bit depth first, THEN dither down to the desired bit depth

well, ideally

indeed, that's why you can't dither to 8-bit in linear light

:D

you can

(if the intent is to end up with an 8-bit sRGB image)

ah

you can always have 8-bit linear light, but it's usually not what you want or the end goal

low bit depth linear is generally to be avoided

indeed. my point wasn't that it couldn't be done but rather it didn't make sense to do it

32-bit float linear as intermediate step in processing is fine

since dithering has to be the last step

yes

Dithering to 8-bit is essentially just a way to make the low-freq signal effectively 9 or 10-bit. It's not really important to do that in a great way, since the step sizes are small (unless you want to do HDR in 8-bit or something, but I assume this is only something you do for SDR spaces). Doing dithering to go from continuous-tone to bilevel (1-bit pixels, or e.g. halftoning) does require nice methods with error diffusion etc to avoiding visible dithering patterns. For going from rgb888 to 8-bit palettes or to rgb555 or rgba4444 it is still important to do it nicely, since the step size in these target bitdepths is still large enough to cause serious banding or color shifts. But 8-bit sRGB is already more or less banding-free at its reference viewing brightness of 80 nits. When viewing on a modern 400-nits display, the steps become a bit too large and you do get visible banding, so it is useful to apply some dithering. It has to be done during the final colorspace conversion (to the display space), which is why we needed to first add a decode API that can guarantee to return a buffer in the desired colorspace. Now we have that, we can add an option for dithering. <@179701849576833024> how about we add an option to enable dithering, and then we can decide later if we want this option enabled or disabled by default? (I'd say enabled, but I am open to arguments against that)

There's no option, it's just enabled 😛

Lots of early adopters were vocally surprised by the startling ugliness of libjxl due to the lack of dither. Most of them do not know that it isn't a problem with the compression itself, but just the decoding.

oh ok, I missed that it was done already 🙂

dithering should be done after resizing the image when zooming in or out, right?

Yes, mostly when zooming out

Zooming out will undo the dither

Always before/when reducing bit depth. Dither noise appropriate for a certain bit depth will not necessarily survive scaling done before the bit depth reduction at a high enough level to prevent banding.

If you resize to a smaller size you need to redither.

And a blue noise pattern is profoundly better than a bayer pattern from what I see.

yeah dithering should be done after

Dithering can't be done after

Well, it's done after resizing but before reducing bit depth

When you resize it should be in a higher precision to avoid banding

i mean dither after resizing at same bit depth

or dither after reducing bit depth

You have to do the resizing at a higher bit depth.

than the original

Then you dither when going back down to the original bit depth.

Same when resampling audio.

my problem is the wording of that

you cant dither when going down

you do it after

it doesnt make any sense

theyre two different operations

You dither before reducing the bit depth.

if you dither before then you can get banding

you need to do it after

it's more like if you dither before then the dithering is undone when you reduce bit depth

And often it isn't a separate operation. Often truncation is built into the dithering algorithm.

No, if you reduce bit depth before you dither, then the banding is already there and cannot be removed.

Unless you reduce the bit depth even further.

if there's no banding before and you dither then it wont do anything

and after you reduce bit depth then it will be banded

No, after you reduce bit depth, the dither you added will prevent the banding. That is what dither is for.

That's why it's there

It increases dynamic range

i think i understand now

so the noise has to be greater magnitude than a unit of the lower bit depth

The noise has the amplitude of half of the least significant bit step size in the target bitdepth.

So if you go from float32 in range 0..1 to uint8, it's basically `result = round_and_clamp( input * 255.f + random number between -0.5 and +0.5 )`

Exactly. It's between 0.5 and -0.5 the amplitude of the smallest possible step (the least significant bit)

you normally resize in float, practically no rounding errors

That dither signal is added to the signal first, and then the result is truncated to the target bit depth.

yeah but who resizes in float

that's expensive

Is it really?

yes

libjxl DCT is float and it's fast

bit depth reduction without dithering creates rounding errors that can not be removed by dithering afterwards

eh, please check Wikipedia or something on how dithering works...

float division is generally faster than integer division

yeah i already get it

read til the end

Most image processing applications do processing in higher precision before quantizing to lower precision as the final step

But not all the time

If you don't do things like resizing in higher precision, then you're gunna get banding

And if you don't dither before saving to lower precision, you'll get banding

video games always have banding

Because people don't implement cheap and easy dither into the render pipeline

Because people who make video games don't always know what they're doing all the time

some don't

By the way, this is the exact same dithering algorithm, "ordered dither," but one is using a bayer pattern and one is using a blue noise pattern. The only difference is the LUT. Check out this comparison. https://en.wikipedia.org/wiki/File:Michelangelo%27s_David_-_Bayer.png https://en.wikipedia.org/wiki/File:Michelangelo%27s_David_-_Void-and-Cluster.png

I know that at 8 bits the difference might be a lot harder to notice, but it's the same algorithm, only the LUT matrix needs to change. So I think this is enough reason to change it, even if the change will be hard to notice at 8-bit depth...

I mean look at how cool that is, the cool factor is there :D

I actually expected blue noise to be used, with the whole theme of perceptual uniformity throughout the codec's design...

one way to get an intuition about dither is to imagine a signal that is constant and equal to 0.4

you want to round it to integers

if you add uniform noise in [-0.5, 0.5], that means that 60% of the time, you end up in [-0.1, 0.5] and therefore round to 0; and 40% of the time, in [0.5, 0.9], which you round to 1

and 60% 0 and 40% 1 has the correct average, 0.4

you just happen to oscillate around that average

whereas if you were to round without dithering, you’d just get 0

a constant error of 0.4

you don't really need division to resize though 🙂 but then unless you do 16-bit, float *multiplication* is also faster, so...

(integers of course have a massive advantage for addition)

Blue noise is... a lot of coolness, with very little that needs to change.

if float multiplication is faster why does anyone make fixed point algorithms

Well... (a) on x86, not on other CPUs/hardware (b) there's addition too 😛 (c) with 16-bit integers, fixpoint *is* faster (d) integers are usually more HW independent

AFAIU at low bit counts multiplies are effectively quadratic in bit lengths (at least in terms of surface area), which would explain a few things (float multiplies are effectively 24-bit multiplies + some extra logic)

For video specs, fixed point makes more sense: dedicated hw can have the exact bit width needed for a given profile/level, and you need to spec bit exact decoding because errors would otherwise propagate/accumulate arbitrarily.

is 16-bit integer math actually faster than 32-bit on typical 32 and 64 bit CPUs?

one, no

a vector of those, not sure if it's faster *per se*, but it's also twice as many elements 😉

what a disappointing omission!

disappointing but unsurprising 😛

also the table is missing the magical instruction "(u16 x u16) >> 15"

(or maybe i16, not sure)

it can be pretty useful for fixpoint arithmetic 😛

a shift by 15? 🤔

i16 (_mm256_mulhrs_epi16)

yeah

it basically implements fixed point multiplication by any constant < 1 if using 15 digits past the "decimal" point

ah! that was listed way further down in the "other" section

they don't describe it with a bit shift heh > Briefly: for each signed WORD, calculate (1) * (2) / 32768, and set the rounded result to (3).

wonder why there's no unsigned version 🤔

The dithering is applied always, even if the source material was already 8 bits. Is this necessary? In lossless the dithering is no-op, and in lossy, as far I can see, the dithering only takes effect on compression artifacts.

I ask because part of the design of libhydrium is that it's supposed to be hardware-agnostic

which means explicitly no simd or intrinsics

in that scenario is there actually much to gain from fixed-point arithmetic?

depends on your autovectorizer...

well, on embedded ARM chips how true is what you said

basically, if I'm trying to avoid manually writing assembly, including the magical u16*u16>>15 instruction, I'm wondering if there's actual value in trying to stick with integers

Cortex-M chips?

some of the newer ones probably have pretty cool math instructions, I'll check

¯\_(ツ)_/¯

apparently recent ARM microcontrollers have straight up fairly normal SIMD support lol

huh

I'd have guessed floats would be fairly slow there

I guess the days of software floats are over 😛

it's an evolution of earlier DSP functionality, and it *seems* the logic block implementing it is optional, so customers who don't want fancy FP support don't need to pay the cost

mostly limited to 32-bit and smaller elements, as you might expect

well, that depends on the instruction - things like vpconflictd are actually easier to implement with larger elements xD

> Helium provides support for Q15 and Q31 numbers. There is a sign bit to show whether the number is positive or negative, followed by a binary point and then 15 or 31 binary digits. The 16- or 32-bit value is being used to represent the range from −1 to 1. For example, the decimal value 0.75 represented as a signed Q15 value would be the integer value 0x6000. Fixed-point arithmetic allows fractional numbers to be handled using only integer hardware. looks like it would probably handle the above discussed case well.

a couple years ago a 16-bit fixpoint DCT was 2-3x faster on "common android phones" wrt the fixpoint one

I assume you mean compared to the FP one?

err yep

Is VarDCT always 32 bit float, no matter what was the source bit depth?

in libjxl, yep

So I suppose the only meaning of the bit depth info in the file is to choose the more appropriate bit depth when decoding to png.

For me at least, VarDCT decodes faster than modular, so floats are not an issue, I guess.

seems like SVE has a rich set of 8x8 multiplcation instructions, including that magical fixed point thing. Intel can't compete.

now if only I could actually get my hands on a CPU that implements those...

I wonder if recent Qualcomm laptops have SVE2, since it's supposed to be mandatory in Armv9.

apparently Qualcomm laptop ships are way behind their mobile chips 🤦‍♀️

Yes, dither noise is mostly lost during lossy encoding, when reducing bit depth from float to 8 bit again, dithering needs to be done again.

although that's because modular is inherently branchy and hard to vectorize/parallelize

Modular _can_ be inherently branchy, if you use stuff like arbitrary MA trees and the Weighted Predictor. If you restrict yourself to a subset like what is produced by e1/e2 encoding, most or all of the branchy-ness can be avoided and you can have a specialized decoder that is faster.

``` In file included from /Users/jonsneyers/dev/libjxl/lib/extras/alpha_blend.cc:6: In file included from /Users/jonsneyers/dev/libjxl/lib/extras/alpha_blend.h:9: /Users/jonsneyers/dev/libjxl/lib/extras/packed_image.h:201:21: error: use of undeclared identifier 'JxlChunkedFrameInputSource' std::function<JxlChunkedFrameInputSource()> get_input_source) ^ /Users/jonsneyers/dev/libjxl/lib/extras/packed_image.h:201:21: error: template argument for template type parameter must be a type std::function<JxlChunkedFrameInputSource()> get_input_source) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ ```

getting this build error when trying to build on macos

``` /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/__functional/function.h:78:16: note: template parameter is declared here template<class _Fp> class _LIBCPP_TEMPLATE_VIS function; // undefined ```

nevermind, I had something old somewhere

Dithering also shows up when decoding losslessly transcoded jpegs, those were originally encoded from 8 bit source. That's interesting to see that the old jpeg encoding produced additional levels between the original 8 bit levels.

I mean, regular jpeg decodes don't take the inbetween levels into account, jxl decoder does.

If I recall, the JXL decoder will also use extra precision if the Jpeg's values haven't been rounded yet. Or at least something similar to that. Just woke up and the mention of it was months ago

correct, it’s possible to extract more than 8 bits of precision from the jpeg format

jpegli takes advantage of this, making HDR in traditional JPEGs potentially a realistic option

(when decoded by jpegli or libjxl)

(Actual HDR too, not just merging images or using gainmaps)

such JPEGs are already possible to make, and they’ll display as HDR in Chrome, but with more banding than if they were decoded by libjxl or jpegli

I would be curious to see one already, I assumed it had yet to be implemented into jpegli

unless I messed up, this should be one

maybe not the best example because the sky is a bit noisy, which provides a bit of dithering

(this is from just one shot, not a merged bracket)

Last I recalled Discord re-encodes jpegs if you upload them normally, might have to remove the extension

it doesn't re-encode (atleast the last time I checked) it just strips extra stuff like metadata

though "extra data" can be a lot more then just text

it’s preserved the ICC, at least

I couldn't find any noticable difference decoding with libjxl versus a standard jpeg decoder, although I did run my own test with a 16 bit Rec2020 gradient, turned into XYB with jpegli. Seemed to work, decoding back to png with jpegli resulted in no noticeable banding, but opening with normal jpeg viewers had severe banding (Unless that's just XYB without the extra bits anyway)

For some reason cjxl fails to transcode it too though

(Supposedly) 247 colors in the standard jpeg decode, 117,000 colors in the jpegli png output set to 16 bit (Still defaults to 8 bit where banding is noticable)

Original 16 bit Rec.2020, XYB Jpeg, 16 bit Jpegli Output

All naturally looking like blobs because Discord doesn't support ICC profiles

wow the original looks real bad in firefox

Yeah, not sure what's going on there but all seem a bit broken in Discord and Browser, locally works fine though (I don't have HDR but the banding goes away at least)

One thing I love is that the lossless JXL is naturally even smaller than the jpeg thanks to the gradient predictor

yea, that's true, but it's still an inherently sequential operation as a whole unlike VarDCT

interesting, discord strips jpegli --xyb iccs inline but not when you open in browser

for weesen.jpg? looks identical in firefox as it does in mpv for me

however my monitor is SDR (i.e. sRGB)

It has the ICC, Discord just ignores them when decoding, for whatever reason

I assume they meant the one I posted, which looks like a brown-green smear in Discord

oh that one, yea it looks like an orange -> geren gradiant but it's got terrible banding in firefox

Some of the banding gets mitigated at 100% zoom, including viewing locally to get a band free image for me, but there's still a difference between the images at least

why does the last one look so horrible?

oh wait it's not the last one it's the first one

In Gwenview, the JPEG has banding but the other two are silky smooth.

In Firefox, all of them have banding but the original looks butt fugly

weird

Also, jpegli --xyb produces 420 by default instead of 444. Unless you use a really long parameter to tell it to use xyb444

Isn't this a bug or oversight? since libjxl cannot losslessly recompress it unless it's 444

Also, jpegli --xyb seems to make images noticeably a lot darker.

I don't have a github account, someone should open an issue...

Or I should stop being lazy and create one...

Wow, what the hell? Here's another weird thing with libjxl

`cjxl -d 0 -e 8 Gradient.png Gradient.png.jxl` -> 17,775 bytes

`cjxl -d 0 -e 9 Gradient.png Gradient.png.jxl` -> 77.985 bytes

so effort 9 produces a file that is over 400% bigger than effort 8

How does that make sense?

This is using 0.8.2 by the way since arch linux is slow to update

The last image is the same jpeg decoded through jpegli and output to a png, so at least they worked like I expected

That would explain the subsampling values I saw while analysing the files, I assumed it was just an error from the tool not understanding XYB

Huh, really? I know there were issues transcoding in the past due to subsampling the blue but that was meant to be fixed

Huh, I used e 9 but forced the predictor to 5 for gradient only, assuming it would make the most sense. Got smaller than the default at least but apparently e 8 found a whole lot better

Yeah, why is e 8 producing a file so much smaller than e9?

it's not fixed. jpegli still produces 4:2:0 XYB unless you use a really long extra parameter to disable chroma subsampling

Huh... Guess that also means the gradient could be even smoother too

In latest git version, e8 is also much better than e9

Also the compression ratio is worse than 0.8.2

When building libjxl tools, should libjxl be built with jpegli and sjpeg support, or is it better to choose one or the other since they are both jpeg encoders?

That is weird. Can you open an issue about it? With the image for which this happens.

The image is the original here

Gradient.png

I don't have a github account yet.

And I wouldn't recommend waiting until I make one to file an issue...

ok thanks, I can reproduce the issue

wait I can reproduce with cjxl 0.9.1 but with current git I cannot even encode that image anymore: ``` JPEG XL encoder v0.10.0 7b1d243a [NEON] Encoding [Modular, lossless, effort: 9] lib/jxl/cms/jxl_cms.cc:971: JXL_RETURN_IF_ERROR code=1: skcms_Parse(icc_data, icc_size, &profile) lib/jxl/cms/color_encoding_cms.h:520: JXL_RETURN_IF_ERROR code=1: cms.set_fields_from_icc(cms.set_fields_data, new_icc.data(), new_icc.size(), &external, &new_cmyk) lib/jxl/encode.cc:1098: ICC profile could not be set JxlEncoderSetICCProfile() failed. EncodeImageJXL() failed. ```

I was able to encode it just fine on git build I made yesterday

maybe it's an lcms vs skcms thing

I see headaches coming here...

I mean Gradient.png

I'm adding thumbnail support on my lib for android, is there any way to create jxl files with preview or any existing files to test on ?

One way to do preview is partially load progressive files

I was looking for ways to try JXL_DEC_PREVIEW_IMAGE, as I already implemented a solution based on JxlDecoderSetProgressiveDetail

If I understood properly, decoding a jxl file up to the first group should give you a 1:8th thumbnail...

But I didn't played enough with the libjxl to give you more details

I notice jxl files have a tiny jxlp in the beginning and a much larger jxlp at the end

libjxl does this in order to put codestream-level metadata earlier in the file

JXL codestreams support embedded preview frames, but as far as I'm aware libjxl doesn't have an API to produce them yet

Ok, thank you !

You could open an issue to allow them to track the feature

What about progressive encode mode?

There's a few ways to do it. Depending on how much effort you want to put in, you could first try finding a thumbnail frame, if that fails then try checking for a progressive scan, if that fails then fallback to standard decoding (Or whatever you'd usually do)

Did a quick test and group size seems to be a large factor, at `-e 9` it was `-g 0 0.305 bpp` `-g 1 0.301 bpp` `-g 2 0.049 bpp` `-g 3 0.596 bpp`

Strange that it finds such an efficient path at 512x512, but nothing else near the rest

now that's interesting, but this is valuable insight to see what it's doing with a group size shift of 2

and to try to figure out why it isn't doing that with the others

i'll try to understand what is going on here. I'm thinking maybe some catastrophically bad choices in palette or channel palette decisions...

I remember I sometimes get better results with palette disabled completely

hm, in both the e9g2 (which is good) and the default e9 (which is very bad) it does apply a global channel palette on the R and on the B, reducing the 16-bit range to 1919 colors each. Looks like the G doesn't get reduced globally. Then in every group, in both cases it applies a local 3-channel palette, which has about 300 colors in the case of 256x256 groups and about 600 colors in the case of 512x512 groups.

<@179701849576833024> the only difference I can see between the 19kb e8 and the 77kb e9 jxl files, is that the e9 uses a different MA tree for the palette-indexed group data (it has 69 nodes at e9 and 61 nodes at e8, haven't checked how different the trees are), that apparently leads to a lot worse compression (2-5 times as many bytes in every group). Everything else (i.e. the global/local transforms) seems to be identical.

it's a rather spectacular case of tree learning producing a worse outcome at a higher effort, which might indicate a bug...

the thing with MA tree learning is that whatever tree the encoder uses, we won't detect a bug because it will still be a valid bitstream that losslessly represents the image data, so functionally nothing is wrong except compression becomes suboptimal — though even with a really bad tree, ctx clustering will limit the damage...

I mean, it's a heuristic, so not sure if we are that lucky xD

yeah but it's supposed to be based on an entropy estimate that should be more or less accurate, no?

I really should get down to do that MA tree variant where a bunch of layers are computer at once (and all use the same property)

Yes, but it's greedy

So you never know if the second layer would be able to do magic

yeah, I just didn't expect the greedy result to be able to get that much worse than another greedy result (where presumably the bad greedy choice is simply pruned from the search space because of the lower effort, so by luck it arrives at something better)

That's greedy algorithms for you 😛

this means that the potential for some less greedy approach to compress better is more significant than what I assumed

The nice thing of doing that is that you also get to decode faster

yes, so it could lead to both better compression (because less greedy) and faster decode. Until now I was assuming that it would be mostly useful for faster decode at the cost of worse compression (because no arbitrary trees but some forced structure), but now I'm starting to think that the advantage of seeing more than one binary split at a time could very well outweigh the issue of no-arbitrary-trees — especially because we're doing ctx clustering at the end anyway, so it doesn't matter that much if the tree is bigger than it could be when not forcing a structure that allows avoiding branching by doing a bunch of layers at a time as a lookup.

<@853026420792360980> will you give a shot at fixing the gdk pixbuf?

I can put it on my to-do list but I can't necessarily get to it today

I did figure out what the issue was last night (per the comment on that issue)

yep I did see that

and it's both a bug in libjxl and a bug in the plugin

😭

I'm prepping for a meeting that's in 1.5 hours. depending on how long that meeting lasts I might have time to get to it afterward

btw wrt https://github.com/libjxl/libjxl/issues/3131

the issue with *this one* is that the ICC is invalid but cjxl doesn't notice that

it occurred to me in general though that the design of the ICC transform to make it more compressible might not work for invalid ICCs

and I'm wondering what the general idea for that is, in general

e.g. in the above scenario, with an invalid JPEG ICC

(1) refuse to losslessly transcode and error out? (2) refuse to losslessly transcode unless --strip is passed? (3) assume that an invalid ICC is not present, and encode as sRGB, with a warning (4) possibly store the invalid ICC in the jbrd box somehow so it could be reconstructed anyway?

I don't know how futureproof jbrd is

wdym not work?

I mean, the forward-transform on the ICC profile to make it more compressible assumes the ICC is valid, right?

what if it isn't?

for example, what if there's a length mismatch? what if it's too small? etc.

these sorts of invalid ICCs can be embedded in png and jpg files in the wild, and can be passed to the library

Mhhh I think it will just do stuff that gives you a less compressible icc, it shouldn't fail

in the above issue it produces an invalid jxl file and reports success

which is definitely wrong

I do agree that cjxl should probably give an error

in case the ICC profile is broken

We don't really specify that a jxl file with a broken ICC profile is invalid, or do we? In principle we can still represent such a jpeg file and reconstruct it too.

welll things such as modular use the ICC to read channel counts no?

modular uses the ImageMetadata bundle for that

The ICC isn't actually needed to decode an otherwise-valid JXL file - you can decode the compressed entropy stream and ignore what's in it

the problem is that in the image above, it doesn't attempt to ignore what's in it and instead de-mangle it, and that process fails

mhhh fair point

I think three things: 1. It would be better for a decoder not to fail when faced with an ICC profile it cannot understand, but produce a warning and replace it with the assumption that it's sRGB or something. 2. It would be better for an encoder to at least produce a warning when it gets invalid ICC input (could be a warning if it doesn't need to convert colors, an error if it does) 3. We should probably consider jxl files with an invalid ICC profile as invalid files, so decoders can do whatever (refuse to decode or do something best effort). In particular, libjxl could still reconstruct jpegs but refuse to render them to pixels.

Well, now that's interesting

``` ==18868== Process terminating with default action of signal 11 (SIGSEGV): dumping core ==18868== Access not within mapped region at address 0x1 ==18868== at 0x668133D: JxlDecoderGetICCProfileSize (decode.cc:2261) ==18868== by 0xF26C26B: load_increment (pixbufloader-jxl.c:426) ```

the line number is wrong though

this is the line that's erroring out

``` if (size) { *size = jxl_color_encoding->ICC().size(); } ```

this implies that `0x1` is being passed

but the pixbuf loader is passing an address

like it's using the & operator to pass the address of a stack variable

so it shouldn't be passing 1

oh, I see what's happening, this is the ABI break that occurred between 0.8 and 0.9

is it loading the wrong libjxl at runtime?

it must be loading the system libjxl, which is 0.8.2

but the pixbuf loader I'm testing is not that

this is the ABI breakage where the `JxlPixelFormat *` argument was removed from the middle of a function signature

however, using LD_PRELOAD=`build/lib/libjxl.so` doesn't appaer to help

what's `ldd` saying?

and running `ldd` on the pixbuf loader is saying it's already linked

ah

perhaps it linked the system one at build time ~statically?

kinda weird though

``` $ ldd build/plugins/gdk-pixbuf/libpixbufloader-jxl.so linux-vdso.so.1 (0x00007ffd10563000) libjxl.so.0.10 => /home/leo/Development/libjxl/build/lib/libjxl.so.0.10 (0x00007efdf4200000) libjxl_threads.so.0.10 => /home/leo/Development/libjxl/build/lib/libjxl_threads.so.0.10 (0x00007efdf45a3000) libgdk_pixbuf-2.0.so.0 => /usr/lib/libgdk_pixbuf-2.0.so.0 (0x00007efdf4533000) libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0x00007efdf419e000) libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x00007efdf4052000) libc.so.6 => /usr/lib/libc.so.6 (0x00007efdf3e70000) libjxl_cms.so.0.10 => /home/leo/Development/libjxl/build/lib/libjxl_cms.so.0.10 (0x00007efdf3e30000) ``` plus some additional system libraries

yeah, seems right

how are you running the loader?

perhaps whatever you use loads libjxl.so for other reasons

``` LD_PRELOAD=build/plugins/gdk-pixbuf/libpixbufloader-jxl.so eom red-room.jxl ```

I'd strace to see when libjxl gets loaded 😄

I'm not sure how to do that, can you please elaborate?

something like `LD_PRELOAD=build/plugins/gdk-pixbuf/libpixbufloader-jxl.so strace -ff eom red-room.jxl 2>&1 | grep -C 5 libjxl`

oh well that's a lot of system calls

ah yes, yes it is

okay I can see the system call where it opens /usr/lib/libjxl.so.0.8

it's right after `[pid 20601] openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 14`

and then an mmap and then a close

more specifically

``` [pid 20601] openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 14 [pid 20601] newfstatat(14, "", {st_mode=S_IFREG|0644, st_size=171219, ...}, AT_EMPTY_PATH) = 0 [pid 20601] mmap(NULL, 171219, PROT_READ, MAP_PRIVATE, 14, 0) = 0x7f74c8133000 [pid 20601] close(14) = 0 [pid 20601] openat(AT_FDCWD, "/usr/lib/libjxl.so.0.8", O_RDONLY|O_CLOEXEC) = 14 ```

LD_PRELOADing the built libjxl doesn't change it

and just to be sure, it doesn’t load libjxl.so if you don’t load the plugin, right?

if I don't LD_PRELOAD anything then it loads the system libjxl and system pixbuf loader

(which works fine)

This might be more of a bash question, but is there any way to output a jxl with the same filename as the input, just with the .jxl extension? The plan is to have input be *.jpg and there's only one in the directory

I tend to use `parallel` even then

```console $ parallel cjxl '{}' '{.}.jxl' ::: *.jpg ```

ooh thanks!

but if you don’t want to do that: ```bash shopt -s nullglob for f in *.jpg; do cjxl "$f" "${f%.jpg}.jxl" done ```

yea, generally speaking you use `"${var%foo}"` to expand `"$var"` but with a trailing `foo` removed

useful for replacing file extensions with another one

nice ! in zsh you can do `${var:r}.jxl`

tysm!

Hi guys, what does `uses_original_profile` acutally mean? It seems changing this parameter in `libjxl` can affect the result size a lot.

if `uses_original_profile` is set to *false* it tells libjxl to use XYB-encoding when encoding the output image

which means that it encodes it in the XYB color space, which is an absolute, perceptual space

this is typically what you want to be doing for lossy encodes

`uses_original_profile` set to *true* disables colorspace conversions on encode

which is usually not what you want, unless you're encoding losslessly

if you're doing lossless it's required to set it to true

but otherwise you usually want to set it to false

so if the input is raw RGB values and `uses_original_profile` is set to `false`, and still the process is set to lossy encoding, what will happen? I've tried some tests, sometimes `uses_original_profile=true` generates larger file and sometimes doesn't

If it doesn't convert to XYB, does it process the compress in RGB mode?

if `uses_original_profile = false` it will use XYB encoding the file size may be larger or smaller based on the number of factors, but XYB should give significantly better quality at a similar filesize for lossy

Oh sorry, I mean `true`

that's correct, if it doesn't convert it to XYB then it just leaves the pixel data in the original color space

that's why the option is called `uses_original_profile`

profile referring to color profile

Thank you, this solves my question.

do note that XYB is an internal optimization

which you typically want enabled

libjxl does the conversions for you

so if you ask libjxl for pixel data from an xyb-encoded jxl file, it will return RGB

in whatever space you request (which is probaby sRGB, or the tagged space)

sure, now I understand this parameter.

I dunno what the desired behavior would be in this case. Produce anything else than the original jpeg file and you break the promise of lossless JPEG recompression...

How do you attach XMP to a jxl after lossless transcoding?

and if you do that, should you even expect it to be preserved when converting back to jpeg?

I guess we could try to identify which boxes have been ignored and warn about ignored boxes, but that'll get tricky to implement

It's unclear that the user in the linked issue uses a lossless JPG transcode though

Just add a box with some isobmff tool

> I have jxl image with integrated xmp data inside (lightroom). After conversioning jxl to jpg this data (editions history, keywords, faces and e.t.c.) is gone from jpg file It's possible they're using pixels-to-jpeg

since they referenced lightroom, which just supports JXL

tho it *is* a reconstructed jpeg

I just checked the sample in the issue

not sure why lightroom is exporting jxl as a recontructed jpeg

looks like that's what happened the box layout of the JXL file is `JXL `, `ftyp`, `jxlp`, `jbrd`, `jxlp`, `Exif`, `xml `, EOF

That's kind of weird indeed. It can also just output real jxl...

If you got that image by exporting from lightroom, then yes

why does ubuntu msan randomly fail

e.g.

weird. seems like the failing test is third_party/zlib/test/example.c

there’s an upstream bug to make it optional https://github.com/madler/zlib/issues/308

but the maintainer is reluctant

(I almost wrote “obsolete” instead of “optional” – Freudian slip?)

(I actually did write it, but caught myself in time before sending)

it may be a good idea just to not include zlib as a sumbodule

how many systems don't actually have it built-in

even windows has deflate built-in although I don't know if they expose a zlib-like API

You cannot do msan tests with system libraries

Or rather, you can, but they will fail

let's see if updating to a more recent version of zlib helps...

Pretty simple imo: Always transfer JPEG metadata to JXL metadata and all possible JXL metadata to JPEG metadata, and nothing else. JXL should have no special knowledge of the original JPEG metadata.

JPEG bitstream reconstruction is designed to reproduce exactly the same JPEG file. You would change that and make it produce a different JPEG file that is like the original JPEG but with extra metadata? I think it might be a bit confusing to keep the `jbrd` box around (which is supposed to allow you to reconstruct the exact same jpeg file) while also changing or adding metadata (or changing the image data for that matter), which means you can't reconstruct the exact jpeg file any longer anyway.

At best, it'd have to be with a warning or specific flag, at worst it would break systems assuming the file to be bit-identical

If you want to add metadata to the JPEG-transcode JXL then yea you'd have to convert it back to JPEG, add metadata, and then re-encode it to JXL

at the moment

theoretically libjxl should be able to modify the file to prevent the roundtrip

but atm it can't

Maybe instead of making jpeg reconstruction a "succeeds or fails" thing, we could also allow it to produce a jpeg file with the same image data but with different metadata than what the original had. That is, currently if you take a jpeg with xmp metadata, recompress it losslessly as jxl, then modify the xmp so its number of bytes changes, and then try to reconstruct the jpeg, it will just fail (xmp size does not match what was expected for reconstruction). We could instead make it still work but produce some warning return code...

Right, I don't think anyone really cares about getting the bit-exact input back, as long as the new JPEG is still decoded identically, including essential color metadata, and non-essential metadata as well in case of a simple JPEG -> JXL -> JPEG process... The ability to compress and completely reconstruct the original JPEG sounds quite impressive, but a bit like a waste of space as well... And also like a privacy risk, as metadata can be preserved that the user did not expect to be preserved. The fact that this jbrd box is not essential and can be simply removed is good to know, though... Is there a flag that causes cjxl to not create it in the first place?

being able to get the bit-exactness of input back does matter to a lot of people for archival purposes

but why

for archival purposes

the people who care about bit-exactness of the input file aren't going to be the same people who will be modifying the metadata of the transcoded JXLs

why would those people mind lossless recompression

but they're both important use cases

for archival purposes

how is anything but the image data and metadata relevant

for archival purposes

those people wouldn't use JXL in the first place

they would if it guarantees bit-exact reconstruction

which it does

what you're basically saying is that for *your* purposes, a perfect reconstruction of the original file is not necessary but that's not the case for everyone

are you thinking "for archival purposes"? 😁

Dropbox built [Lepton](https://dropbox.tech/infrastructure/lepton-image-compression-saving-22-losslessly-from-images-at-15mbs) to save space in their cloud storage service, where integrity of user data is essential. Other use-cases could include preserving cultural artifacts or storing digital evidence.

huh, lepton GH page suggests using jxl

I don't use cjxl for photos because I want to keep the camera metadata

but if jxl is meant for photos...

it's fail

doesn't it?

should be a bit perfect reconstruction of the original file

not for non jpeg reconstruction

are you talking about camera raw -> jxl?

what does jxl do here that jpeg doesn't

exactly they all suck

the camera metadata is usually Exif isn't it?

yeah

which both formats support

if your camera raw software can't export the exif metadata properly that sounds like an issue with a specific camera raw software, not an issue with jxl or jpeg

is there just nothing that can read it all?

I guess exiftool simply lacks support for that, it's not fundamentally difficult to do especially when it's in a container

I mean, png and jxl are different formats so exiftool needs to learn more about jxl to support it

but having libjxl to convert it back?

well libjxl does know that

yeah and it's still all gone

uh I seem to have missed the context then. so you meant that png -> jxl -> png doesn't retain exif tags while some tags are retained, right?

yeah so that makes it appear that exiftool is reading all that's in the jxl

and it's simply cjxl is not copying it all

(and the exiftool warning seems to suggest that this might be the issue of the file itself, see the warning)

I see

grr what kind of intermediary should I be using if tif isn't supported

I tried to look at exiftool but I can't pinpoint the exact origion of that warning message... wat

the metadata is similar(mostly gone) when i use magick directly from tif to jxl

this is a sucky experience

jxl is just like webp!

ugh found one, https://github.com/exiftool/exiftool/blob/2766977eee43be0f2ac602e7e42fb12c54b33e35/lib/Image/ExifTool/PNG.pm#L1579

(did you know that exiftool was written in perl?)

so it's simply warning `tEXt` `zTXt` `iTXt` `eXIf` tags coming after either `IDAT` `JDAT` `JDAA` tags

https://github.com/libjxl/libjxl/blob/1f9b04f743530c47ef9c0feba31e164ce7390f73/lib/extras/enc/apng.cc#L109-L110 ultimately the culprit turned out to be this

so libjxl does correctly read `eXIF` chunk, but doesn't yet write `eXIF` chunk by itself

so im not crazy

hmm.... for me it seems to pass around exif metadata. <@288069412857315328> could you try using `cjxl --compress_boxes=0` or using more recent exiftool (support for jxl brotli compressed boxes was added in exiftool 12.63 (June 8, 2023), and also requires perl IO::Compress::Brotli module)

maybe it's a regression in v0.10; on 0.9 the procedure `jpeg -(convert)-> png -(cjxl)-> jxl -(djxl)-> png` preserves exif till the end

it doesnt pass all of it

yurume found the issue

https://github.com/libjxl/libjxl/pull/3218

thanks 🙂

looks like all the vcpkg builds failed, but it's not clear why to me or if my change had anything to do with it

CI seems really fragile

> D:\a\libjxl\libjxl\lib\jxl\enc_icc_codec.cc(241,1): error C3493: 'kSizeLimit' cannot be implicitly captured because no default capture mode has been specified [D:\a\libjxl\libjxl\build\lib\jxl_enc-obj.vcxproj]

yeah not your fault

I thought that that might happen

https://github.com/libjxl/libjxl/commit/f5a4d64e5f91069f76ac086984053a1264e885c1

looks like this is the culprit

yep

prior commit passed

I don't really see the point of having all these CI checks if merges that break them will happen anyway

writing C++ for multiple compilers is the stuff of nightmares, lemme tell you...

did you set the CI:full label perhaps?

Nope, I just opened an MR

then I am very confused

only label on the PR is the one you added which is `merge-0.9`

it should either have failed in the OOB PR too or in nothing at all