JPEG XL

I am just back in Belgium after a pretty long trip (Daejeon conference center - Daejeon station - Seoul - Incheon terminal 2 - Amsterdam - Brussels - now on the way home), so it is very much possible I don't.

> the array window[] is initialized to all zeroes when the first symbol is decoded from the stream

Section C.3.3

since it just does `window[copy_pos++ & 0xFFFFF]` that would mean that it hits a zero if it's not written

however, it also has the following: ``` distance = min(distance, num_decoded, 1 << 20); copy_pos = num_decoded − distance; ``` this means that if `num_decoded` is less than distance, for example, if it's zero, then `distance` is set to `num_decoded` and `copy_pos` is set to 0

This means that if `num_decoded` is nonzero, then the backreferences can only go to decoded symbols

if `num_decoded`is zero, then it looks like it sets `copy_pos` to zero as well, which will end up being the calloc'd zeroes

Is it specified how frames are supposed to compose the final image? I might be missing something, but the only relevant parts that I've found are: 1. > If can_reference, then the samples of the decoded frame are recorded as Reference[save_as_reference] and may be referenced by subsequent frames. 2. > All blending operations consider as “previous sample” the sample at the corresponding coordinates in the source frame, which is the frame that was previously stored in Reference[source] – if no frame was previously stored, the source frame is assumed to have all sample values set to zeroes. 3. > If duration is zero and !is_last, the decoder does not present the current frame, but the frame may be composed together with the next frames, for example through blending. As an example, let's try to apply that with `cmyk_layers` which has 4 frames: "Background" depends on None "Layer 1" depends on "Background" "Test Name" depends on "Background" "Black" depends on "Background" This is not an animated image, so the duration is always zero, and only "Black" is `is_last`. So according to 3., all except "Black" should not be presented. So we take "Black" we see that it refers "Background" (no reference so blending on zeroes according to 2.), then we render "Background", all dependency are resolved so we blend "Black" on top of it. And that's it? (according to the spec, I know current decoders do more)

no, first "Background" is decoded and stored in ref[0] (or whatever reference id that file uses), then "Layer 1" is decoded, blended with background (then in ref[0]) and the result of that is stored in ref[0], then "Test Name" is decoded and blended with Background+Layer 1 which is what is then in ref[0] and the result of that is stored in ref[0], then "Black" is decoded and blended with ref[0] which at this point contains the previous three layers merged.

in other words you have to apply things sequentially in the order the frames appear in the bitstream

Thanks for clarifying this, that's what I had in mind but it's good to have a confirmation. I was also asking the question because this seems to be underspecified (or is it me being unable to read the spec?), and I thought that you might want to know about this kind of thing.

Yes, I agree the frame blending process could use some clarification in the spec

From `F.3.1 General`: > Otherwise, there is one entry for each of the following sections, in the order they are listed: > - one section for LfGlobal; > - num_lf_groups sections, one per LfGroup (in raster order); > - one section for HfGlobal; > - num_groups * frame_header.passes.num_passes sections, one per PassGroup. The first num_groups PassGroup sections are the groups (in raster order) of the first pass, followed by all groups (in raster order) of the second pass (if present), and so on. There is even this note: > NOTE 1 Some of these sections may be empty in case encoding == kModular, i.e. their size in bytes is zero. However, in `Table F.1 — Frame bundle`, there is the `encoding == kVarDCT` condition on the `HFGlobal` field. Which seems wrong to me, the `HFGlobal` field is always here (and thus has its section in the TOC) but its content is dependent on the condition, as it is already explicitly written in `Table G.4 — HfGlobal bundle`.

right, it will be empty in the modular case but it would be clearer to say it is still part of the frame bundle, since it will be in the TOC

(Apologies if this is the wrong thread.) Hiya! It seems the implementation differs from the spec in how WP handles the rightmost column of pixels. The spec states (H.5.2; Second edition): > The weights `weight[i]` for each of the 4 sub-predictions are computed as specified by the following code, based on the `err` values already computed for sub-predictors of earlier samples. > ``` > [..] > err_sum[i] = (err[i]_N + err[i]_W + err[i]_NW + err[i]_WW + err[i]_NE) Umod (1 << 32); > [..] > ``` `UpdateErrors()` in `context_predict.h` is a little tricky so that `Predict()` can take a shortcut:: > ```cpp > JXL_INLINE pixel_type_w Predict(size_t x, size_t y, size_t xsize, > [..] > for (size_t i = 0; i < kNumPredictors; i++) { > // pred_errors[pos_N] also contains the error of pixel W. > // pred_errors[pos_NW] also contains the error of pixel WW. > weights[i] = pred_errors[i][pos_N] + pred_errors[i][pos_NE] + > pred_errors[i][pos_NW]; > weights[i] = ErrorWeight(weights[i], header.w[i]); > } > > [..] > > JXL_INLINE void UpdateErrors(pixel_type_w val, size_t x, size_t y, > [..] > for (size_t i = 0; i < kNumPredictors; i++) { > [..] > // For predicting in the next row. > pred_errors[i][cur_row + x] = err; > // Add the error on this pixel to the error on the NE pixel. This has the > // effect of adding the error on this pixel to the E and EE pixels. > pred_errors[i][prev_row + x + 1] += err; > } > } > ``` But as `pos_NE` turns into `pos_N` if it doesn't exist (also H.5.2), for each rightmost pixel this becomes: ```cpp // pred_errors[pos_N] also contains the error of pixel W. // pred_errors[pos_NW] also contains the error of pixel WW. weights[i] = pred_errors[i][pos_N] /* = N + W */ + pred_errors[i][pos_NE] /* = "pred_errors[i][pos_N]" = N + W */ + pred_errors[i][pos_NW]; /* = NW + WW */ ``` This gives an extra `W` compared to the `N + W + NW + WW + N` the first quote suggests.

When I update `UpdateErrors` and `Predict` to look more like the spec: ```diff --- a/lib/jxl/modular/encoding/context_predict.h +++ b/lib/jxl/modular/encoding/context_predict.h @@ -141,12 +141,13 @@ struct State { + size_t pos_W = x > 0 ? cur_row + x - 1 : cur_row + x; + size_t pos_WW = x > 1 ? cur_row + x - 2 : pos_W; std::array<uint32_t, kNumPredictors> weights; for (size_t i = 0; i < kNumPredictors; i++) { - // pred_errors[pos_N] also contains the error of pixel W. - // pred_errors[pos_NW] also contains the error of pixel WW. - weights[i] = pred_errors[i][pos_N] + pred_errors[i][pos_NE] + - pred_errors[i][pos_NW]; + weights[i] = pred_errors[i][pos_N] + pred_errors[i][pos_W] + + pred_errors[i][pos_NE] + pred_errors[i][pos_NW] + + pred_errors[i][pos_WW]; weights[i] = ErrorWeight(weights[i], header.w[i]); } @@ -195,17 +196,12 @@ struct State { JXL_INLINE void UpdateErrors(pixel_type_w val, size_t x, size_t y, size_t xsize) { size_t cur_row = y & 1 ? 0 : (xsize + 2); - size_t prev_row = y & 1 ? (xsize + 2) : 0; val = AddBits(val); error[cur_row + x] = pred - val; for (size_t i = 0; i < kNumPredictors; i++) { pixel_type_w err = (std::abs(prediction[i] - val) + kPredictionRound) >> kPredExtraBits; - // For predicting in the next row. pred_errors[i][cur_row + x] = err; - // Add the error on this pixel to the error on the NE pixel. This has the - // effect of adding the error on this pixel to the E and EE pixels. - pred_errors[i][prev_row + x + 1] += err; } } ``` the predictions differ occasionally. (Rounding helps, though.) Some tests (`RoundTripAlphaResampling`, `JxlTest.RoundTripLossless[8|8Alpha|8Falcon|8Gray]`) now fail with different compressed image size, but `AlphaResampling` also fails from the image actually changing. Should I make an issue (or PR) in the repo?

Generally at this point, if libjxl and the spec disagree, we will define libjxl to be right and the spec to be wrong. Otherwise we will break existing bitstreams.

That part of the libjxl code is such a convoluted mess, with that tricky shortcut — that dates back all the way to Alexander Rhatushnyak's original code for the weighted predictor back when it was PIK's lossless mode.

Good catch though, I think you're right that err_W ends up getting double counted on the rightmost pixel. I propose we fix this by just making it spec that err_W is added twice for the last pixel, with a NOTE explaining that this allows some optimization. WDYT <@179701849576833024> ?

I guess so, also ugh

Alright, that makes sense! While you're at it, there's also a small wrong comment (but nothing more than that) in the pseudocode a little further down H.5.2: ``` // if true_err_N, true_err_W and true_err_NW don't have the same sign if (((true_err_N ^ true_err_W) | (true_err_N ^ true_err_NW)) <= 0) { prediction = clamp(prediction, min(W3, N3, NE3), max(W3, N3, NE3)); } ``` Due to the `<=` instead of `<` this also triggers when `true_err_N`, `true_err_W` and `true_err_NW` are all equal, so it might be helpful to update it to `// if [..] don't have the same sign, or are all equal`. (The codebase at least matches the spec here, so this is much less of an issue `:p`)

Very small nit in `F.1 General`: > If lf_level > 0 (which is also a field in frame_header), then width = ceil(width / (1 << (3 * lf_level))). and height = ceil(height / (1 << (3 * lf_level))). There is one too many dot in that sentence (just before the "and").

Hi, i'm making tests for lossless compressing/decompressing png with latest libjxl and i found colors difference (they're pixel duplicates); due to metadata of the decompressed png not corresponding to the original?! They're actually visually different. Is there a way i can make sure to keep the information somewhere (with the help of exiftool maybe)?

I understand the difference between my original png and the jxl can be due to the rendering software but i think the decompressed png should look the same as the original, else it's.. Not lossless. I'm probably missing something

what file are you using and how are you encoding and decoding the image?

I have not yet encountered a roundtrip which did not result in an exact replica of the source file

I'm using cjxl/djxl on windows

Here is an example image

what are the parameters you are using with cjxl and djxl?

I'm not sure it's going to work if i drop it on discord so here are the two pngs from my pov

Sorry it was a lossy conversion i'm doing it again

Gif quality is bad but there is a color difference

please share the commands with parameters you used and I will try to replicate

simply cjxl test1.png test1.jxl -d 0

Then djxl test1.jxl uncompressed.png

With cjxl & djxl exes in the directory from latest release

round trip results in bit identical files for me, latest release

maybe the windows version is borked?

I am on linux

No idea, if i set the metadatas back to original the pngs are back to being the same.

what kind of metadata?

just encoding and decoding with the windows build still results in identical files for me

If i copy the new icc profile from the decompressed png to the original with exiftool -tagsfromfile decompressed.png -icc_profile test1.png

It's actually v0.11.1 of libjxl. Here's in a bad windows photo viewer. In my other software there is still a light difference though

(the two pngs, the jxl has the same color as the decompressed)

Here are the metadatas if it can help i don't know

Ah, I assume that's the (known) problem with image viewers rendering colors etc. differently. Use ssimulacra2 on both images and they should be identical

Or run ImageMagicks `identify -format "%M %#'\n'"` on both

I'm not sure we have identify on windows, they're pixel duplicates though

ssimulacra should be included with the the libjxl binaries for windows though, so try that

I'll try that. Now that i'm here i also had trouble converting apng files to jxl with the Getting pixel data failed. Seems like a known problem though.

That should be fixed after 0.10, so with latest 0.11.1

It says 100 between the three files. I'm a little confused since there is a color change though in my softwares and i suppose there is no way to retrieve the original metadata if needed...

It's just a visual difference caused by image viewer

The files are identical

dunno why but this just triggered memories of that losslessly denoise it dude

Should be this issue

https://github.com/libjxl/libjxl/issues/4307

afraid I am not aware of what occurence you are talking about

The software i use uses pillow for jxl support i believe

whaaaa, lemme try and find it, it was a wild twitter thread

Yes the problem is they're both pngs in that case so it's a problem with ICC profiles support then?

I guess so

Thanks for the help, i'll store the metadata somewhere just in case and convert to jxl then

ahaha I found it https://x.com/lookoutitsbbear/status/1794962035714785570

I think what's happening is the PNG does not have any ICC profile or colorspace defined meanwhile JXL itself does not support not being color managed and assumes that if an image does not have anything defined that it's sRGB and then when you convert it back to PNG it adds a sRGB profile since the JXL it ends up as is registered as being sRGB. so something like: PNG (nothing) > JXL (sRGB) > PNG (sRGB)

uhhhhhhhh lmao

wow, it gets wilder the more you read

right?

he doubles and triples down, amazing

<:KekDog:805390049033191445> Yep it's kinda what jxl does in that case Edit: In the end i'm still not too happy with jxl changing the metadata as it sees fit even if for technical reasons it's better with an sRGB profile or whatnot. I'll have to backup the metadatas while converting now for peace of mind. Though simply deleting the profile with exiftool works.

Do the two png files look different when viewed in Chrome?

is this a valid JXL file? https://github.com/libjxl/jxl-rs/issues/431#issuecomment-3473137851

I guess the resulting sample values are way out of range, so it's technically valid but unspecified?

Yes they do in chrome/chromium and there is the same small difference in color. ||on latest chrome. windows 10 Pro N 22H2 19045.5737 If i take two random pixels on the same location: Original RGB(70,6,71) -> (68,1,70) decompressed Original RGB(103,12,94) -> (103,4,94) decompressed||

https://imgur.com/a/lv0pvw4#l09golK It's indeed due to the ICC profile, if i paste it on the original image i get the uncompressed image's colors.

The icc profile because why not <:Hypers:808826266060193874>

can you open a github issue for it with the example png in it, so we don't forget about this? looks like something that needs investigation

Yes i can't reproduce it with the imgur one so it must modify it. Do i just drop it in github?

Would be nice to have a way to actually share the exact same .png <:KekDog:805390049033191445>

Here if someone can try to replicate on windows it would be nice https://drive.google.com/file/d/1wEmris70JkIyLoWZrKElOINs23pLxrTL/view?usp=drive_link

Here's the issue they opened https://github.com/libjxl/libjxl/issues/4503

Also let's move this discussion to <#804324493420920833> since this is very likely not a specification issue but an implementation issue.

Sure well anyone can ping me there if they find something related or for more infos. <:PepeOK:805388754545934396>

Again small nit, in `I.5.3 HF dequantization`: > Every quantized HF coefficient quant is first bias-adjusted as specified by the following code, depending on its channel (0 for X, 1 for Y or 2 for B). I think "quant" should be inside quoted to look like `quant`

It seems that the composition of VarDCT frames is not specified. For modular groups (both LF and pass) it is written that the data should be patched into the GlobalModular's channels, but nothing is said for VarDCT's data. IMO there should be a section that says something around the lines of: With the data from Lf and Pass groups, one should: - Dequantize coefficients - Apply IDCT - Copy the data to three channels of the size of the destination image

I suppose more broadly it would be useful to have a description of the frame rendering, something like this: (of the top of my head, maybe I got some order wrong but that's exactly why it would be useful to make this more explicit instead of having to read it between the lines all across the spec) - main color channels (XYB/YCbCr/RGB) from VarDCT data or from modular if it's a modular mode frame - extra channels from modular - Gaborish and EPF are done - patches applied on top of that - splines applied on top of that - noise applied on top of that - frame saved to reference slot if save_before_ct - XYB or YCbCr converted to RGB - frame saved to reference slot if !save_before_ct

This already a bit covered by `A.2 Decoding process`

Might we get an updated version of the spec posted here? Been more than a year since the last one.

Yes in principle the info is already there, but it could be made a bit more clear, in particular upsampling frame blending and reference slot saving is now not very clear since these are not mentioned explicitly in A.2 so you have to understand it from the prose in F.2 which is pretty dense

the most recent version I have is 2024-07-19, nothing has been updated yet since the second edition got published. This channel is collecting stuff that can be improved, but so far I haven't yet edited any of it into a draft for some future third edition...

ah, alright

More nits: in `I.5.3 HF dequantization`: > Every quantized HF coefficient quant is first bias-adjusted as specified by the following code, depending on its channel (0 for X, 1 for Y or 2 for B). I think "channel" should be quoted `channel` as well. > For the X and B channels, it is then multiplied by by pow(0.8, frame_header.x_qm_scale − 2) and pow(0.8, frame_header.b_qm_scale - 2), respectively. There are two "by" before the first pow.

found another spec issue when debugging jxlatte

Annex J.4 - EdgePreservingFilter

It specifies that you must iterate between 0 and 3, but it makes no statement on what to do if the input is grayscale modular (i.e. 1 color channel)

grayscale-public-university.jxl is an example of such a file (in conformance)

the answer here is that you pretend the image is 3-color with three identical channels, but only when calculating Distance0 and Distance1

For example, it has the following:

I was typing the same thing

``` DistanceStep2(x, y, cx, cy) { dist = 0; for (c = 0; c < 3; c++) { dist += abs(sample(x, y, c) − sample(x + cx, y + cy, c)) * rf.epf_channel_scale[c]; } return dist; }``` it should instead have ``` DistanceStep2(x, y, cx, cy) { dist = 0; for (c = 0; c < 3; c++) { i = num_channels == 1 ? 0 : c; dist += abs(sample(x, y, i) − sample(x + cx, y + cy, i)) * rf.epf_channel_scale[c]; } return dist; } ```

and similar for DistanceStep0and1

What if there are different values in epf_channel_scale? That could affect the end result

there are highly likely to be different values in epf_channel_scale

if you don't use them then you'll get a result that doesn't match libjxl

Sorry I misread your code, thought you were taking epf_channel_scale[i]. Forget it.

I'm not sure if this was already reported, but there appears to be a missing parenthesis in J.1