JPEG XL

I set djxl to use level 1 zlib IIRC, so oxipng is doing the same without filtering, or any of the JXL en/decoding

I'm working on some Resampling tweaks and noticed this... There's 3 different types? `DownsampleImage2_Iterative`, `DownsampleImage2_Sharper` and `DownsampleImage` The comment at the top says regular should be used if shaper isn't fixed, and the comment I've replaced (also tagged `lode`) said iterative was too slow for squirrel and it should be raised, which I did. Testing seems to show roughly equal visual quality between iterative and sharper, so I made it effort 11 since lossy does nothing there currently, but how does regular compare to the other two?

Iterative seems to retain the original details of the image better, but the lower resolution makes it look like a sharpen filter with ringing artifacs on non-photo content. Combined with the 2x slowdown (10x slowdown at effort 7) and 3x memory increase, I considered disabling it entirely or making it an experimental flag

Well, I got all 3 downsampling methods. Orginal, Downsample, Sharper, Iterative

last one seems best to me

similar sharpness as the third image (second downsampled) but with slightly less aliasing

It scores around 1 point higher in ssimulacra, but varies image to image. Particularly on non-photo, the last downsampling method causes severe ringing

For now, I set the last method to enable only at effort 11 lossy, since it's a relatively large slowdown for highly varied results (And e11 does nothing different to e10 otherwise). If/when it's sped up and the TODOs are cleared, it can be re-enabled at standard effort settings

I actually wanted to ask. I know distance is capped to 25 in cjxl, but I also know you can override it and set whatever you want using the library/a dev build. I got resampling 4 to look competitive at distance 25 compared to resampling 2, but should I add this to compensate for values above 25 `2.5 + ((Distance - 25) * 0.25)`, or just explicitly set the value since it's the cap?

~~If I had a cracked build I could set resampling 8 too~~

Scrapped the resampling 4, made anything below 1440p far too blurry to be useful. Noticed one or two images that actually had significantly higher bpp and quality than no resampling, but it was an issue with the current settings too so I didn't bother tweaking it and breaking the other images

bpp is a smidge higher, but it's faster, more memory efficient and scores a lot better

comparing to git head is probably safer. adding bits to increase fidelity is totally unremarkable (assuming ssimu2 captures reality in this context), so the compelling thing would be the speedup if density doesn't diverge from expectation in general

having curves of bpp vs score might make it easier to compare two versions without having to match the bpp exactly

That's git head at distance 10, my PR at distance 10, and git head manually set to resample with the same parameters as my PR for the speed/memory comparison

The slight bpp and SSIMULACRA2 difference with the same parameters is from me disabling the iterative downsampling at effort 7, which gave the speed and memory improvements

hm, version/commit hash indicates differently

Have some hashes, fresh out the workflow

The problem is the image content changes the effectiveness and the internal butter scores. So on some the bpp is higher and on others it's lower, but overall the quality is higher than the bytes otherwise

I was originally using `3 + ((Distance - 10) * 0.25)` to calculate the resampled distances, but on some images it was half the bpp and lower quality. So I raised it to `2.5 + ((Distance - 10) * 0.25)` which seemed to be a good middleground

this is just distance / 4

If you input `-d 15`, it does 15 - 10, then 5 / 4, followed by 2.5 + 1.25 to give a final result of `--resampling 2 -d 3.75` instead. This usually results in equal or lower bpp, but at higher quality. Some images it's too low, some it's too high, but I thought I'd aim for the higher end.

15 / 4 = 3.75

expand it out. `(distance - 10) * 0.25 = distance * 0.25 - 10 * 0.25` and `10 * 0.25 = 2.5`

so you're just taking `distance * 0.25` and then subtracting and adding `2.5`

Right, yeah. Because 2.5 is already 1/4 of 10. Didn't catch that since lowering from 3 was a last minute decision

So if it is the right value, that can just be changed to distance / 4, but if it's any other value then it probably needs the equation

if you want to add 3 instead then change it to distance/4 + 0.5

still much simpler

and notably, much more readable code

The current/original line was `p->butteraugli_distance = 6 + ((p->butteraugli_distance - 20) * 0.25);` so I just assumed there was a reason for it... Thanks for spotting that

Added to the PR, thanks again

This is the worst case I've had so far. 0.340 bpp on the left, 0.438 bpp with the PR in the middle. It seems to focus more on the whole image than things like text. Going to 3 instead of 2.5 reduces it to 0.358 bpp, but I'd have to test more images to see if it's worse for other types of content

It's essentially a balance of photo vs non-photo. 2.5 means it aligns with difficult photos, but overshoots on non-photo. 3 means it's equal on non-photo but undershoots on photos Guess I'll add 0.25 to put it in the middle

The first image has unacceptable quality imo

The other two are slightly better

I don't think the first image is usable quality at all

Yeah, it seems like butteraugli is falling flat on it's face for some reason

It's a useless and ruined image

The other two are more acceptable and usable

The one in the middle looks the least distorted

For now, I've changed the PR *yet again* to use a setting between the last two. Distance 2.75 at 0.380 bpp. 40 higher than main, but 60 lower than the second image. In my opinion the best quality without drastically changing size between distance 9.9 and 10

15% higher bpp for 20% higher score in the worst case. On a random photo it's -7% and 11%, generally with visual inspection looking better than SSIMULACRA2 scores anyway. This is the quality 0 to 18 range, so really people shouldn't be going this low anyway... But at least if they do, they get usable results now

In the end, as with most things in cjxl, it depends on the image :P

Which pr

https://github.com/libjxl/libjxl/pull/4147

This doesn't change anything except when distance >=10?

I hope people aren't actually compressing images at libjxl distance 10...

0.5 to 1.5 is the recommended range I think.

I would even be in favor of making those the hard minimum/maximum distance values

But I'm radical like that

I think software should be designed to not give people a big red "DO NOT PRESS" button since they're always GUNNA press it thinking they know better. 😂

People who know better can always fork or make a pull request

Yeah, at that point resampling is competitive, which says a lot about what kinda quality you're using... But it is better, usually smaller and multiple times faster to encode

And decode.

Surprisingly not in my minor testing

But nothing would be lost either if distance values were just hard clamped to 0.5-1.5

At that point I would say libjxl is not adequate or competitive there

Unless it were to receive a lot of specific tuning

It's mostly people comparing it to AVIF, which unfortunately is quite often. The PR actually increases quality at distance 25 compared to main, but otherwise the images are unusable

All the more reason to clamp the values

To prevent bogus automated comparisons that are done without actual visual confirmation

Like the comparison the avif team did at absurdly bad quality settings

That does raise a point....

You made me realize that it's actually a serious issue and now I strongly suggest clamping the values unless MORON_MODE=1

And change it from "expert options..."

It's not for experts, it's for crazy people 😂

The best looking images actually scored lowest, probably due to the DCT noise you love so much, caused by the resampling forcing it to use smaller blocks

What metric?

SSIMULACRA2, I can rub some butter on it when I have my computer on

I just choose the image that has the most discernible features and doesn't look like someone smeared their fingers all over an oil painting

It's ironic. Resampling the image to half the resolution brings *back* fine details

Yeah I don't trust ssimu2 at all sadly, it sounds exciting when I first heard of it but it loves zeroing out crucial image data and visual features

0.8 has the same resampling gains, just with both scoring higher than main

It's not a good sign when a codec performs a lot worse or better with resampling.

These are good ways to stress test the encoder though I guess

DCT noise only looks bad if there are obvious block boundaries and if that noise doesn't blend into the image behind it.

The biggest thing I noticed was chroma noise at high distances. Large banding areas of yellow across flat surfaces, so while resampling was sometimes a dubious fidelity improvement, it removed the most distracting artifacts

Tends to distribute bits across the entire image instead of focusing on the 'attention points' like the UI in the game screenshot above

Imagine an image codec that looks like an analog picture getting static interference when the compression is too strong 😂

That would kinda be a cool idea. The artifacts would be honest, easy to tell when they are there and easy to see underneath them

Just like the stated goals of jxl

Oh, you reminded me. I have a natural example of an image aligning with the DCT coefficients

The roof of my church image has a single block that remains perfectly sharp even at distance 25 or intensity targets like 1 (IIRC)

Jon I think spoke a great deal about how dangerous and undesirable deceptive artifacts like blurring and smearing can be in avif for example. But shortly afterwards, libaom became a lot less blurry and libjxl became a lot more blurry, so the roles were reversed.

It's a little embarrassing especially considering how the creator of avif is trying to wield his influence over Chromium to prevent jxl from coexisting and competing with his avif and webp

libaom made some real strides in the psychovisual quality

<@207980494892040194>

TagStudio (a tagging software) is trying to support jxl but there is some incompatible at the moment. If anyone like to help out feel free! Or just want to use this software in general. PS. I'm just an user, so dont bother asking me any technical questions. Here is the git: https://github.com/TagStudioDev/TagStudio

And here is their discord https://discord.gg/hRNnVKhF2G

Or should this be posted in <#794206170445119489>

Hi, TagStudio maintainer, to correct the above: there’s no incompatibility with JXL, there was a small build issue related to a [Pillow](<https://pypi.org/project/pillow>) plugin used for JXL that’s now been fixed. Not at all related to libJXL itself.

PS. It was all a ruse to advertise for more contribute so version 9.6 will be e release*yesterday*

Please, no

We have nothing to do with the above, nor condone it. Please disregard it and I hope everyone is enjoying their day. Sorry for the disturbance

Regardless, good to know it has support. I don't think we were aware of it before

yes, we make use of [pillow-jpegxl-plugin](https://github.com/Isotr0py/pillow-jpegxl-plugin) which seems to be the same thing hydrus uses according to https://discord.com/channels/794206087879852103/803574970180829194/1335062020269150208

one thing im a tad wary on is how things will look in an eventual rust rewrite, but thats my issue to tackle, it does look though that with this plugin being part rust it obviously does work. im just confused because i know mozilla is waiting on the completion of jxl_rs

libjxl can benefit a great deal from a rewrite or major overhaul. Regardless of Rust or not.

Interesting, so some of this is being propagated by libjxl at the “top” needing some rework?

It's not something to be wary of. The codebase is showing its age. It was originally 2 separate codecs written by different people, merged together into one. So the seams and stitches are showing.

What I mean is, a rewrite would be more concise and simpler to maintain.

gotcha, i see what youre saying

That's why some people consider jxl-rs as the ultimate solution

for some reason i glanced over the mention of it being a reimplementation

when i was looking at the pillow plugin i kinda assumed it was a wrapper for the C lib

i see now thats not the case

huh? but it is? wait what are y'all even talking about?

jxl-rs being a reimplementation rather than a wrapper

the pillow plugin isn't using jxl-rs. jxl-rs isn't even ready for use

i dont believe i said it did, if i gave such an appearance i apologize for the confusion

ah it's fine \👍 also sorry if I came across as rude

no worries at all!

jpegxl-rs is the wrapper, jxl-rs is the reimplementation

I got confused as well

the names are very similar

Working on TagStudio is the reason I ever heard of JXL, so I’m trying to figure out all this stuff haha

Hey that's cool, it's nice to have more tools for viewing and modifying and converting metadata

Other than exiftool

It should be noted, jxl-rs is only a decoder, and jxl-oxide is an existing rust decoder too

It's likely to expand to encoding later. Decoding is the hard part

And decoding happens more often than encoding

is libjxl_extras_codec.a supposed to be installed?

No

Hey. (Apologies if this channel is not for general peanut gallery questions, redirect me if so.) Is it expected that progressive lossless encoding (not JPEG→XL reconstruction!) with `cjxl` can be very slow for large inputs? Input: ``` $ magick identify /path/to/input/jpeg /path/to/input/jpeg JPEG 24316x18544 24316x18544+0+0 8-bit sRGB 142.977MiB 0.000u 0:00.001 ``` Command line: ``` cjxl -v -v -e 7 -j 0 -p /path/to/input/jpeg /path/to/output/jxl ``` Cumulative CPU time is 2h08:33 so far (on a ~4.5 GHz Zen 3 desktop chip), and what seems to be a ~18 GiB working set.

Could you try with effort 4 instead? Progressive lossless has density bugs at effort 5+, along with disabling chunked encoding, causing patches to be enabled at effort 5+ (very expensive on large images), IIRC

2h still seems excessive, are you hitting swap?

I did, briefly, at one point, but that was really tiny portion of total runtime

yup, `-e 4` finished the entire batch in 3 minutes _and_ generated 20% smaller files

I never even touch the -p flag anymore I have had results that are just too funky

funky, as in...?

one time I had a file size that was 8x the size of using `--progressive_dc` instead. another time it made it take 3x longer to decode

what does `--progressive_{ac,dc}` mean?

The issue is the chunked encoding I mentioned. Usually it disallows progressive files due to requiring the TOC at the start of the file, to contain data only written at the end. Or something to that effect

iirc progressive_dc encodes a second small second image that has a good amount of progressiveness it adds very little file size and is IMO one of cjxls best features

what exactly do "DC" and "AC" even mean in this context?

https://github.com/libjxl/libjxl/issues/3823

DC is the 1:8 scale image of lossy encoding, usually the first preview before the rest of the file. Progressive DC can turn that into another lossy image, with a second DC, now 1:64 of the original image for further progressive steps. Progressive AC is how the remaining data is ordered. There are 2 modes, the QAC which simply sends the least significant bits last for a more blocky image. And 'regular' AC, which has a rougher initial image but better quality passes, if that makes sense

I tried to read whatever docs I could find on the conceptual overview of the format and encoder, but I still can't understand what exactly "DC" and "AC" are

DC = direct component, AC = alternating component; if you do a DCT (or other types of frequency transforms) you end up with one coefficient representing the block average (this is the DC) and a bunch of other coefficients representing 'waves' at various frequencies

in the jxl spec we use the terminology LF and HF instead (low frequency / high frequency) since we mix DC and AC coefficients a bit to harmonize variable block size data into a low frequency part corresponding to an 1:8 image (in case of the old JPEG which only uses DCT8x8, that's just the DC) and a high frequency part which is the bulk of the data corresponding to the remaining image data.

DCT is not used for lossless

Wasn't the question about lossless?

Progressive lossless is not well tested or optimized afaik

Is this a bug others can reproduce? Maybe an issue on github can be opened.

A random thought, would multithreaded extra channel and layer decoding require a format extension? I'm not sure if the decoder can tell if they reference other channels/frames beforehand (Using `-E` or blend modes)

https://github.com/libjxl/libjxl/issues/4156

.....what?

They want to completely replace the encoding methods so it beats ZPAQ

Breaking the spec now would be one of the worst things to do as adoption is ramping up

Also isn't ZPAQ like super slow and single threaded? That would be unusable for larger images imo

Breaking the spec *bc some dude asked chatgpt how to beat zpaq* <:KekDog:805390049033191445>

Everyone is free to fork libjxl and make a research prototype thing out of it, but no way the spec is going to change in breaking ways.

it's not single-threaded, but the CM is slow and aging. obviously density improvements are possible for jxl, but the particular comparison is amusing

It could still be possible to make small, breaking changes, as long as existing old files will always work, but if future files would change in breaking ways, you just keep it disabled in the encoder until no one is using an old enough decoder anymore for it to be an issue any longer.

You can add experimental features to the encoder but keep them disabled indefinitely until there's literally no one using an old enough decoder

It would have to be important enough to be worth it, but if it's something you really think is worth it, I don't think you should feel like your hands are completely tied

It's still pretty early into the life of "next gen alien technology from the future."

You wouldn't even have to announce it or make a big deal about it. It could be something done completely behind the scenes. Adding support to the decoder and the spec first, and waiting until everyone updates before changing the encoder.

even such breaking changes are quite undesirable, it is really hard to tell when "no one is using an old decoder anymore" so it really complicates interoperability. If there's a really good reason to do it, it can of course be considered, but something like just some compression improvement is not enough of a reason imo...

Isn't the AVIF team using the "added to decoder/disabled in encoder" method for lossless YCoCg support?

At this point in time, it's not THAT difficult to tell, yet, at this stage.

And you can always wait a few years AFTER the spec+decoders have all been updated and you THINK everyone already switched, BEFORE allowing the encoder to generate a new bitstream that wouldn't work with an old decoder.

I kinda doubt there's an important issue that's big enough to warrant this methodology. Since the spec was so thoughtfully designed to begin with. I'm just saying that if there WAS, I wouldn't want you feeling as if your hands were completely tied.

But I think all the "hindsight regrets" were all pretty minor, weren't they?

Just minor little things that could make things slightly more neat and cool

Probably not big enough to warrant a new incompatible bitstream structure.

Or maybe, small and harmless enough to be added and co-exist with the existing one?

For use in the distant future, when there don't exist any incompatible decoders anymore.

> CMYK-tailored predictors optimized for print-oriented content. that would be jxl’s killer feature, for sure

MA trees and dots/patches should be able to do half toning and crosshatching, ect

Seems to be causing a lot of failures https://github.com/libjxl/libjxl/pull/4132

https://github.com/libjxl/libjxl/issues/4157

https://github.com/libjxl/libjxl/issues/4159

Bring it up it in <#848189884614705192>?

Equal visibility here and Jon is already in the PR, just thought I'd make the rest of you aware

<@1346460706345848868> can you take a look? Looks like you have an edge case not handled

Some int arithmetic was replaced by float arithmetic in RebalanceHistogram(), I bet this is causing some shenanigans. Ideally that function should just never return false and that template option `minimize_error_of_sum` should not be needed, but if that's too tricky then at least it should never return false when `minimize_error_of_sum==true`. I guess it's hard to test this exhaustively since the number of possible inputs is too huge and it probably only fails in rare cases, so I suppose we need to use brains and math and stuff.

Oh no! Not brains and math and stuff! Anything but that!

Yeah, will take a look.

I tried to find a mathematically correct way to rebalance any histogram, but due to the increase of low frequencies to 1/2048 and coarsing by `SmallestIncrementNonzero` it got too complicated quickly and I have not succeed.

Fix is ready https://github.com/libjxl/libjxl/pull/4160

I've put an experimental version of rebalance that cannot produce invalid histograms, testing it now on a large corpus of images

Hello there! I wanted to integrate the format into my program on my raspi. It works great, but I noticed an recurring artifact present in the images. I am using my raspi as an intermediate where I compress rather large TIFF images (>200Mo, 16bits) of astronomical observations into jxl. The images are split into bits, so there's no problem there (even though I'd like to make it a tiled jxl, I did not figure that out yet). My problem is: - blue boxes appearing around stars (close to dirac) See below a few images, one with lower quality (d=0.085) and one with good quality (d=0.001), and the stock image. These are closeups to see the issue, and the histogram has been way stretched since it's 16bits. The original tiff is there: https://we.tl/t-Ri6XFzZtkH I noticed that when using cjxl PNG->JXL, this artifact is not present. When exporting to regular PNG from my solf, I also do not experience this. Only when using libJXL. Note that I load the tiff in a `uint16_t*`, which I pass directly to libjxl using ` JxlPixelFormat pixel_format = {image.getChannels(), JXL_TYPE_UINT16, JXL_NATIVE_ENDIAN, 0};` Here are my settings: ``` basic_info.uses_original_profile = JXL_TRUE; basic_info.exponent_bits_per_sample = 0; color_encoding.color_space = JXL_COLOR_SPACE_RGB; color_encoding.white_point = JXL_WHITE_POINT_E; color_encoding.transfer_function = JXL_TRANSFER_FUNCTION_LINEAR; color_encoding.primaries = JXL_PRIMARIES_SRGB; color_encoding.rendering_intent = JXL_RENDERING_INTENT_PERCEPTUAL; ``` When passing the same buffer to PNG, no artifact, so there is something in my settings I suppose. Where does this artifact come from, and what kind of settings are likely to modify it? I have tried changing the distance and the effort with no luck.

Should I create an issue on git?

Is lossy a requirement?

I need about 10 - 12x compression ratio, it will be hard to get with lossy I presume. Being on embedded device, I also cannot use process-intensive operation, the processor isn't that great. This 300Mo file gets compressed in about 20seconds on target, which is about twice as fast as jp2.

how sure are you that you have absolutely no over/underflow in the values you pass?

Images come from the TIFF. They are in uint16_t, so I believe over/underflow are not possible there

I added a little more details to make sure this is clear 🙂 Thank you for asking clarifications!

I wonder if there is a significant difference between modular and vardct with this issue

I wonder if perhaps underflow occurs because of the conversion to linear sRGB

does the problem still occur if you change the white point from E to D65?

(E is a bit of an unusual choice)

also, which decoder produced the result with the green square?

`basic_info.uses_original_profile` should be false for lossy, as XYB is used internally instead of RGB

oh, yeah, missed that

it also means that what I said doesn’t apply

...why does that exist? Lossy only works with it set to false, and lossless isn't lossless unless it's true, right?

it does seem to be a source of a lot of confusion

I know too little about this application to say whether this might be one of the instances in which there might be a case to be made for it

Lossy modular may find a need for it

Lossy modular still uses XYB interally (by default)

I assume its because Jpeg XL is the first of its kind, where color information is part of the encoding and decoding process. Every other image format has optional color space information defined in an icc profile. Very few are aware that Jpeg XL handles all color internally.

AVIF also has per-frame color information through the CICP metadata.

You can opt out of all color management in AVIF with cicp 2/2/2 (Not that's a good idea). Jpeg XL *has* to be color managed every step of the way. We should really have a big fat warning with red text in the docs saying, "DO NOT USE LOSSY COMPRESSION WITH THIS OPTION ON". RGB compression is a sin.

I presume RGB lossy could be made to not be *horrible* as I assume it's just the encoder not being tuned/setup for it at all really, though it doesn't make much sense to put effort towards fixing it up currently since if you are doing lossy then XYB is the way to go for most use cases. But yeah with how many people keep mistakenly using RGB lossy it should probably be pointed out in the docs.

rgb compression is probably a lot better then xyb if you want a fast decode option

that can be 4 xD

I wonder whats faster, jxl-oxide or yuvxyb https://cdn.discordapp.com/emojis/1113499891314991275?size=64

im assuming the loss accured through rgb->xyb->rgb roundtrip is significantly lower than the actual lossy nature of the codec

maybe a better way to put it would be that with JPEG XL color management is apart of the **full** pipeline and not something that can be ignored like in other formats. Jon/wb worded it nicely here: https://discord.com/channels/794206087879852103/794206170445119489/1281531968688750664 Also this blog post from Kampidh showcases the kind of negatives that come with not treating color management as first class: https://saklistudio.com/miniblog/modern-image-format-linear-color-analysis/ <@833420862334042152> sorry for the direct ping but Discord doesn't seem to have a way to reply to multiple messages at the same time.

animated examples from that blog post: https://saklistudio.com/miniblog/blog-img/jxlvother/1bpp-anim.webp https://saklistudio.com/miniblog/blog-img/jxlvother/06bpp-anim.webp

Using the vips api might be easier <@1353777338843795608>

Compared to libjxl directly

IIRC the transformation is fully reversible

I’m thinking like floating point math weirdness

Maybe it should be changed to `basic_info.uses_xyb_internally` or something similar. Since the output *is* the original profile, just not used for the internal compression

32float can represent up to 24int with exact precision, so for any normal 8int or 16int input it shouldn't be an issue

lossy image formats try to approximate low "perceptual" distortion but this TIFF is raw sensor data with no hints about how to interpret the brightness values, so there's no way of knowing what type of distortion is "perceptible" or not. I see you added some color encoding info with the libjxl API, but... there's still no way of telling how bright the pixels are, and therefore how much distortion is acceptable or not. I think that weird square you're getting might be a separate bug though.

Also, lossless is about 1/3rd of the size of the TIFF and it's extremely fast on effort 1 and 2. Higher effort doesn't compress the file any smaller.

Thank you all for your excellent responses! I chose E because my images are mostly black and I thought this would better suit my data, but I will try with D65. I used multiple decoders: decoded the jxl directly in gimp, also tried decoding it through my own means with libjxl and converted to TIFF.

I did not understand this, thank you !

Okay, so I tested with `basic_info.uses_original_profile = JXL_FALSE;` And the squares vanished! I did not try to put the white point to D65 as it seems the issue is now resolved. If this is actually a bug with `uses_original_profile`, I can provide data and code to reproduce it. I will further experiment, but everything seems ok now

I see. Is there a way to give better hints to the encoder ?

The most important thing is to get the encoder to see the image in the same way as a human. If you decode to png, it should be a png you can view in a browser or other standard png viewer without adjustments. One of the causes of bad lossy compression is doing things like storing 12-bit RGB values in a 16-bit PNG with the padding in the most significant bits instead of in the least significant bits, which means you're effectively making the image near-black and the encoder will compress it too aggressively.

at d=1 the image should be more or less visually lossless when viewed without zooming, if that's not the case then something is not right

I see, this seems close to what I am doing, but I do need the large dynamic. Stars are bright, nebulaes aren't ! This is an intermediate format for transfer over wifi (because fast compression, fast decompression and good image quality) The image is supposed to be processed after, on a more powerful device, so we need as little artifacts as possible but still in linear. It's not an image meant to be viewed right away, it doesn't have any gamma applied to it yet!

I understand, but if in the postprocessing you're going to increase brightness a lot in the darker regions, then probably it's a good idea to give the encoder an image that is closer to the brightest it may get after processing rather than letting it think it's very dark

So I would get better compression performance if I pre-gamma it as I understand. That could be done, performing a simple gamma is relatively reversible.

This is the input

and this is the processed output

same region

I will try to spread the dynamic a bit better, and report back with the results if that's of any interest to you 🙂

Some light related reading <https://github.com/libjxl/libjxl/issues/314#issuecomment-880620569>

Good read thanks !

I come to report my findings. I applied a moderately complex gamma. A simple exponent wouldn't do, so I had to use a curve (for which I do not have the exact values, I did it on gimp) I then fed this "proper" image into my lib using jxl and noticed a few things: - the "-d" parameter now behaves as expected, a value of about 0.9 - 0.8 yields similar compression rate (x12) to previously 0.065 - the compression is worse in places of high intensity, as in, the center of the galaxy is badly compressed, whereas it was fine on the unprocessed input You can see clearly the blocks in the output that had a gamma on it already. Should the parameter given to `transfer_function` be something else than LINEAR in this case?

In summary, in my use case so far, it seems that using the pipeline TIFF->JXL (->gamma for display) is better than TIFF -> gamma -> JXL (-> Ungamma if necessary)

it might be that using a higher-than-default `intensity_target` is a justifiable alternative in your case

except with PQ images, `intensity_target=N` means that (1, 1, 1) in the image has a luminance of `N` cd/m²

the default is 255

Ah! Even for a 16bits image?

yes, that 255 doesn’t have _that_ much to do with 2⁸ − 1

it’s numerically equal to it but these days, you can treat that as a coincidence

maybe there is less of a temptation to relate it to that if it’s written as 255.0

You sure you can't just compress the 300M tiff to a 117M lossless jxl?

It's even faster than lossy compression.

Maybe just change the default to 300? Then there's even less confusion. And I agree it's very common for people to be confused about the uses_original_profile thing, something should be done to prevent it from being misused by developers.

Making the libjxl api less confusing to use is very important

And reducing the chances of mistakes.

Intensity target is in nits correct?

Yes

And the default value is in the spec, you can of course signal any value but the value you get by not signaling anything is 255

I need to read the spec more thoroughly some day...

need tirr's jxl document <:PepeHands:808829977608323112>

no, yuv rgb conversion is very fast

no, not really, isolating luminance into its own channel is crucial

A) it still takes time, and when you want the lowest possible decode speed, its a waste of time B) xyb to rgb iirc it a bit more involved then yuv to rgb

or maybe im thinking of xyb to yuv

in any case, its extra operations

It's never a waste of time, because encoding in YUV is way more efficient, and chroma subsampling gives a performance boost way beyond the minuscule speed boost of skipping YUV<->RGB conversion

wrong

if I need faster decode, i need faster decode

If it's good enough for jpegli, it's good enough for us

you were talking about XYB to RGB?

yes

well that changes things

but still

even then yuv to rgb still takes time

directly encoding in RGB is awfully inefficient

sure

no one said differently

For the fastest possible decode speed just use ppm

some comression still needed https://cdn.discordapp.com/emojis/867794291652558888?size=64

But then it's not the fastest decode speed possible as you said

Come on. You're basically asking for magic

PPM with filesystem compression

If you use libjpeg-turbo you don't need to ever worry about decode speed. And chroma subsampling is a significant computational reduction too, meaning speed boost. You would probably gain much more speed than the negligible, absolutely marginal amount of time yuv conversion costs...

Unless you actually measured the difference in time it actually takes, saying this does not matter at all. Premature ejaculation is the root of all evil.

chroma subsampling is actually not super cheap if you want a high quality upsample

You already said you don't care about encoding efficency, so low quality upsampling shouldn't matter

it matters a shit ton in an NLE,

I increased it to a 1000, now I need to have d=3 to get x12 compression. The image is still worse than the input without an applied gamma in critical parts. I will keep the original image mostly black as it's still the easiest for me (no gamma - ungamma procedure) and no parameter setup. I might tweak the intensity to bias it toward compressing the noise but apart from that, I'm satisfied of the lib and happy with the results 🙂 Thank you all for your support and insights!

If I understand correctly, I'd actually suggest trying an intensity target of 10,000 on the original very dark image. That should give all areas roughly equal quality regardless of brightness. In the end, what works, works. So feel free to stick with your own solution. Could definitely do with a 'Range extender' option that doesn't crush blacks and over smoothen whites

I see ! Okay I will try that 🙂 Thank you Maybe this use case is a bit too specific, it's hard to ask for a format to be that versatile.

As you've proved by lowering the distance to 0.1 and 0.001, it is possible, just not very easily currently. We've had a few people doing satellite/multispectral imagery, along with people using it for photography where they want to change exposure after. There's even multiple issues on Github about this exact problem, but no easy way to fix it other than changing the input, using the intensity target workaround or lowering the distance drastically as you did

oh, right, I meant without the gamma

just the same as you were doing before but with the higher intensity_target on top

Hell yeah. +1 for "range extender mode" for raw sensor data!

I see! yes then what you suggested did what it was supposed to do. In the end what I'm doing is trying different combinations of distance and intensity target to keep the compression ratio the same, and checking which one suits the requirements best (more compression on noise, less on signal) Thank you for the pointer in the right direction

> it's hard to ask for a format to be that versatile Well, given enough tuning (or a custom encoder), JXL can be that format!

I'm noticing that depending on the images, I'm not getting the same compression (which makes sense). I know there is a way to specify a target compression with cjxl, but how to do it from libjxl?

which `cjxl` option are you thinking of? I don’t believe we still have it nowadays

target size is in benchmark_xl only

I see :/

Is there a way to achieve that anyway?

IIRC it was just a trial and error binary search for the nearest match to a certain threshold

indeed

I see, so it compresses the whole thing several times? Argh. JPEG2000's quality parameter was quite consistent in giving the same compression ratio on all my images. Sounds like I should just hope for the best then :/

it was never consistent or made sense for other formats, just a best effort that worked because it tried quality % until it approached the kilobyte number/ratio you asked for

jxl should do these approximations a lot quicker and since it identifies the most detailled parts of images can tune compression more agressively for those regions (without redoing compression for the rest or the full image - or the opposite, if youd prefer preserving those details)

The other, much more jank way, would be progressive encoding and then truncating the file. But only jxl-oxide can decode that currently. That's essentially what JPEG2000 did. There's no compression ratio target since images can have vastly different requirements, and we don't want people shooting themselves in the foot when a simple image is wasting a lot of data and complex images look unrecognisable

You could try `--disable_perceptual_optimizations` which might also remove the need for `--intensity_target` but memory usage and encode speed will suffer drastically

Oh! interesting... Speed and memory usage are both very important because it's time sensitive and on embedded device... So I'll just hope for the best 🙂

Keeping compression ratio constant means the same thing as having wildly inconsistent quality. There are few use cases where that is useful, notable exception being cases where there is a hard filesize limit imposed and you want to fully use it even if it is not needed.

A bit of a far fetch, but I don't suppose you're trying to do JXL encoding on a satellite before downlinking?

No, it's astrophoto sensor data

but onboard memory and processing capabilities are busy doing something else so these images should have as little impact as possible.

Yes the filesize isn't a strictly speaking hard requirement, but transfer time should be low for transfer stability reasons. Having a bit of a swing is expected, but basically what I'd like is "at least this much compression rate", then I have to roll with the quality that goes with it. In all likelihood, It'll be easiest to compress images with less noise, and what I want to compress is precisely the bottom of the histogram, the noise. In any case, if we find that we need a bit more quality, then we'll increase the quality.

It sounds like a custom encoder could be more than 100 times more effective in speed/quality for what you want, rather than stock standard libjxl

libjxl doesn't have the capability to target a specific average bitrate, and I don't think the quantizer is designed to compress a really, really dark image with a faint signal peeking through the noise

The libjxl devs also said they are afraid of making drastic changes to the lossy encoder cuz they're afraid of regressions and the lack of alternative encoders if they mess things up. Which is very poor reasoning in my opinion but whatevers.

I see :/ For sure a custom encoder would be miles better, but I don't have the knowledge nor the time. The devs could always hide the implementation behind an opt-in flag, so that doesn't seem very reasonable to say they're afraid of regressions ._.

If transfer stability is an issue, this may be of interest too https://github.com/libjxl/libjxl/pull/4062

Another idea... How much of the 16bit range is actually used? Maybe you could bitshift the image to raise the levels, similar to the [prior example](<https://github.com/libjxl/libjxl/issues/314#issuecomment-880620569>) where a PNG was incorrectly padded

The entire range except for the first ~200 values (dark offset of the sensor)

Interesting, this streaming option. I think it's gonna be just a file transfert though, and not stream-decoded, but having the option is great for a later iteration

It's just the middle part of the image is quite empty as nebulas are faint but stars are bright :/

(middle as in "histogram space", i.e. the middle values)

By the way, is there a standardized way to compress an image tile by tile? I couldn't really find information about that, except adding several frames.

There's a chunked encode api

I assume it is:`JxlChunkedFrameInputSource` I thought it was for streaming but maybe that's the same thing. Is this transparent for the decoder?

You still get just a normal jxl file in the end

Interesting, this new balancing histogram approach improves compression of large JPEG files up to 5 %. And it restores normal size sequence from effort 7 to 9: files become smaller, instead of paradoxical growth before between 7 and 8.

I think it is the result of getting closer to theoretically best histogram on nearly flat histos..

Nice!

5% is a huge amount, are you sure?

Yep, I was surprised as well

Default effort

Left is before - right after

Note 70MPix result

There is a jump from slight improvement of 2-3*10^-3 % to percents between 52 and 70 MPix size

Now if only I had enough RAM for 70MP Transcoding xD

The problem to get the best histo is interesting, but I suspect is NP-complete

What is the “best” histogram even mean

For the most compact encoding we need to prescribe frequencies of tokens is exact correspondence with their real frequency, but it is not possible with finite accuracy (1/4096 currently). So we need to choose a histogram "quite close" to the theoretically best with given accuracy of each bin. This is achieved by maximizing entropy of the histogram

So basically optimising for the quantisation of the histogram?

Yes, taking into account balancing histogram technique from the standard

Is there any speed or memory penalty to this?

Very small, usually only several improvement steps are taken

Nice, worth checking

Why can’t the histogram just increased the accuracy

12-bit precision is from the spec

It is possible by the standard (up to 16 bit, IFAICR) - but then you loose space on encoding the histograms themselves

ah 12-bit is for ANS, I think it's something like 15-bit for prefix code?

not very sure though

I will have to revisit https://github.com/libjxl/libjxl/pull/4130 after the better histogram balancing

Ah so the histogram also is signalled. Is it one histogram column for each hybrid uint configuration?

distributions are clustered, then there's one histogram and hybrid uint config for each cluster

where there are at most 256 clusters

I mean for inside the histogram the histogram is predicting of the each type of uint?

ANS histogram precision is 12-bit. Histogram signaling itself is a bit complicated.

What is puzzling me - that threshold of improvement, I suspect 64MiPix, but had not investigated yet

As in like does the histogram store the probables of each hybrid uint type?

No, first best uint and histo accuracy is chosen, and then stored

what?

isn’t the uint a type of value in the clustering of the accurate histogram, and then it quantises the bit depth?

Then clustering?

Depending on effort, different uints and accuracies are probed

Like each cluster of 256 clusters gets its own histogram

I don’t mean the encoding I am thinking of how this theory work of the how the theoretically stored and how the abstract data types

Like I am trying to understand the fundamental theorem here

Does the histogram store the probabilities of each hybrid uint?

Like hybrid uint config 1 is 53/4096 and config 2 is 124/4096

As far as I got it, each hybrid uint provides different separation between literal bits and remainder bits of each value to be encoded. Then literals should be compressed the best possible way - that is where histogram of literals distribution comes into play

Each uint produces different histogram and different number of remainder bits

On the topic of histograms, is there a reason LZ77 gets disabled in certain cases? I'm trying to debug why cjxl is outputting completely uncompressed files with certain parameters, and keep finding switches that set LZ77 to kNone I'd have expected that worst case it's a en/decode speed hit

The hybriduint config is per (clustered) context. The config defines how a symbol is split into a token and raw bits. E.g. in a typical config, symbol values 0..15 get their own tokens (with no raw bits) while for higher symbol values, the token represents only the exponent and some of the most significant bits, while the least significant bits are raw bits. This allows not needing a huge number of tokens even if the range of the symbols is large, while still getting most of the benefit of entropy coding — the raw bits are not entropy coded but since they are least significant bits that are usually pretty noisy, entropy coding wouldn't help anyway. The histograms represent the probability of each token value.

LZ77 is pretty much orthogonal to all of this: if enabled, a range of tokens is added to represent lz77 (distance,length) copy instructions. The main reason why libjxl doesn't use it much is because it generally doesn't help much for the type of data we are encoding (DCT coefficients, or in case of lossless, prediction residuals with variable predictors), while it is expensive on the encoder side to do the search.

The reason I ask is lossless faster_decoding 3 and 4 are meant to use LZ77 instead of an MA tree, but 3 and 4 disable LZ77...

Not to mention the switches seem flipped in cjxl but I'll dig into that later. I'm slowly working on a PR to fix both faster_decoding and progressive lossless. Thanks for the insight around it, I suppose using a fixed predictor makes it slightly more worthwhile

Oh I get it, I thought a hybrid uint config was a singular set of (exponent,implied bits), what you are called a token. So now it makes sense the histogram is storing the probability of each token?

And the hybrid config is stored along side

yes, every context has a hybriduint config plus a histogram for the tokens

So the first 2^split tokens are literals and then from there the tokens store (exponent-split, msb’s / lsb’s) ?

yep

ah that makes so much sense now! tanks

RLE is still just lz77, it just changes the search algorithm to essentially only look back one symbol

it keeps reading until it sees a different symbol and then it dumps a L/D pair

yup, smaller tokens are literals, otherwise what it does it the token splits into a higher order half and a lower order half

with RAW bits in the middle

how many of those most-significant-bits and least-significant-bits are in the token itself and not residue is determined by the hybrid-uint-config

for example, 4-0-0 means 0-15 are literals, and 16+ are hybrids. 16+ have 16 subtracted

`lsb_in_token = 0` means that 0 bits of the lower-order of the final result, are actually part of the token

`msb_in_token = 0` means that 0 bits of the higher-order of the final result are actually part of the token

both 0 means the entire symbol is residue

in this case, the value encoded by the token tells us how many bits of residue to read

or rather, that value - 16

but yea

so for a 4-0-0 config there's a fasttrack

`token < 16 ? token : readBits(token - 16)`

Yeah, but look at the last line of the second image. If speed tier is >=3, and modular mode is used, it's set to kNone instead of kRLE or kLZ77

I think that may be intentional since RLE helps very little when you have modular predictions

unless the MA tree is perfect, it makes more sense to disable lz77 than to try to go for rle

overhead not worth it

In the first image, speed tier >=3 disables MA trees and predictors too

So it causes a completely uncompressed JXL even larger than the image in memory

it also disdables MA trees and predictors?

well disable MA tree I assume means use a dummy one

you can't disble both at once

I wonder if we just hardcode gradiant predictor and call it a day if that's easier

or at least, west only

isn't there a "none" predictor?

zero, but yea

I figure disabling it means always predicting zero

which would allow you to just encode the actual value as a residue

that said I don't see much harm in using west-only for higher modes

doesn't require fast memory access

simdable across lines

etc.

Any of the first 5 predictors are fast, since they're standalone while the others are combinations of directions, using 6 directions for AvgAll. Fixed MA trees are also fast. Effort 2 decode is almost as fast as faster decoding 4 already. Lots of room for improvement, so gonna run some tests and see what works. At the very least, remove that if statement so it stops outputting uncompressed files

Does it also disable entropy encoding?

Seems like it

Heh, that size improvement is not my histogram - it is your extending counters in `ComputeCoeffOrder`

Oh, right, I suppose those counters would overflow for large jpegs

Yeah, I applied only extension (even without new buckets structure) and it gives this size drop

BTW, do you think it will worth it to apply more sophisticated algo to find best coefficient order? Now it is simple zero counts histogram, and it can be prone to coefficients correlations

Maybe?

That is a variant of patching consecutive ones property problem, that is unfortunately again NP-complete, but some heuristics can be applied

webp definitely retains a lot more detail here. I know libjxl is not meant to be optimized for this low quality setting, but maybe it's indicative of a more underlying problem with the encoder's visual decisions?

I've got some theorys on it, for now my resampling PR seems to compensate for it, but v0.8 also improves with the resampling for much higher quality results. So something can be changed

I think there are more fundamental problems with the encoder that affect both the low-quality and high-quality range.

Only a few very specific issues are specific to a certain quality range. Most changes have a similar effect on all quality levels.

There are lots of possible techniques and optimizations that are much more useful at low bitrates and completely useless at higher bitrates, but libjxl does not do anything like that since it's optimized for high bitrates, so that's not relevant here. Even without any optimizations for low bitrates, libjxl is performing a lot worse than it should there.

The most important issue is in the amount of psychovisual info preserved in the DCT transform itself

Always will be, always has been

I could not make the chunked API work. Is there an example somewhere, where it is used? I could not find one after looking for quite a while. :/ I set up `JxlChunkedFrameInputSource` with callbacks, which I then feed to `JxlEncoderAddChunkedFrame(frame_settings, true, chunked)` This essentially replaces the call to `JxlEncoderAddImageFrame(frame_settings, &pixel_format, image_data, image_data_size)`

for reference here are my callbacks: ```c++ struct ChunkingState { const uint8_t* image; uint32_t width; uint32_t offset; // Current offset in the image data uint32_t channels; }; void get_color_channels_pixel_format(void* opaque, JxlPixelFormat* pixel_format) { const auto* state = static_cast<ChunkingState*>(opaque); pixel_format->num_channels = state->channels; pixel_format->data_type = JXL_TYPE_UINT16; pixel_format->endianness = JXL_NATIVE_ENDIAN; pixel_format->align = 0; } const void* get_color_channel_data_at(void* opaque, size_t xpos, size_t ypos, size_t xsize, size_t ysize, size_t* row_offset) { const auto* state = static_cast<ChunkingState*>(opaque); const uint8_t* image = state->image; // Calculate the starting position in the image data size_t start = ypos * state->width * state->channels + xpos * state->channels; *row_offset = state->width * state->channels; // Return a pointer to the requested data return image + start; } void release_buffer(void* opaque, const void* buf) { } ```

Well, with <@207980494892040194>'s help, enabling LZ77 fixes both faster decoding and progressive lossless (Somewhat)

Need to do some more tweaking, but promising for just commenting out 3 lines

Worst case, it is faster *de*coding, so slower encode should be a decent trade-off. We'll see what we can figure out though

I did want to check, <@794205442175402004> I noticed libjxl has a function called `MakeFixedTree`. I always thought the fixed tree was an effort 1 and 2 specific thing, with the decoder using a lookup table to skip most of the MA traversal. Are fixed trees a more universal concept, signalled globally and traversed once then re-used or something similar? Wondering if using the fixed tree for faster decoding is a good idea, or if it doesn't apply to other decoders and could be broken (Much slower) if the tree is changed in future inside libjxl

Hopefully that makes sense

Effort 1 uses no tree at all (fixed tree with single node, I guess), MakeFixedTree is used for effort 2 and 3. But it is also used for some of the modular-coded stuff in VarDCT, including LF, iirc.

some MA trees have structure that can be fused into single lookup table to speed up tree traversal (which jxl-oxide also does), I think that's what MakeFixedTree is for

These fixed trees do have that property, since they're using a single split property

But the tree traversal speedup also works for non-fixed trees iirc, if it locally has a subtree that uses a single decision property

Okay, so it is a specialised speedup rather than a general rule, good to know

We'll probably have to test the faster decoding parameters on djxl, Oxide and Latte to make sure it's not an isolated improvement. Don't want to inadvertently create files that are *only* faster in libjxl

"textbook" MA tree traversal can be very very slow since it's super branchy and (maybe) cache unfriendly. I'd expect decoders that are advanced enough to implement at least some of the optimization techniques like lookup table thing

I think it's indeed reasonable to assume that decoders will implement fast paths. There is still room to do more btw. I am dreaming of some kind of JIT compilation of MA trees. It's just that those are dreams that are nightmares for people worrying about security surface, understandably.

(I think it can be done without serious security issues, termination can be trivially assured so it is very different from JIT compilation of some Turing complete language, but I can still understand some nervousness)

An image codec with self modifying code...

I think it's inherently unsafe for executable memory to be modified based on untrusted input. Web browsers get away with it by using a separate process with the OS kernel enforcing restrictions on what that process is allowed to access.

There's no cross platform way to do that

Which is a shame because it really should be the norm

Plan9 has process namespaces where each process has a restricted view of the system. And instead of linking binary code into a single process, multiple processes can communicate using the same read and write commands as everything else, if you want shared libraries that are isolated in their own restricted process. But other operating systems do not make it easy to write secure code. Therefore software is always going to be insecure by design until it's no longer difficult to write secure code. Nobody except web browsers want to write platform specific code for each platform they want to support having a sandboxed JIT

The 'programming language' would be just MA trees, so no loops, no writes, no reads besides to well defined previously decoded pixels, etc. Something very different from something like JavaScript.

But it's pretty inherently risky to give untrusted input the ability to modify executable memory. Also, to be honest I'm not sure how a JIT compiler can be made to run on a system with W^X enforcement

That's a gap in my knowledge

With a little cleverness you can probably make a very basic JIT with guaranteed safety, but the main security risk is the lack of separation of process and privilege namespaces inherent to modern programming.

None of this would be an issue if it was the norm for libraries to run in separate processes with minimum privileges instead of linking code into one process and running all processes with the same privileges as the "user"

It's a flaw in modern OS design

It's possible to make a library that handles untrusted input by creating a separate process that revokes its own privileges, but it's a real PITA

It's a shame. I wish security was easy and convenient instead of an afterthought today.

Yup, this! Did someone say WASM? :D

Sounds like a no-brainer for any kind of untrusted code thing

Also, prior art: harfbuzz supports a font hinting/shaping bytecode which _is_ WASM

If you want a laugh: https://fuglede.github.io/llama.ttf/

WASM is way slower and way more complicated than having libraries running in a separate process and opening/reading/writing to file descriptors instead of linking binary code into a single process, and having the OS and kernel isolate processes into their own namespace with limited access to the system.

Basically if the OS was better designed and provided easy to use APIs for writing multiprocess code that revokes privileges and communicates with other processes through file descriptors, similar to Plan9

But security is not a priority for OS devs

then everything would be slower due to context switches 🙃

Not as slow as WASM, no.

Linux already has namespaces and multiple processes running as different users and in different namespaces and containers and it isn't slow at all. It's fast as native.

There's nothing slow about this at all. Linux already technically has all of these abilities but they are not made convenient and easy to use.

Sure about that?

And developers also have this mindset that linking code into a single process is taken for granted as the "normal" assumption of how to do things without questioning if it makes sense or if it's actually good or bad design

Besides, what you are asking for already exists and is called seccomp mode 1

Plan 9 isn't the be-all and end-all of OS design

Try creating a separate namespace with systemd-nspawn and run some benchmarks inside the container, and see if it's any slower. It shouldn't be.

You were talking about passing data between processes via file descriptors, not about benchmarks in a namespace.

Of course not, but it's a shame that no one learned anything from the mistakes of the past.

Linux namespaces have pretty much no relation to secure computing-only processes

Well then the main slowness there would probably be serialization and copying of data, since it would be going from 1 process into 2.

Correct. That's what I'm talking about.

But that is a necessary payment for the security of process separation

Well worth the cost

Which can also be achieved without process separation by WASM.

Suddenly, you don't have to worry about malicious input anymore. That's a ridiculously massive win

WASM is an even bigger compromise, and it's way more architecturally complicated.

"Architecturally complicated" is subjective, and could you share data on why exactly it is a bigger compromise?

It's a JIT compiler

Yes, and?

It is also architecture-independent.

It has the same overhead and complexity as trying to mix Java and C together

That's just not true.

with multiple processes it's all plain old boring ass code

native code

with wasm you would have multiple processes anyways. Unless everything is all compiled to WASM together

You don't gain or save anything

Who said that? That's just blatantly wrong.

maybe I'm misunderstanding something?