JPEG XL

not even just libjxl it's also the format itself correct?

yea

Ohhh, I see... so it should never be set to true when lossy. You can still embed whatever color profile you want, though, in the jxl file. It won't affect how the binary image data itself is encoded though

It will just be there in case someone wants to know what the original or intended profile is

I have a question regarding the jxl-rs BT.709 PR, it looks like it is using oetf and the inverse oetf when going to linear. As far as I know according to BT.2087 one should use either 2.4 power or a 2 power for doing conversion. Is there any reason why jxl-rs should have the inverse function? or even the encoding function considering it's use is supposedly purely for a "source" capacity?

It makes sense to add encoding to jxl-rs later down the line

It may be beyond the scope right now, but that might not hold true in the future

It will almost certainly outclass libjxl at decoding and might become the preferred jxl support library

Once it's complete it's possible many projects that don't need encode will stop using libjxl entirely

It's likely to be much cleaner and simpler for outside developers to read and start hacking on too, since it's written from scratch.

even then, I can understand maybe having the encode function for an encoder since maybe some sensor wants to be able to pipe data directly to libjxl/jxl-rs if it does get encode support, but I don't believe there is a lot of sense in having the inverse function

agree, the oetf has nothing to do with how images should be displayed, so 2.4 would make way more sense

What about in the case of decoding an arbitrary lossless image with an arbitrary colorspace, to an arbitrary output colorspace

in that case the chances that you want the inverse oetf is extremely slim [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&name=av1_dogelol)

it's copied from jxl-oxide. oetf is used when decoding xyb encoded images, because xyb-to-rgb procedure returns linear samples. inverse oetf is used when the input is lossless and some other color encoding is requested. lossless samples are already encoded with a specific transfer function, so it needs to be converted back to linear.

(of course the result won't be lossless in that case 😛, but jxl-oxide allows requesting other color encoding)

It won't be lossless but sometimes you want to decode to whatever it's going to be displayed as

right, but the bt.709 oetf exists to achieve a certain non-unity end-to-end gamma, and really shouldn't be used for anything other than encoding scene light and for image colorimetry, you always (?) want to work with display light instead

doing inverse oetf -> xyb -> some other format will result in a wrong image. Like wise, any processing done may also result in the wrong image assuming the processing is supposed to take into account the stuffs.

(and the only use for the inverse oetf that i'm aware of is making certain adjustments like exposure in scene light)

also likewise, going from XYB -> "rec 709" should almost certainly be done using BT.1886 unless you are dumping raw camera info into an xyb encode without doing the pre-requisite processing

There is a case to be had for that granted, but im just not sure how that pipeline would look and if the oetf would be valid there.

pretty sure libjxl does the same, fwiw

well the spec says "As specified in ITU-R BT.709-6" for `k709` transfer function kind, so I think we should follow that. if BT.1886 is intended then it can still be signalled as gamma 2.4

I was also going to say that 😄

uh well it seems that bt.709 refers to bt.1886

maybe not, color specifications are such a mess

I usually invoke <@604964375924834314> whenever I have doubts about this stuff

Just one tiny nitpick. It's < 0.018 not <=

Likewise for 0.081

At least according to wikipedia

doesn't really make a difference 😛

it's supposed to be continuous so it shouldn't matter so much

The difference is arguably infinitesimally small even

in the end, as far as I know, bt.1886 is intended to be the "accurate as possible" way to decode rec.709 footage while taking into consideration issues that CRT's "generally had" as for the hard "it should be decoded using a pure 2.4 or 2.0 power" comes from https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2087-0-201510-I!!PDF-E.pdf page 3 (of the document, page 5 of the pdf)

But it does save you 2 less characters

``` Non-linear to linear conversion from normalized R′G′B′ colour signals E′RE′GE′B (Rec. 709) to linearly represented, normalized RGB colour signals EREGEB (Rec. 709) is accomplished by one of two equations which produce slightly different colours from each other: Case #1: In the case where the goal is to preserve colours seen on a Rec. 709 display1 when displayed on a Rec. 2020 display2, an approximation of the electro-optical transfer function (EOTF) from Recommendation ITU-R BT.1886 (Rec. 1886) is used: 𝐸 = (𝐸′)2.40, 0 ≤ 𝐸′ ≤ 1 Case #2: In the case where the source is a direct camera output and the goal is to match the colours of a direct Rec. 2020 camera output, an approximation of the Rec. 709 inverse opto-electronic transfer function (OETF) is used (see Annex 2): 𝐸 = (𝐸′)2, 0 ≤ 𝐸′ ≤ 1 ```

well I was hoping copy and paste would format that a bit better, dunno why, it never does.

note: 2.4 gamma isn't just an approximation, but it's exactly what you get on a display with 0 black levels (e.g. oled)

it's not that 2.4 is an approximation, it's that 0 black levels is an approximation [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&name=av1_dogelol)

so it seems that bt.709 basically says, use bt.1886 eotf when decoding to display

and jxl decodes xyb to display-referred linear (this is why we need hlg ootf when encoding back to hlg signal) so we need bt.1886 inverse eotf

not only when decoding to display, but also when decoding to linear for the purposes of color conversion, since jxl want's to keep the intent of any color, decoding and encoding using 2.4 is prefered

yes, though it gets complicated, since TRC and EOTF/OETF aren't quite teh same thing

if the TRC is tagged as BT.709, by spec, if you're linearizing for the purpose of presentation you should use BT.1886

If you're linearizing *not* for the purpose of presentation (e.g. if you're scaling in linear light) then you use the inverse of the TRC of BT.709 specified in H.273, and then you use the forward on the way back

really? I've always used 2.4 as per rec.2087 even while doing things like colorwork, while rec is specifically about rec 709 -> rec 2020 I have found that to be the optimal results.

but if you use the bt.709 oetf (+ its inverse), you're specifically scaling in scene light and not display light. which is almost never what you want

unless you're specifically trying to simulate something like an exposure adjustment in a camera that uses the oetf

I think you might be confused? scene light, or more accurately referred to as scene referred or scene linear, *is* linear light. Linear is the direct energy values of light. Display light is the signal when the oetf is applied. you typically, not always but usually, want to scale in linear colorspace.

not sure if i'm using the terms right, but what i meant by those is: scene light: the linear light in the real world that goes into a camera display light: the linear light that's emitted by a display when showing the picture

Yes, Display light is light that has been modified by an OETF and Scene light is the linear representation of light in essence, it can get a bit more complicated but in the end, "scene light" is what most scalers typically will prefer to work

the issue with the rec709 transform is that it was in essence "a transform to make content look OK on a CRT" which means it had to be both computationally simple and "generic". Rec.709 is *Not* a hard spec, it is a recommendation for cameras to use, and camera's almost always modified the spec to make it look good, because not all cameras look the same, and not all displays looked the same. However we *do* know that regardless of source image, rec709 encodes are designed to do one of two things, Look good on a CRT, or look good on BT.1886, which is a result of trying to emulate CRT displays. If the intended destination of your encode is a 2.4, or at least, can be most reliably represented with 2.4, then using the inverse oetf is wrong. while in **some** cases you may better preserve the source data, you do not preserve the *intended data* from the artist or device. Using the inverse rec 709 oetf to me only make sense in an authoring workflow and only when you know the source data is encoded in an actual 709 oetf, which as I stated before, is not always the case, and for context I will leave a quote from jack holm https://community.acescentral.com/t/where-exact-numbers-of-rec709-formula-came-from-and-a-question-about-p3/3940/5 ``` It’s all about what results in a pleasing image on the display! ... The Rec 709 OETF is a “reference” OETF – a reference for camera makers and for conversions of scene-referred colorimetry to the Rec 709 encoding. In practice, camera makers may use slightly different OETFs to optimize results for specific camera characteristics and to produce their desired default looks for their cameras. ... It is not reliable to assume that the Rec 709 OETF has actually been used, unless this is known to be the case. ... A far better reference for the desired colorimetry is the Rec 1886 EOTF, and even in this case one is assuming the content was mastered for the Rec 1886 reference display and viewing conditions. ```

the reason why this doesn't really apply to sRGB is because A) the "collection of official and unofficial sRGB specs" have a long history of contradictory information that has caused both displays and produces to encode and display sRGB content in a wide variety of wrong ways. B) It is generally safe to assume that content that has been produced via a linear -> display transform would have been done using the sRGB oetf, however when doing color work, this discrepancy in how the author likely wanted it to be displayed needs to be taken into consideration and compensation needs to be done C) no one cares about sRGB, everyone just kinda agrees that sRGB will never really be consistent so we can only do a "best guess"

tbf if the input is tagged as rec709 with cicp or similar, you have nothing else to work with other than the assumption of reference

so if your goal is to do some sort of image processing (e.g. scaling) in linear light, then roundtripping through the function defined in H.273 (and its inverse) is kinda the best you got

also, for sRGB, it's not fair to say that you never know. if, for example, libjxl receives an image tagged as sRGB (without an icc profile, just tagged, e.g. sRGB chunk png) then it will use a specific linearization when converting to XYB, so if you're decoding a lossy JXL, it's safe to assume that the formula in H.273 is the correct one if you want to produce a PNG file tagged as sRGB

since all current JXL encoders use that formula to linearize before applying the Opsin matrix, then any decoder can reasonably use it after applying the Opsin Inverse Matrix

It's just a transfer curve...

thats not really true though. We can reliably assume the destination is going to be either BT.1886, a CRT, or a display that roughly mimics a CRT's characteristics. also for sRGB while what JXL does is reliable, it's not reliable to assume how the png itself was crafted. we don't know if the host monitor was a pure 2.2 decode, or an inverse sRGB oetf, nor do we know if the source application used a pure 2.2. The best thing we can do is guess in that case. sRGB will always be a mess lol. even if jxl's solution is perfect, garbage in garbage out

in that case, we should roundtrip with the bt.709 curve defined in H.273

I don't really see the benefits of that. since we know that a pure 2.4 power curve is the most accurate for decoding rec709 as bt.1886 establishes, doing work in a 2.4 linearization seems to be the most accurate when it comes to doing processing. rec 2087 also concurrs that when doing rec 708 -> rec 2020, you should do linearization utilizing the the approximation of the bt.1886, the pure 2.4 power curve. It does also explicitly acknowledges that when the source is a direct camera output, and the goal is to match the colors of a "rec 2020 camera output" that you should use an approximation of the inverse 709, which they use a 2 power for. In nearly all cases that we would care about, the goal should be to preserve the colors as seen on a rec 709 display, and only in the cases were we know for sure that this isn't the goal, should we use the inverse 709 oetf, or an approximation of it

ofc that's not to say the inverse oetf should never be used, if I am recording footage and importing said footage into a video editor, I will often default to using the inverse oetf, but that's simply because the color is better to work with, and im using it to craft ofter things. If you are taking already mastered content, the 2.4 should be used.

Perhaps <#1288790874016190505> so people can find this more easily later?

uh maybe https://github.com/tirr-c/jxl-oxide/issues/318 is actually related to this

yeah, it seems to preserve colors if I modify libjxl to use gamma2.4 on BT.709 input

nevermind, I was comparing with wrong image

I forgot to modify ICC profile generation part; when I modify it to generate gamma2.4 trc, it does preserve colors

the discrepancy between offsets wouldn't cause such a harsh difference in general, though it would cause some

like, without the fix, bt709 to jxl-xyb to srgb doesn't roundtrip visually

srgb-decoded version looks too bright on dark areas

im not sure im following

ill try to re-read the issue

Encode an image tagged as BT.709 to jxl. Then decode the jxl to BT.709 (no options) and sRGB (`--color_space=RGB_D65_SRG_Rel_SRG`). I expect those two decoded images should look identical, but with current libjxl the sRGB one looks too bright.

(yeah, now I think this isn't a roundtrip)

when I modify libjxl to use gamma2.4 on BT.709, two decoded images look identical

ah I see. sorry, had a hard time following.

so libjxl is currently applying inverse OETF to encode a BT.709 image so it's encoding in scene light. libjxl also applies OETF when decoding so it works okay when decoding to BT.709, but it breaks when sRGB is requested, because sRGB inverse EOTF works on display light (if I'm getting this right)

that’s odd – in an ICC workflow, they should look identical regardless of the curve, because the curve that’s used in decoding should match the one in the ICC profile that is attached to the file

are you maybe viewing them in Chrome and it’s giving precedence to the CICP chunk?

because libjxl doesn't use ICC workflow when decoding XYB image? I guess it's just FromLinear stage that ignores tagged transfer curve

I meant in viewing the image

well macOS Preview, Chrome and Firefox all agree that sRGB one is brighter than BT.709 one

im a little confused, ill try to break this down. jxl gets image - applies inverse oetf - converts to xyb - converts to linear sRGB - applies rec 709 oetf - rec709.png jxl gets image - applies inverse oetf - converts to xyb - converts to linear sRGB - applies srgb inverse oetf - srgb.png ? the first one will look fine since it is undoing the transfer effectively, but as far as I am aware, the latter will look wrong.

libjxl will ignore the tagged curve when requesting a specific output colorspace, but which one is requested shouldn’t affect how the decoded image looks, because libjxl should attach an ICC profile to it that matches the curve it effectively used

if the “original image -> xyb” part is wrong, it should be wrong equally for both of them

that the original image happened to be in BT.709 should be relevant only to whether the decoded image looks like the original

not whether the two decoded images look the same

at least as far as I can tell

ah I think I got what you mean, yeah they seem to give precedence to cicp tag then

that's what I think libjxl is doing, yeah

Chrome shows images look identical if I remove cicp tag from the ICC profile, but in a way that looks too bright

in general the top one is fine since you are just reversing the change, the real issue comes when you apply the inverse oetf and go to different spaces. It may not be so bad in many cases, but in others it might be a rather different result

Hey, I extracted the zip file jxl-x64-windows-static and added it to my PATH, but I still cant use it. (I am on windows 10)

What command are you trying to run?

djxl --version

Hmm

https://github.com/google/jpegli

any nightly binaries that are also accessible publicly ?

is there ready avaialble simple package for a .dll .so and .dylib builds of libjxl?

I originally discounted this as 'higher effort is higher quality' or patches being used, but twice the file size in an image where no patches should be found... It might be worth looking into as a bug, especially as it's a 16bit image https://www.reddit.com/r/jpegxl/comments/1htswuo/comment/m5unvcc/

> The reference image is a blender rendering for GNOME wallpapers (this particular image is this). I got the original blender assets and re-rendered myself to a higher quality and preserve noise, which is what I'm encoding in the screenshot. The image has some unpredictable noise, but the results seemed pretty good for the file size. It is a reduction from ~400MB (5000x5000x16bit) to some megabytes. I simply found strage the file increase the better the effort.

I don't think its a bug. For images with a large flat areas, the more context libjxl has the better it can achieve its target quality. It seems to detect that it's wildly under shooting the desired butteraugli score and increases the bitrate to compensate.``` cjxl glass-chip-d.jxl glass-chip-compressed.jxl -d 0.5 -e 9 Compressed to 2669.4 kB (1.273 bpp). SSIM2 Score = 84.56 butteraugli = 1.36 cjxl glass-chip-d.jxl glass-chip-compressed.jxl -d 0.5 -e 10 Compressed to 4550.3 kB (2.170 bpp). SSIM2 Score = 88.63 butteraugli = 0.72 ```

A ssimulacra2 score of 84 is quite low for a distance of 0.5.

Granted I'm working with the lossy image from the reddit post but I'm gonna retest with a clean source.

higher file size for higher effort usually means that the lower efforts are undershooting the quality target

His image source is 16bit and 5K instead of 4K, and with real rendering noise instead of photon noise. Though, the results you've posted do match rather well

I doubt you can relay the find as a comment on Reddit?

I do have a reddit, lemme dust off my old account and get to it.

Still thinking about this... I wonder how much tweaking and tuning it'll take to be as good as the old method. It seemed to make encoding slower and result in worse Ssimulacra2 scores, which our eyes have been agreeing with ever since https://github.com/libjxl/libjxl/pull/2836

It would be nice to have multiple completely different approaches for encoder heuristics, either with multiple settings of libjxl or by having more alternative encoders.

It'd be nice, but the use would be limited to those who know what setting fits that particular image best. Back to `-tune`... Since this regression was caused by the metrics giving different results, and only Ssimulacra2 was right, it could even hurt performance

I am not in favor of manual per-image tuning, but having some healthy competition between encoders (all aiming to be good general-purpose encoders) could be a good thing that over time leads to improvements.

Currently with only one good encoder, we're a bit suffering from the problem that only small incremental changes are made at this point since doing big changes is too scary. That would be less of an issue if there are multiple experimental encoders around...

Yeah. Looking at the Hydrium encode I just made, it's even using the same block decisions as `-e 7 -d 0.6`. So we haven't seen any other strategies past using cjxl 0.8

Looking forward to trying this https://github.com/libjxl/libjxl/pull/4062

Though is the corrupt DC setting the output black, or only the DC so the AC can still show?

The thing is that AC metadata is sent together in the same section with DC groups so if DC is corrupt, the AC metadata will also be corrupt.

So you cannot really decode or show AC if something is wrong with the corresponding DC group

Ahh right

Someone should fork libjxl-ng

Maybe those psy tuning avif guys...

But it really just needs a massive code cleanup more than anything

Simplifying the directory tree and separating out the main library code from the tool code

That would make it way easier for newcomers to hack on it

why not an `experiments` branch for the encoder, or specific branches for specific experiments?

I've been talking to the psy av1 guys and they have been working on a psy fork for jxl

Currently libjxl has multiple different very distinct encoders or encoding modes. It's more accurate to call it a single common frontend to multiple different encoders than to pretend that it's just a single encoder.

The vardct stuff is pretty separate and contained and so is fjxl

Also the Squeeze mode has incredible, insane potential considering how it looks better than vardct despite vardct getting all the tuning

Also I heard there's another interesting palette based encode mode that seems super promising too especially for near-lossless coding

I forgot the name though

Honestly all these different encoders could be split into separate folders or something and fjxl can be the default

Since the speed will instantly knock peoples' socks off and you can never go wrong with lossless

Δ palette?

Yeah, lossy palette or delta palette

Something like that

Seems like a promising and underused/underrated option

With this many coding tools which can be combined in interesting ways, I wonder if we're going to see encoders tuned for every specific purposes. Like one specifically for screenshots that might eagerly use patches for text, a frame with most of the rest of the image in Modular mode and a VarDCT frame for photographic sections that happen to be in the screenshot, like a wallpaper photo

Maybe JPEG XL could do with an encoder *engine*(?). A library which gives you the low level building blocks a JXL encoder would need, and allows you to create an encoder tuned for a specific purpose yourself. In a similar sense to Smithay, which is not a Wayland compositor but provides the things you need to build a Wayland compositor, which several compositors are using

But I'm just thinking aloud

If you wanted to create an encoder that makes use of a specific combination of coding tools that seem useful for your purpose, you bind to the library and write relatively little code to define the logic of your encoder. Though maybe this can already be achieved with the libjxl API, I just remembered that's a thing and I'm not familiar with it

I have tested `webp` `libx264` `libx265` `AVIF` `AVIF` produces smallest size but biggest CPU usage then `x265` `x264` was the smallest cpu usage with a medium size `webp` good quality and size but also high cpu My software will encode images constantly send them through a websocket and decode them on the server side What about libjxl

I haven't compared them, but with cjxl you can fairly easily change the CPU usage of the encoding process using the effort flag `-e`

webp and libjxl both have a range of possible speeds, but at any given speed, libjxl compresses better https://cloudinary.com/blog/jpeg-xl-and-the-pareto-front#the_pareto_front

you can experiment with `-d` or `-q` for quality and `-e` for encoding effort (how hard it'll try) to strike a balance that fits your purpose

here is lossless compression

unlike avif encoders which I wouldn't personally even attempt to configure

Cool, i'll try the c lib

I'm encoding images of size 960x540, while trying to maintain a quality of around 60%

The graphic you shared doesnt include `libjpeg-turbo`, have you ever tested it?

this graphic is for lossless, which libjpeg-turbo doesn’t do

see further down in the article for lossy

(where libjpeg-turbo is featured)

There's a package on vcpkg listed as `libjxl:x64-windows` but its not mentioned in the [documentation](https://github.com/libjxl/libjxl/blob/main/doc/developing_in_windows_vcpkg.md), is it "official"?

libjpeg-turbo actually does do lossless, but it is pretty much always worse than PNG

So libjxl could be a common front end and have some way of selecting the best backend encoder for each image (or even section of image)?

Is it not possible to cache the encoder, frame and frame settings?

Cache the encoder?

Like CPU cache?

Or like if an image is being re encoded ?

```c++ class Jxl { public: struct Data { int width = 0; int height = 0; int stride = 0; BYTE* Scan0 = nullptr; using client = websocketpp::client<websocketpp::config::asio_client>; client* wsClient; websocketpp::connection_hdl* connection; } _; std::vector<uint8_t> _compressedData; JxlEncoderPtr _enc; JxlBasicInfo _basicInfo; JxlPixelFormat _format = { 3, JXL_TYPE_UINT8, JXL_NATIVE_ENDIAN, 0 }; JxlEncoderFrameSettings* _frameSettings; Jxl(const Data&& data) : _(data) { _compressedData.resize(223887); JxlEncoderInitBasicInfo(&_basicInfo); _basicInfo.xsize = _.width; _basicInfo.ysize = _.height; _basicInfo.num_color_channels = 3; _basicInfo.bits_per_sample = 8; init(); } bool init() { _enc = JxlEncoderMake(nullptr); if (!_enc) throw std::runtime_error("JXL encoder initialization failed"); JxlEncoderSetBasicInfo(_enc.get(), &_basicInfo); if (_frameSettings) { JxlEncoderFrameSettingsCreate(_enc.get(), _frameSettings); //_enc.get()->encoder_options.emplace_back(_frameSettings); } else { _frameSettings = JxlEncoderFrameSettingsCreate(_enc.get(), nullptr); // Testing encoding settings similar to WebP JxlEncoderSetFrameLossless(_frameSettings, false); JxlEncoderSetFrameDistance(_frameSettings, 13); JxlEncoderFrameSettingsSetOption(_frameSettings, JXL_ENC_FRAME_SETTING_EFFORT, 1); } return true; } bool encode() { if (!init()) throw std::runtime_error("JXL encoder initialization failed"); if (JxlEncoderAddImageFrame( _frameSettings, &_format, static_cast<void*>(_.Scan0), _.stride * _.height) != JXL_ENC_SUCCESS) { return false; } JxlEncoderCloseFrames(_enc.get()); uint8_t* next_out = _compressedData.data(); size_t avail_out = _compressedData.size(); JxlEncoderStatus processResult; do { processResult = JxlEncoderProcessOutput(_enc.get(), &next_out, &avail_out); if (processResult == JXL_ENC_NEED_MORE_OUTPUT) { size_t offset = next_out - _compressedData.data(); _compressedData.resize(_compressedData.size() * 2); next_out = _compressedData.data() + offset; avail_out = _compressedData.size() - offset; } } while (processResult == JXL_ENC_NEED_MORE_OUTPUT); if (processResult != JXL_ENC_SUCCESS) return false; _compressedData.resize(_compressedData.size() - avail_out); websocketpp::lib::error_code ec; _.wsClient->send(*_.connection, _compressedData.data(), _compressedData.size(), websocketpp::frame::opcode::binary); //_compressedData.clear(); return true; } }; ```

```c++ class Webp { public: WebPConfig _config; WebPPicture _pic; WebPMemoryWriter _writer; struct Data { int width = 0; int height = 0; int stride = 0; BYTE* Scan0 = nullptr; using client = websocketpp::client<websocketpp::config::asio_client>; client* wsClient; websocketpp::connection_hdl* connection; } _; Webp(const Data&& data) : _(data) { WebPConfigInit(&_config); _config.quality = 60; // 0-100, lower = smaller file _config.method = 0; // 0-6, higher = better compression but slower _config.lossless = 0; // Use lossy compression _config.pass = 1; // Number of encoder passes _config.thread_level = 0; // Enable multi-threading //config.near_lossless = 90; WebPPictureInit(&_pic); _pic.width = _.width; _pic.height = _.height; _pic.use_argb = 0; WebPPictureAlloc(&_pic); WebPMemoryWriterInit(&_writer); _pic.custom_ptr = &_writer; _pic.writer = WebPMemoryWrite; } bool encode() { if (!WebPPictureImportRGB(&_pic, _.Scan0, _.stride)) { WebPPictureFree(&_pic); return false; } if (!WebPEncode(&_config, &_pic)) { WebPPictureFree(&_pic); return false; } websocketpp::lib::error_code ec; _.wsClient->send(*_.connection, _writer.mem, _writer.size, websocketpp::frame::opcode::binary); WebPPictureFree(&_pic); WebPMemoryWriterClear(&_writer); return true; } }; ```

I'm asking if its possible to "cache" the encoder, create it and its settings only once and reuse to encode next images. I'm more constrained by CPU usage.

Can you use the same context to encode multiple images?

You shouldn't need to... I think the best possible approach is to optimize each encoding tool as much as possible, even if some tools like patch detection need to be run in their own separate pass, and then make it easier for the individual tools to be combined in effective ways, depending on the effort level.

Like with more effective image decomposition and layering techniques than just patches

Grain detection, spline detection, cartoon layer decomposition even

Imagine if the encoder could automatically tell when DCT is ineffective

So you don't have to manually specify different options for different types of images

That's the dream

Then you don't have to worry about those cases where lossless is actually smaller than dct

These are plans for the far future though, the major milestone this year is the completion of the rust decoder and (hopefully) its adoption by Mozilla.

Yeah this would be good, but heuristics will always be hard to get right. So if you know you're only going to be encoding certain types of images you can specifically tune your encoder for those, instead of hoping the one true encoder makes the right guesses. For example a scanner in 'document mode' would spend more effort on patch detection, and using splines for lines like borders of tables and pen strokes like signatures, while in 'photo mode' it might just spend a lot of effort on the VarDCT stuff

Just because it's hard doesn't mean we shouldn't pursue it. I don't believe adding tunes is the right direction for libjxl. It's certainly the easy way out but I think it would make things more complicated later on. There is *so much* that can be done to make the encoder smarter. I believe with good heuristics the codec can make good decisions 99% of the time. It's hard to imagine a "dumb codec" existing in the far future.

To be clear I wasn't suggesting these domain specific encoders be made a part of libjxl or developed by the core devs. Rather just existing on their own where they would be useful. I agree that a smarter encoder should be what the effort is poured into for libjxl. It's just that I could see some use cases where you basically know what your data will generally look like ahead of time, and then you could save massively on encode time if it's not trying really hard to figure out things that you already know

At such low effort settings, libjxl essentially works as a JPEG encoder <https://github.com/libjxl/libjxl/blob/main/doc/encode_effort.md> I'd maybe suggest looking at jpegli if not checking what version libjxl you're using and that hardware instructions (SSE, AVX, ect) are enabled properly. On my tests cjxl is working at identical speeds to cwebp following your settings above (Though the webp is 70KB and much higher quality than the 19KB JXL I got...)

I didn't test `jpegli` ill try it, do you? I also tested turbojpeg and from all libs i have tested until now, it was the one with lowest CPU usage.

Finally got round to checking if it's working in Krita, and yep. A lot of settings seem to have no effect on it, but it scores 5 lower in ssimulacra while preserving more texture to my eyes at a similar bpp to VarDCT (Around distance 0.5) Original, VarDCT, Delta

Also <@274048677851430913> the EPF setting in Krita's export dialogue can't be set to -1 to let the encoder choose. Group ordering wasn't working either but you already fixed that for the next release IIRC

Metrics are guiding people into blindly (literally) making really bad decisions

I kinda agree with ssimulacra2 here. Lining up the images side-by-side, I could see that delta palette has a somewhat fuzzier look compared to the original even at 1x zoom. VarDCT was a tiny bit smoother but I had to zoom in 2-3x with a flicker test to see it.

I'm on a phone atm and can't compare them

Those shadows look ridiculously bad though

I don't understand why Unreal engine is so ugly and bad these days

DeltaDoggo is sharpened a bit comparing to the original or vardct

I was on Irfanview so it resampled at 'full' size, then I went to 500% to spot the differences. Delta has more noise, as expected with reduced colors essentially dithering to compensate, but to me it was preferable to the slight smoothing. In the end, both are good enough and it's surprising how well Delta Palette holds up for not being tweaked in years

That was actually over 2 years ago

At 100% I can't see any difference to the original, 200% I start noticing the noise around high contrast edges, 500% I can see the pallete reduction affecting gradients and the edges are obvious, but ignoring areas like the mouth, I could still think it's the original

Mozilla hardly matters at all anymore, hardly anyone uses firefox now that it's essentially in maintenance mode aka life support

What matters is reversing the decision to remove it from chromium

Maybe Firefox support would be the tipping point for that? But that is kinda silly considering Safari has way more users

I think the rust decoder has more leverage than people think

Maybe I'm being a bit naive

Maybe I'm being a bit naive in underestimating the influence mozilla has because of their history and (former) reputation despite having almost no users anymore

imo it has pull power. sites with a proper jxl strategy could end up more lightweight than on other browsers - saving money, becoming profitable instead of not breaking even, having more room for instantly loading ads that dont slow site

just making usable good code isnt everything though, there should still be militant adoption lobbying for important services like image hosts, cdns

Also I know js/wasm decoders have disadvantages like waiting for DOM to load for example but most of the time no one is going to notice and you can even have progressive decoding in browsers that don't support it yet using that method

So I'm honestly surprised it's not used more often

If people just started using it more often with js/wasm fallback there would be more impetus for browsers to integrate native support

custom webdev is hard enough, wasm should be part of that but many webdevs arent even aware jxl exists or how people can consume it without safari

Nothing is really stopping all websites from using it immediately

In all browsers

You literally just add 1 line of code to your html pointing to the js decoder

It requires no brains at all to start using

link it ?

im aware of a ludicrously outdated one, anything better ?

oh thanks for the report, on it

https://github.com/niutech/jxl.js

I just know about this one. It's braindead simple and just works so what more is there to ask for

iinm magick wasm can handle jxl but some of such solutions are supposed to be part of custom webdev

small sites wouldnt move the needle though, ones with high reach/impact ought to push (highest being cdns then photo/image hosts)

take flickr or photobucket, its 2025 and neither supports dng 1.7

flickr is egregious since its parent company (smugmug?) supported jxl

There was this on Reddit recently too https://bevara.com/

Disagree strongly. People aren't even aware that it exists rn. People just need to start using it, doesn't matter who, just to prove it's possible and so people start running into it more often

DNG 1.7 is mostly unsupported due to libraw not releasing with support yet. I think that was resolved but the next release is still months away. Then RawTherapee were concerned about the Adobe SDK with the libjxl decoder causing licence issues. https://github.com/Beep6581/RawTherapee/pull/6887#issuecomment-2241745859

It's much easier for small sites to flip the switch than big ones

Less process and bureaucracy

Unfortunately even the apps from the LibRaw developer still does not support jxl-in-dng

Yeah, it's in a 2024-03 snapshot, but not an actual release yet https://github.com/LibRaw/LibRaw/issues/615#issuecomment-2027946301

i think the raw plugin for windows 10/11 got upated with that wip code without waiting for the next big update, worth rechecking

We checked it a while ago but there was a bug causing it to error

JPEG XL DNGs with JPEG previews load, but JPEG XL previews fail

```[EXIF] Subfile Type : Full-resolution image [EXIF] Image Width : 5712 [EXIF] Image Height : 4284 [EXIF] Bits Per Sample : 16 16 16 [EXIF] Compression : JPEG XL [EXIF] Photometric Interpretation : Linear Raw [EXIF] Subfile Type : Reduced-resolution image [EXIF] Image Width : 5712 [EXIF] Image Height : 4284 [EXIF] Bits Per Sample : 8 8 8 [EXIF] Compression : JPEG [EXIF] Photometric Interpretation : YCbCr```

remember those results represent the specific case tested and should not be assumed general <https://github.com/jxl-community/jxl-community.github.io/issues/41>

It would be an interesting research project to create a corpus of ~10k images that is representative (in terms of compression) for "all images". How to construct/collect such a set is very nontrivial though. Just crawling the web will not do, since the set of images on the public web is not representative for all actual images (e.g. private photos, medical images, game graphics etc would be underrepresented), and also because you typically only get lossy downscaled images that way, not originals.

Wikimedia Commons might be a good starting poing

Yeah but that is only representative for the specific category of images suitable for wikipedia, which is likely not distributed in the same way as "all images", e.g. it will have more diagrams, logos, maps, and pictures of famous people and landmarks, and fewer random holiday pictures, insurance claim pictures, medical images, etc.

Wikimedia Commons has tagged images (or more like categories it seems) with a lot of different genres, I was just looking at the "pictures of art" category but there are many. I expect those present on Wikipedia (which is its own category) are probably a large portion of all images though yeah

some examples

And yeah those random images you mention, probably not many of those

I guess the question is: how to estimate the distribution of non-public images, like stock photography, medical archives, images on all kinds of forums, chats and social networks, screenshots people keep on their phones or computers, etc.

But yeah, maybe a representative sample of wikimedia would be at least representative of _something_ that is also relevant.

Estimating the distribution of types of images sounds hard yeah. It would still be interesting to see if it does particularly poorly on certain types of images without initially knowing how common they are. You could figure out how common that type of images is as a second step

allow people to publicly submit images I guess? 10k would be small, you would wind up getting like, 100k xD

God forbid a troll finds the upload button

Ask our Google Research Zurich friends to send over a copy of all the pictures on Google Photos

a few trillion images should do it

https://youtu.be/tVQsxLfKPNI

honestly images uploaded by trolls probably make up a significant amount of all images in general so it'll be good representation in the dataset <:galaxybrain:821831336372338729>

Soyjack detection algorithms

I guess that would have a disproportional amount of photos taken on Android phones, and they're usually already JPEG-compressed before getting uploaded (and I guess often further transcoded by Google Photos)..

Yeah I think Google Photos turns them into WebP lol

Unless you have the subscription to store them uncompressed (is it still a subscription? I haven't used google photos in years)

To be clear it wasn't a serious suggestion to use random people's Google Photos data, BUT if someone generously donated their whole uncompressed Google Photos library and tried several encoding options including lossless JPEG transcoding, the results might be appealing to the Photos team, possibly making them take an interest in JXL? 👀

It's a subscription for Google's cloud service to store them uncompressed

I have been thinking about extracting a corpus from all images on Cloudinary (if I can get something like that cleared with Legal, since the images are owned by our customers, not by us, so I have to be very careful), which would also of course only be representative of Cloudinary's customers/users, but still, that might be a diverse enough set of use cases. Question then is: how to do it. Extracting randomly gets you a lot of long-tail images that are not at all representative for typical web traffic, but extracting weighted by popularity will bias it the other way towards landing page hero images and logos and stuff like that.

I think this might be subject to selection bias. People will manually seek 'efficient' images for those that see lots of traffic. If there is a connection between traffic volume and image characteristics at all, that correlation is the only one I could see it being

also there are bunch of use cases that still wouldn't be covered, either because our terms & conditions don't allow it or because we're not a good fit for such use cases. E.g. porn, military imagery, medical images etc. And probably e-commerce would be overrepresented.

I'm not a lawyer, but I feel like it might be more legal and less controversial to do all the comparisons in-memory and just save the statistics in the end. So instead of creating a corpus, you're just using (some subset of) all images for every experiment. But this of course prevents others from using it, unless they can request an experiment to be run on your images or something

Yeah having a corpus that can be shared would be nice but that's really hard to do since you need to get a license for every single image. An alternative is a list of URLs, if the images are on the public web, which circumvents the issue but is prone to link rot.

I wonder if you can get a giant dump of images from the Internet Archive

Even if they're not representative of all images in general, knowing how valuable JXL is in use cases like the Internet Archive, or Wikimedia Commons, is valuable in itself, because these are orgs that just have lots of images, regardless of what types of images, and probably would have an interest in storing them efficiently

I have at some point gotten a dump of images from the google indexer, but of course that's of limited use to everyone not at G (and also to many people at G xD)

Storing already-lossy images is something quite different from compressing a pristine image though.

I don't remember what it's called, but there's a site that has collections of raw files from various camera models. It might be possible to automate some software like Darktable that develops them and exports them to a lossless format, and then you have a corpus of real photographs from from various camera and lens combinations. The processing done wouldn't be identical to what the camera itself would do though

So then you effectively have a bunch of photographs from right before the final "encode to jpeg" step

I mean in practice collecting original lossless things just sounds hard, I imagine lots of pngs online are just screenshots of jpegs... you'd need to filter that or it inadvertently becomes a generation loss experiment instead... sounds like a mess

dpreview ?

Nah, some other site just dedicated to collecting raws

https://raw.pixls.us/ ?

That might be it

If RawTherapee can get the PR done, they'll have native JXL export, so you could just directly use the command line *and* use the automatic preview matching to try and be similar to the in-camera processing

Hang on, automatic preview matching actually sounds amazing

I've had a hard time trying to recreate the original of my out of camera JPEGs in Darktable

I am experimenting with the API and have made a puzzling observation: My program reads a lossless uint16 Jxl image (created in Lightroom) into a buffer, decodes it twice (first as uint16 and then as float), and saves both versions. Both display nicely in XnView and are reported by jxlinfo to be: "JPEG XL image, 4745x3163, lossy, 16-bit RGB" "JPEG XL image, 4745x3163, lossy, 32-bit float (8 exponent bits) RGB" What puzzles me is that the two saved images have exactly the same size, and a byte-by-byte comparison shows they are more than 99.999% identical. The remaining five differences are located near the beginning of the file: 0000002E E2 92 0000002F 27 4B 00000030 A1 28 00000031 B8 2E 00000032 10 04 Is it possible for a uint16 image and a float image based on the same source image to have JPEG XL representations that are so similar? I would have expected no resemblance at all due to the different bit formats of the datatypes, and I would have expected the float version to be much larger than the uint16 version. I have taken steps to verify that the code path that I *think* is executing actually is the one that *is* executing, such as adding a tiny float value to each RGB component. This removes all similarity, and leads to a modest increase in size of the float version (456 bytes). I would appreciate it if someone could take a look at the images at https://www.bosnes.net/ and help me figure out what is going on.

Lossy JXL is always stored as float32, the only difference is the metadata specifying what format to decode to

If you tried with lossless, you'd see the differences you expected

Thank you for the swift response! This also explains another puzzling thing I had noticed: why images "stored as" float16 and byte also have approximately the same size

Any time. I'd recommend trying to decode them to pfm files, they should be identical if I'm right. I also tried to find a relevant portion of the spec but this is the closest I found

In case of lossy, the bit depth is just metadata in the header but the actual encoding does not depend on it. It is just a suggestion of what bit depth to decode it to, if you decode to RGB.

Thank you both! Much appreciated

I'm pretty sure float32 can perfectly represent any int16 value too

That's interesting, what's the reason for this? Just simplifying the format/code by always using f32 in lossy?

I know float16 for a 'low precision encoder' has been mentioned a few times, so I think mostly for simplicity and to avoid unnecessary precision loss causing banding during quanization/decoding

Pretty old, but seems relevant still https://www.reddit.com/r/jpegxl/comments/l6rv3u/comment/gl7fc7i/

I really wish Reddit didn't cut off the message with a gradient...

I wonder if/how redlib does previews https://redlib.nl/r/jpegxl/comments/l6rv3u/comment/gl7fc7i/

poorly, it seems

Anyway thanks for the info

There is a compile flag for lower precision in certain decode stages, iirc it uses int16 instead of float32 for the XYB to RGB. But yes, in general most things are float32 so there is just a single code path.

does djxl --allow_partial_files work for anyone? doesn't seem to work for me

When I tried deleting bytes instead of zeroing for leniency testing, it asked for the parameter and worked for me. I can try again when I get home, but IIRC it never actually decoded partial files when trying to test progressiveness, just errored

I seem to always just get "Input file is truncated and there is no preview available yet."

oh, I guess this is a side effect of defaulting to chunked encode

Oh... *Ohhh that makes sense* I don't think I tried the truncated argument since learning --progressive_dc=1 fixes progressiveness

Didn’t someone find out HfGlobal is written at the end when using chunked?

LfGlobal IIRC but yeah

https://discord.com/channels/794206087879852103/805176455658733570/1314895435730649108

https://discord.com/channels/794206087879852103/805176455658733570/1314868178140266546

Turns out in different situations it’s both!

That might've just been me getting confused

We should really make some jxltran tool that can reshuffle the sections into an order more suitable for progressive decode

It's not super critical since for image sizes under 2048x2048 it will be progressive anyway, and that still covers most web images.

But still, it would be nice to have. If you make it as a tool that requires an input file in which it can freely seek (as opposed to the streaming io model libjxl uses), it should be pretty easy to implement all kinds of bitstream remixing stuff.

I've said it before, but cjxl already buffers the output unless `--streaming_output` is specified, so the HfGlobal could theoretically already be moved without issue. At least for manual encodes

I might slowly go though the github issues and comment on any that have since been fixed. There's quite a few high memory usage ones from 2021/22 that never got closed after v0.10 Don't want to be spammy though, so thought I'd ask here first

Sure, any help with reducing the amount of open issues would be appreciated!

The preview matching in this is so good thanks for mentioning it

I can finally stop messing around in Darktable to get my raws to look like the out of camera jpegs

My DNGs don't actually have a preview, but the matching still gives the same image that my phone does when viewing them so I'm happy enough (The JPEGs have far too much smoothing/AI upscaling)

Looking into this a little while I'm cleaning up old issues https://github.com/libjxl/libjxl/issues/1470

Do jpegli and libjxl use the same upsampling for the chroma channels?

I'm noticing a slight hue shift towards red, as if the chroma is bleeding or was sharpened slightly

I forgor about the CFL, but it's still.. Strange. Jpegli almost seems to do some kind of local correction. The mosquito noise is lessened and the region matches the orignal better, but high contrast details are desaturated too

May god help your notifications xD Up to page 6 of 14 so far

you could also have closed them while you were going through them 😛

don't think they have write perms on the repo

Ah you need write permissions to close?

Sigh

is JxlEncoderOutputProcessor assuming the buffers are write-only? or can you assume that if it supports seeking, then reading would also be ok?

it might be nice to have a way to avoid the non/less-progressiveness of streaming encode, at least as an option. That requires having HfGlobal before HfGroups, and to make room for that you'd need to do seek/read/write, but that is probably usually not a big problem/overhead.

Actually, what is in HfGlobal that is only available after writing HfGroups but is not needed to write HfGroups?

Yeah, letting anyone close other people's issues wouldn't be a *great* idea usually. But someone might have to sort by "Last Updated" when I'm done and close the ones I marked (some are asking for additional info or answering questions)

Uh, I don't think it assumes much

I think it currently is doing write-only

Yes, I think so too

It might not be related, but could chunked JPEG Transcoding be feasable too? Or would that need a new API for the bitstream input

If it's a sequential jpeg then I guess in principle it could be possible to do it in a chunked way. With a progressive one though... I don't think so.

Do you put HFGlobal after the PassGroups using a TOC permutation?

Yes

Toc permutations is how I did it in hydrium

but hfglobal is fixed there

so I just put it at the front because it's only one

Or rather, wait no that has the histograms

hm

I just double checked, apparently what hydrium does is buffer the HF coefficients, build the histograms, then entropy-encode

since I tend to only output one lf group at a time I can do that without being memory greedy

You write multiple frames, right?

yes, unless `--one-frame` is passed on the CLI

then it only writes one frame. I think I might still buffer all the HF coefficients, but I've found it doesn't use that much more memory

I suppose you could also make histograms when encoding the first LF group and then in the other LF groups just encode the HF with fixed histograms. Then you could write everything in a single frame, and probably get some speedups since less need to buffer/compute histograms, at the cost of some compression but probably not that much.

doing some benchmarking I found that doing 2048x2048 frames was the fastest. doing one frame was almost as fast, and 256x256 frames were notably slower. but also used like 2MiB of memory so it's a different use

Might even compress better since there's less histogram / frame header signaling overhead

using one histogram for each 2048x2048 gave the best ratio

in my benchmarking

because the histograms were most accurate to the tile they were encoding

using one frame with one histogram had lower signalling overhead, but the histograms weren't always as correct for each tile

Makes sense

``` malloc(): unsorted double linked list corrupted Aborted (core dumped) ``` ooo, fun. I don't know what this means

time for a valgrind run

valgrind is so useful, bug found and fixed

at least, that was my observation, and it's the guess that veluca came up with as to why that observation was holding

I just tested on a *massive* PNG

got 160.41 MiB for the one-frame-per-LF-group and 161.20 MiB for one frame total

turns out for the massive massive files it does inflate memory usage quite a bit to encode one frame

which is why I don't really recommend it

'quite a bit' meaning the 0.8 MiB? Or do mean something else

that's file size

4.5 GiB vs ~700 MiB

Ohhh right, yeah that sounds more like my results

I am experimenting with the libjxl API, and have an anomalous result when writing and retrieving extra channels. I add extra channels to an image by using the green channels from images that have the same dimensions as the main image. When I use lossless compression, the size of the main image and the sizes of the added images add up nicely to the size of the main image with the extra channels added: 12 955 437 ExtraChan_0.jxl 13 094 838 ExtraChan_1.jxl 40 001 982 MainImage.jxl 66 043 467 MainImage_2_extra_channels.jxl >jxlinfo MainImage_2_extra_channels.jxl JPEG XL file format container (ISO/IEC 18181-2) JPEG XL image, 2560x1920, (possibly) lossless, 32-bit float (8 exponent bits) RGB+Alpha+Thermal Color space: 940-byte ICC profile, CMM type: "KCMS", color space: "RGB ", rendering intent: 0 When using lossy compression, however, the size of the main image with extra channels added, is more than 10 times the sum of its components. In fact, it matches the size of the lossily compressed main image plus the sizes of the losslessly compressed extra channels: 786 095 ExtraChan_0.jxl 610 665 ExtraChan_1.jxl 877 991 MainImage.jxl 26 985 570 MainImage_2_extra_channels.jxl >jxlinfo MainImage_2_extra_channels.jxl JPEG XL file format container (ISO/IEC 18181-2) JPEG XL image, 2560x1920, lossy, 32-bit float (8 exponent bits) RGB+Alpha+Thermal Color space: 940-byte ICC profile, CMM type: "KCMS", color space: "RGB ", rendering intent: 0 I suspect that the reason for the large size, is that lossless compression has been applied to the extra channels even though the main image was compressed lossily. Is there something I should have done to tell the API that I wanted lossy compression of the extra channels?

<@1328688054348812390> you want `JxlEncoderSetExtraChannelDistance` though this will set the distance for all extra channels, there's no individual control currently <https://github.com/libjxl/libjxl/issues/1707> <https://github.com/libjxl/libjxl/issues/3906>

It's set to lossless by default due to Alpha usually compressing better

Thank you, that makes sense, but unfortunately, it didn't make much difference to the size of the image with extra channels. I made the call to JxlEncoderSetExtraChannelDistance just after the calls to JxlEncoderFrameSettingsCreate and JxlEncoderSetFrameDistance. I used value 3.0 for distance, and called the function once for each index. JxlEncoderFrameSettingsCreate returned success, and I confirmed that the value in frame_settings->values.cparams.ec_distance[index] still was 3.0 at the time JxlEncoderSetExtraChannelBuffer was called. Its late here, but I will try and debug further tomorrow. Any pointers as to how I can proceed, will be much appreciated. Thank you for your help!

I'm trying to use the cjxl/djxl command line tools to do progressive with middle-out. I'd really like to be able to get a file that just getting the first, say, 24k lets me do something meaningful and then scale quality as i get up to a meg or whatever. Application is for a student-built satellite with a 4800 baud modem that expects to download ~72kbytes on a typical pass; would be cool to get meaningful previews of 3 images in a way that "counts" towards downloading the entire file

If attempting that use JXL-oxide as libjxl doesn’t support full progressive decoding

Even worse for converting Greyscale now

Fine without Alpha but the file size still increases much more like before

You want to use `cjxl --progressive_dc 1 --group_order 1` to make sure it's progressive and loads centre first. Depending on the hardware and image resolution, quality and encoding speed could be adjusted too. As Jia said, you should probably use jxl-oxide to decode the partial files. That allows decoding down to 0.1% loaded in some cases, though there is a PR for libjxl that allows decoding corrupt files if the transmission isn't stable

I've thought about doing this for satellite imagery for quite a while, so by all means share the results and give us updates if you can. Just give me a ping or ask here again if you have any questions or encounter any issues

What kind of satellite imageries do you use?

Hey, thank you, this is is very helpful. I've been using those command line options but didn't realize djxl had trouble on partial decode. I will try jxl-oxide.

It's just going to be pictures from a raspberry pi camera. About a third of the frame is one side of the satellite, and if we have attitude control, the other 2/3rds will be the Earth

I was thinking of you being able to generate imageries like this

Do you know the resolution? And how much memory will be available for encoding? The downside to progressive currently is quite a large memory hit due to buffering the entire image, but it's being worked on https://discord.com/channels/794206087879852103/804324493420920833/1330118476731383838

We've got a little baby spacecraft computer that buffers the images for downlink and runs things, but we've got a pi zero 2 w with 512MB of ram and cycles to compress images. 3840x2464 is full res but we'll often be thinking about half in each dim or one quarter in each dim. Downlink is really where we're constrained. So many bytes of image data, so little ability to send it down. Sending a kilobyte costs us 10 joules; running the computer in the imaging subsystem for a second costs 2 joules.

(We also have science and technology development payloads fighting for that downlink time and energy)

OK, very nice: ``` mlyle@sliver avif % cjxl -v -v -q 85 --lossless_jpeg=0 --container=1 --progressive --progressive_dc=2 --group_order=1 bluemarb.jpg trial.jxl JPEG XL encoder v0.11.1 0.11.1 [NEON] Read 3000x3002 image, 6486234 bytes, 477.5 MP/s Encoding [Container | VarDCT, d1.450, effort: 7] Compressed to 901.2 kB including container (0.801 bpp). 3000 x 3002, 11.503 MP/s [11.50, 11.50], , 1 reps, 16 threads. mlyle@sliver avif % ls -al trial.jxl -rw-r--r--@ 1 mlyle staff 901238 Jan 21 08:27 trial.jxl mlyle@sliver avif % dd if=trial.jxl of=test.jxl bs=1024 count=4 && jxl-oxide test.jxl -o test.png && open test.png ```

``` mlyle@sliver avif % dd if=trial.jxl of=test.jxl bs=1024 count=32 && jxl-oxide test.jxl -o test.png && open test.png 32+0 records in 32+0 records out 32768 bytes transferred in 0.000158 secs (207392405 bytes/sec) ``` I notice the group-order only applies to the final image in the progressive sequence

I wonder why some of the lower quality tiles look pixelated while others look blurred

Yah, I thought that interesting. Also, jxl-oxide is not able to decode the full image. Only when it truncates relatively early

`--progressive_dc` 1 gives an image at `9600 bytes 1.2%` and 2 gives an image at `5600 bytes 0.6%` loaded

2 loads sooner, but has higher bpp so you get less in the same amount of data

This was the original for comparison, a game screenshot but 4K so should be close to your images at full resolution (Though memory may require half res)

Progressive DC 2 makes it into Modular 1:64 and VarDCT 1:8 LF frames, so the blur is likely the VarDCT and the pixilation is probably the Modular

```Frame #0 Modular (maybe lossless) Frame type: LF, level 2 (64x downsampled) 60x34 Frame #1 VarDCT (lossy) Frame type: LF, level 1 (8x downsampled) 480x270 Frame #2 (keyframe) VarDCT (lossy) Frame type: Regular 3840x2160; (0, 0)```

Oh, also <@526322998521888768> you can use `jxl-oxide -I --all-frames --with-offset` to get some debug info about the JXL files and when scans of the image should be loaded. Might be useful to calibrate how much data to send per pass

Apologies for all the pings, you gave us something to do haha

Hey, no, I really appreciate all the support and help!

I'm hoping we can be dumb and just send a few K, then a few K more of interesting images, and so on

Is there anything in JXL that can signal how certain layers should be interpolated, or is that completely up to implementations?

Uhh, you mean like upsampling algorythms?

partial files are not considered by the spec as those are invalid bitstreams

libjxl does non-separable upsampling, and jxl-oxide does nearest neighbor

No, more like when the image has finished decoding already, how the viewer is supposed to handle interpolation when the user zooms in. (I asked about layer specific interpolation, but this doesn't make sense when talking about about an image that has finished decoding, so ignore that)

Like usually this is a setting in the viewer but I could see at least some use cases for this to be specified in the format itself

then nope, the spec only defines 1x zoom

Ah okay. Honestly fair, was just curious

I mean, there is this https://github.com/libjxl/libjxl/pull/2571

*Technically* an insane viewer could increase the upsampling scale as you zoom in, but it's a lot more effort than it's probably worth

Huh so JXL supports that, it's an encoder thing

Kinda cool

The benefit of having an expressive format

If you want the upsampling to be baked into the image and done by the jxl decoder, you can use the 2x/4x/8x nonseparable upsampling defined in the spec, either with default filter weights (which are nice imo) or with something custom.

I think upsampling to x8 basically removes the need for the image to signal how it wants to be interpolated when zooming yeah

Issue then is, normal decoders will honor the 8x upscale regardless of zoom level. So you'd need a closed ecosystem for what you want

Visualization of the blue marble image, compressed as ```cjxl -v -v -q 85 --lossless_jpeg=0 --container=1 --progressive --progressive_dc=2 --group_order=1 bluemarb.jpg trial.jxl``` and then having the first n kilobytes peeled off, where n is 1, 9, 17, ...up to 217k (out of 901k total size). Progressive rendering definitely works, but some of the choices the encoder makes are interesting.

https://music.apple.com/au/album/not-alone/1746873815?i=1746873826

Oops wrong server

Hi, Jonathan. Thanks for your awesome contribution in triaging JXL issues. And welcome - you will obtain "triage" role and will be able to close the issues (hopefully).

Thank you, I'll check it out soon

Also, if you're re-encoding JPEGs, there's likely EXIF data too correct? `--brotli_effort 11` will increase the compression time spent on metadata

Cool, thanks for the tip. I had had a command line argument to strip EXIF but I had taken it out when things were failing

I should probably ensure our toolchain puts some meaningful metadata in (and then compress it well)

Seems to have worked, thanks again and hopefully I can lend a hand in future

Does anyone understand this of how it is choosing to fill in the blocks ... how it does the left two thirds of the image top to bottom, then the right two thirds? I'd really prefer it to be middle out. (The last phase is middle out, but this one is weird)

It might be this, I also noticed this behavior for some other image test a while ago https://discord.com/channels/794206087879852103/794206170445119489/1285551387647938580

For your convenience: > It says group_order=1 is center first, is that the one where it spirals outward like the example from the whitepaper? It doesn't look like that's what it's doing, it looks more like a scanline pattern on three columns, doing the center one first, but it's hard to tell > I see what you mean now, and I think I know why. > That happens during the DC/LF part of the image, which are 2048 blocks instead of 256. So it probably is a spiral, but there's only 3 blocks so it just goes middle, left, right (probably)

I mentioned it before, but I think with `jxl-oxide -I --all-frames --with-offset` you can actually see the center-first reordering, and it doesn't seem to have applied to the DC

yah, I have a hard time knowing the group number to spatial relation. I do see how pass_idx=2 they start near the middle. This is already a huge win for us. Seems like the win could be much bigger I could control frame #1 a little bit. 47x47 is great from 64x downsampling on frame #0-- down to 6k or so, able to send a lot of these per satellite contact. But frame #1 at 8x downsampling -- 115k before we get through the first group of the first pass, and the ordering isn't great. I suspect I either want frame #1 to be 4x downsampled -- so that the middle-out does something useful -- or to be 16x downsampled so that it is just smaller.

I wonder if it could be done by editing libjxl to do a 2x or 4x downsample for DC 2 instead of 8x. I can't find anything in the spec mentioning something like `--progressive_dc 2`, only stating that VarDCT requires a 1:8 image for it's LF frame

``` cjxl -v -v -q 85 -m 0 --lossless_jpeg=0 --container=1 --progressive --progressive_dc=1 --group_order=1 -x strip=exif,xmp,jumbf bluemarb.jpg trial.jxl ``` This is with progressive_dc=1 .. e.g. no intermediate frame. Seems to be much more usefully "progressive" in practice (and smaller in size)

same 8k chunks of file

per frame

Yeah, DC 2 essentially embeds a 1:64 preview of the image, so using DC 1 you get more value for your bytes, but takes twice as many for a first image. Depending how far you want to go, you could parse the TOC and send the bare minimum 'first image' per file. It's a shame we don't have a better test image, since the 1/3 satellite could increase the size due to fine details. Though, if you use half-resolution like you mentioned, you could also increase the effort setting for smaller files and higher quality for a memory increase

Yah, but that 1:64 is tiny.. 6k there's something more going on

This is 41k . better than the progressive_dc=2 image gets by 80k

let me pull progressive_dc=2 at 80k

trial == progressive_dc=1, trial2 == progressive_dc=2

Two pngs above corresponding to progressive_dc=1 at 41 kilobytes, and progressive_dc=2 at 80 kilobytes

A slight improvement, modular arguments still apply to VarDCT's modular LF frame. Adding `-g 3 -E 3 -I 100` shaved off 1.5 KiB from progressive_dc=1 for me `LfGlobal: 134708 (0x20e34) bytes` `LfGlobal: 133237 (0x20875) bytes` My exact command was this, the container is automatically enabled/disabled depending on metadata and some arguments have short variants `cjxl -v -v -q 85 -m 0 -j 0 -p --progressive_dc=1 --group_order=1 -x strip=all -E 3 -I 100`

Oh...

https://embed.moe/https://cdn.discordapp.com/attachments/804324493420920833/1331739938294595675/Test.jxl?ex=6792b6f7&is=67916577&hm=826ab605bd614e2eaaac2e753180ffbf92d69a44206ee977609c17e68c22f1c2&

Maybe the modular arguements aren't meant to work...

Turns out it was group size, didn't effect filesize anyway so I edited the command above

Almost seems like a bug, I don't think `--progressive_dc=2` has been tested much

All these LF things are isolated to their groups right? So can't you make the group sizes smaller so that center first actually does something for the first few passes

Instead of the whole image, or a lot of it of it, being in a single group

Making groups bigger did that, I imagine smaller might corrupt it too, but <@526322998521888768> could try adding `-g 0` and see what happens

yah -m 0 does nothing. I have to go teach a competitive math class but i'll be back later

to play with it more

what if you make the group size the height/width of the image divided by three (rounding up to not create groups that just contain an edge of pixels)

That should give you a center group that is decoded first

or should be rather

I will play with encoding settings and middle out. I think I'm most concerned that progressive_dc=2 looks much worse for every size than progressive_dc=1

progressive_dc=2 adds a 6k small preview but then is worse at sizes 40-60k larger than progressive_dc=1

Group size is fixed at 128x128, 256x256 (Default), 512x512 or 1024x1024

Oh huh, I thought they could be anything up to 1024x1024

good to know

One sec, I'll check the spec Well that was fast `naturally aligned rectangle covering up to 2n × 2n input pixels, with n between 7 and 10, inclusive`

You get whatI'm saying? I'd expect 2 to be, say, 4k "behind" 1 in quality if there's a 6k lowres first, not 60k+ "behind" to reach same quality

Yeah, maybe <@794205442175402004> has some idea what's going on

`--progressive_dc=2` is taking twice the data to reach the same progressive qualityhttps://discord.com/channels/794206087879852103/804324493420920833/1331734747914965074

Yay, test being proctored. Re: satellite in frame. I expect it to not use too many bits, because it's going to be lowpass filtered --- out of focus 😄

This didn't get response. Is this a bug?

This is similar to what libheif would cause

I think it's a macOS bug

grayscale jxl doesn't work well in Preview or Finder for some reason

I'll test again when I can access my own computer later today (or tomorrow for some)

Grayscale+Alpha for this

One GIMP3 user complained about ugly blue artifact when he uses distance 0.1 and 8-bit JXL with uses_original_profile = JXL_TRUE in BasicInfo + ICC. I have impression that it was already mentioned somewhere, but not sure. He shared picture in https://gitlab.gnome.org/GNOME/gimp/-/issues/12785#note_2328039

I can replicate, unchecking the "save original profile" fixes the issue

```jxlinfo bubbles-save-original-profile.jxl JPEG XL image, 1920x1200, (possibly) lossless, 8-bit RGB Color space: 672-byte ICC profile, CMM type: "lcms", color space: "RGB ", rendering intent: 0 jxlinfo bubbles-no-icc.jxl JPEG XL image, 1920x1200, lossy, 8-bit RGB Color space: RGB, D65, sRGB primaries, sRGB transfer function, rendering intent: Relative```

GIMP is definitely doing something wrong here

iirc `uses_original_profile` shouldn't be set to true when doing lossy

ICC profile will always be embedded if there's one (but the encoded image will be in XYB), but I'm not very sure

yes, please don't do that 🙂

maybe we should rename that flag, the amount of people that get confused by it is a bit high

I'm not sure there's ever a good reason to have that flag be different from "I am doing lossless"

(well, maybe with modular lossy...)

what does the flag actually do? force decoding using the colour space defined at encode time?

it disables XYB

so it compresses in RGB 🤢

yes

you see the problem

oh geez, ahhhhhhh

You're possibly right. Only Affinity series on my macOS can display Grayscale+Alpha JXL correctly

The JXL one is only 0.2% smaller

I set uses_original_profile when encoding CMYK. Doesn't it mean that lossy CMYK is also not a good idea?

I would not set uses_original_profile 🙂

although if I had to say I know what the encoder does with CMYK I would lie

<@794205442175402004> probably knows better than me

Lossy CMYK is not tested at all, probably does not work.

I mean, YCC like in CMYK jpegs should work (but we have no encoder for that). Using XYB with CMYK is a bit weird since the XYB only represents the CMY components, and I am not sure if the spec is sufficiently clear on how XYB should be converted to CMY - I think we intended to define it to do the XYB to CMY conversion by pretending it is producing sRGB. In any case the XYB will be abused as "just a transform" as opposed to being the absolute color space it is supposed to be, since the K component cannot be factored in.

I don't really see the use case for lossy CMYK tbh. If you want lossy, you can just as well use a sufficiently wide gamut RGB space imo.

<@794205442175402004> Did you see the thing I had above? ```--progressive_dc=2``` resulted in a 64x downsampled modular frame that was like 5-6k at the beginning of the file, but then quality from 10-50k is 50k+ behind ```--progressive_dc=1``` (I'd expect to be paying a tax that was more like a fraction of the 6k spent on that first downsampled frame)

e.g. progressive_dc=1 at 40k truncation looks way better than progressive_dc=2 at 80k truncation

Yeah that makes sense actually

(progressive_dc2 is also like 100k larger)

Oh, I found it surprising. I'd expect for it to be worse / a few K behind, but not that much worse

Progressive DC > 1 is not really useful unless the image is super huge and even then it is not useful for better progression, just for better performance

as it stands now, i'd be better off sending a 5k PNG + a progressive_dc1 file -- better quality at any number of bytes received

well, progressive_dc=2 is better when less than 5k is received

but way worse from 5k on

Progressive_dc=1 does the 1:8 image using modular with squeeze, which already is as progressive as it gets

Progressive_dc does the 1:8 image as VarDCT and only the 1:64 image as Modular with squeeze

Does modular with squeeze benefit in any way from a prior frame layer, or does only VarDCT?

Sorry for noob questions. I don't really know image compression

Application is for constrained downlink from a small student-built satellite.

In any case, thank you very much for your help / listening. Jpeg-XL is a big win for us, but it'd be an much bigger win if we had something with the very-short-truncation performance of progressive_dc=2 but something closer to the longer-truncation performance of progressive_dc=1

You can always replace the PNG with a lossless JXL that should work out to about half the size :)

😉 Handling thumbnails and lots of files adds a whole lot of complexity to what students have to do