JPEG XL

With clang on Linux it's more like 3-4 MB iirc

around how much binary size is taken up per SMID target? also where's the place/file that should be looked in for the full amount of compile time SIMD targets that libjxl can do?

You can control it with HWY_TARGETS iirc

3-4 MB is still too large for some projects, especially at this time where the format is facing a chicken-or-egg problem due to it being barely 1 year since the library got standardized.

At the very least the decoder should have a target to close on the size of libwebp.

If you're on a system where a 3MB library is a problem you should be disabling SIMD targets that aren't relevant for your system

Disabling SIMD entirely might be best for GZDoom, since the textures should be very small anyway and it's made to run on very low end hardware

You can also compile without some features like JPEG reconstruction, which you don't need if you just want to load images

libjxl_dec on Android could be less than 200kb

Yeah, that too. I imagine it might have things like the PNG encoder included by default too

How to do that?

libjxl doesn't but cjxl does

er, djxl

Ok tried with HWY_COMPILE_ONLY_SCALAR, didn't work out.

elaborate

Executable size remains the same.

how are you linking in highway?

you need to compile highway with those options, and highway's static library size should decrease

Damn, too bad j40 jxl decoder isn't being developed anymore. That decoder compiled code size about 240KB and probably the jxl features supported by it would be sufficient for use with textures.

perhaps jxl-oxide would give good size too ?

but is there C interface for jxl-oxide?

I don't know 😄 <@206628065147748352> ?

for C bindings: there isn't any yet

and for binary size: cargo-bloat reports 463.5 KiB for jxl-oxide related crates

the size is for code section only

yeah, for game engines current features may be enough, except that decoding speed is not so good (due to lack of optimization)

I was tempted to mention, even the WASM is only 800KB, so it should definitely be possible to lower the size

There's always a bit of a trade-off between binary size and speed, e.g. things like loop unrolling and inlining can make things a faster but also increase binary size. If binary size is really an issue, you could also consider compressing the binary — it should be possible to compress it a bit

my libjxl.so is 3296 kb but default gzip reduces that to 1365kb

libjxl_dec is 1460kb and default gzip reduces it to 580kb

isn't binary compression sort of a platform/OS specific thing? (I haven't looked into it that much really), I mean I guess there isn't really anything stopping it from being done on most platforms but it's more of an issue of how complex of a thing it is to say integrate into a build system or have at the same functional level between all platforms for a project that aims for many.

let me try building for just SSE4 and no-simd, instead of the default which also has SSE2 and AVX2, to see what the binary size is then

I mean on some platforms you get binary compression for free because the way software is shipped is in e.g. Apple's .dmg or Android's .apk files it does already use compression, at least optionally

I think reasonable would be SSE2-only, since every x86-64 supports SSE2

just like javascript/wasm or svg in practice gets compressed using deflate or brotli because on the web you can use transfer encodings

ah right, SSE2-only does make sense, no point having scalar code on x64

with SSE2-only, libjxl is 2130 kb (914kb with gzip), and libjxl_dec is 792kb (339kb with gzip)

Is this with MSVC and static linking?

no this is just my libjxl.so compiled with clang

after stripping debug symbols, it is 1911 kb for libjxl and 694kb for libjxl_dec (316kb when using gzip)

Not a good sign that it somehow ends up with more code in static executables on MSVC then.

could be some defined or undefined compiler or linker flags bloating the size, I don't really know the full list of them for MSVC but adding something like `/Gw` could reduce the size by a minor or major amount https://learn.microsoft.com/en-us/cpp/build/reference/gw-optimize-global-data

Reducing binary size hasn't been a focus, only for Android Chrome there was at some point some effort spent to keep it low. Probably you can make a jxl decoder in under 100 kb on x64, if you are careful.

the main reason size is such a concern in this situation is because GZDoom wants to be really careful not to introduce third party libraries that are seldom used and bloat up the size and image format wise they are planning to introduce GPU friendly formats in the future so for them they can't see a reason to add JXL unless it incurs a very small increase to the total size of the binaries or unless it's very critical to something content/asset wise which currently it isn't

adding avif then makes even more pointless when its highest advantages are in low fidelity, but very bad lossless compression and for high fidelity it can be worse than even good jpeg encoders and I don't think low fidelity for lossy is very much in demand for games, unlike web especially for GZDoom, used mostly in games with retro stylized pixel textures and images

currently they have dropped the idea of adding JXL or AVIF and have put it as something to possibly be re-looked into in the future while they shift their focus onto other things

FYI I'm trying to revive it recently, though I need to answer some questions first (see <#1020056709831594005>)

Not added recently but missing is Zoner (strong rival for lightroom and acdsee). Silently added since mid 2022, not mentioned anywhere but in list of supported formats.

Seems its running old or unoptimized jxl code so itd be good to check if fully spec compliant or any features incomplete

Does JXL support pure single threading

this is what i'm getting with bloaty on a default libjxl build

With highway library statically linked along with Brotli it would shoot up to 5 MB or more.

The actual static library files of libjxl on MSVC is over 80MB+ in size however.

Even if the final executable is order of magnitudes smaller.

I do know that unused code removal would do the job of reducing to 5.4MB.

don't you have some way to clean static libraries from the unused symbols ? (like what is done when you link an exe ?)

My Qt plugin on Windows is built in MSVC and uses also static linking. With libjxl, hwy, brotli bundled-in, the final size of the plugin is 5.63 MB https://github.com/novomesk/qt-jpegxl-image-plugin/releases/tag/v0.4.3 This is how I built it: https://github.com/novomesk/qt-jpegxl-image-plugin/blob/main/appveyor.yml

Probably doesn't know you can mark it as a draft

I thought they were testing the PR template or something

https://www.bestpractices.dev/en/projects/7845

Answered a lot of questions

https://www.bestpractices.dev/projects/7845/badge

Maybe will see how far we are from gold

Not today tho

Has the 0.9.0 idea been dropped? Now with iOS 17 and MacOS 14 any further optimizations and bug fixes would be great to get out quick. Or 1.0 might be closer than expected

Originally 0.9 was going to be skipped, and major bugs fixed for a 1.0 release, but if I recall the scope grew too large so now it's back to a 0.9 first

it seems to be what ArgyllCMS went with: (3.0 changelog) > Added manifest to MSWindows executables to use UTF-8 code pages on Windows 1903 and later. This should improve non-ASCII filename and path handling.

hi, why decoding jxl images (especially lossless) takes so much time?

How big of a file are you using and what is your system?

files around Full HD 1-2 MiB

Ubuntu 22.04

CPU 10920X

Huh, those are pretty much instant for me on much worse hardware

I'll try to rebuild latest versino

Yeah less than half a second on libjxl from the arch repos

On my school laptop

Watch for the buffer overflows! Years after being released, a 0-day breach was found in libwebp https://arstechnica.com/security/2023/09/incomplete-disclosures-by-apple-and-google-create-huge-blindspot-for-0-day-hunters/

https://github.com/libjxl/libjxl/issues/1097

crap

a fix for that will be available in the near future

it's been blocked on some *amazingly annoying* CMS related work

Is there a way to use the fuzzer or something else to create a corpus of small files that are corrupt in all possible ways? It could be a useful complement to the conformance testing to also have some set of bad inputs so you can check that a decoder properly and quickly rejects them (as opposed to crashing or wasting lots of resources)...

uh, maybe write a wrapper that crashes on decode error?

btw there's a set of files that made jxl-oxide crash https://discord.com/channels/794206087879852103/1065165415598272582/1109524145374502932

(we could then also add that to the libjxl tests and make some good improvement in code coverage stats, since many missed lines are error conditions that are just never reached with valid inputs)

is it on libjxl side or gdk?

libjxl side

The work, I mean

oh I mean, where is the problem lying

ah, the core of the issue is that `lcms` is amazingly slow

and the gdk plugin uses lcms to handle color conversions, if they are ever needed (but also if they are not)

and in particular, most often they are not, but lcms still slows stuff down

the cms work in libjxl allows skipping the conversions if not needed, which fixes the issue

(presumably)

what do you recommend instead of lcms ? if this one is slow ?

the fast version of lcms exists but is not free 😄

skcms does a decent job at it

assuming you don't need the things it doesn't support

but it also has the problem of not being shipped as a dynamic library, which most OS maintainers don't like

couldn't skcms be included in libjxl 🙂

tell that to debian maintainers 😛

they wouldn't see it if it's embedded at source level 🙂

the lcms2 fast float plugin makes things somewhat better: https://littlecms.com/plugin/

but it doesn't seems to be free, nor open

well it's GPL3

so it's free and open, just not permissive

really ? ok... I have missread

you can get a more permissive (i.e. non-copyleft) license if you want to use it in proprietary software, but for something like gdk-pixbuf which is GPL anyway, I don't think there's a problem using the fast float plugin

nice, gdk should do that 🙂

is that an internal libjxl cms that doesn't require third party libs or just a workaround for the slow stuff?

In most cases, the internal libjxl cms can directly produce output in the desired colorspace.

At least any display space in current use (sRGB, P3, Rec2020 PQ/HLG, AdobeRGB 1998, etc) can be output by libjxl if the JXL image is using XYB...

it does require a third-party library (either skcms or lcms2, chosen at build time), but in many cases, it is able to skip having that third-party cms do expensive transforms

in the case of the gdk pixbuf plugin, non-sRGB JXLs will probably still go through that cms’ gamut mapping to convert to sRGB

Hey, I was wondering if the signs for calculating "linear Y" in <https://github.com/libjxl/libjxl/blob/8c36f40991c0966f6cf03abe045f494267ceab32/lib/jxl/enc_photon_noise.cc#L65> are correct? If I'm reading everything right it results in this conversion from linear RGB to linear Y ``` M = // matrix B = // bias mixed = M x Linrgb + B Y = (∛mixed[0] + ∛mixed[1])/2 - ∛B Liny = (Y - ∛B)³ + B = ((∛mixed[0] + ∛mixed[1])/2 - ∛B - ∛B)³ + B ```

Why would anyone bother wasting their breath on them?

No one can save idiots from themselves. If a distro wants to cripple themselves then Debian will continue to do so forever and no one has the power to stop them nor should they care.

Well, that implies no libjxl package in most debian-derived distributions, which is a lot of them

I think it's not a bad distro policy to use dynamic libraries as much as possible. Maybe skcms should be shipped as a library...

Yes, skcms should be packaged as other libraries. We need that skcms developers make some release or at least Tag.

https://skia.googlesource.com/skcms/ doesn't supporting opening issues, unfortunately

ah, I found the tracker

https://issues.skia.org/issues/302602705

I think the distro should respect the intent of the upstream devs, not the other way around...

If Debian is braindead that is a problem with Debian that no one else can solve but Debian

Lots of libraries are intended to be copied into other programs rather than distributed as a standalone library.

That is just reality

Debian can choose to fight with reality but they should not force everyone else to

Just because a lot of people use a braindead distro doesn't mean everyone should be forced to capitulate to stupidity

Just tell those people "sorry your distro doesn't work, use a different distro or here's a direct download... oh wait, how do I make a Debian package? Why is it so overcomplicated to distribute a .deb? Wow this sucks"

It's nice to use dynamic libraries though, to save on disk space and for faster upgrades (including security updates). Imagine if every application would ship their own outdated version of libjxl...

Yeah, but in practice, library upgrades typically require recompile regardless... Like with SSL libraries especially...

Dynamic linked binaries are actually just a terrible method to provide shared services like that. A more logical system would be something like plan9

Dynamic linking is too fragile and poorly thought out. It's a quick and dirty hack

Every distro has their own rules which are based on years of experience: https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies

Security updates are almost always API and ABI preserving patches. Certainly this is the case with SSL libraries.

Or notably, libwebp.

??? dynamic linking prevents you from having to recompile libraries and reverse dependencies to update them

it also saves memory and disk space

it's great

it's not at all a "hack"

That's why ubuntu paddle backwards, trying to impose snaps as the new way to distribute apps... Yuck

I have very mixed feelings about dynamic linking 😛

Pretty sure openssl for example says to always recompile when there is an update, for security reasons.

Technically that is no different for static libraries either, they just need to be re-linked not recompiled unless the headers change...

But either way, linking at runtime is still a pretty fragile hack for shared code and there are much better ways to accomplish the same goals, like the way plan9 does things

Plan9 does not even support dynamic linking at all

So IPC everything?

That's a lot more logical way of doing the same thing, yeah

how do i know what failed here? i copied the [sample](https://github.com/libjxl/libjxl/blob/e4eb1f093e27d91fc8c4e3611d2ae2a5c12fa07b/examples/encode_oneshot.cc#L198-L215) and it no worky

i see `error` in `JxlEncoderStruct` but how do i get that

im not good with pointers

JxlEncoderGetError — but it helps more to build your libjxl with debug messages (e.g. ./ci.sh opt instead of ./ci.sh release) so you get something more informative than just an error code

Hi all, I’ve been noticing that the encoding efficiency (in bpp) gets worse the smaller the image is. Is that normal behaviour ? How can it be explained ?

The smaller the image the less redundant data there is

Yeah that’s what I figured but I’ve been comparing with another format (homemade and a bit intricate but uses jpeg) which is not impacted as badly

jpeg is closer to 'fixed cost per 8x8 block' than jxl — of course both will compress smooth/solid regions better than high entropy regions, but jxl can take more advantage of large 'easy' regions than jpeg can

Ok yes i understand, thanks. I have another question about those large easy region. I have had issues with banding in smooth gradients. I read that you apply a selective smoothing algorithm on the DC image, is that at any effort ?

yes, the adaptive dc smoothing is always enabled I think

Ok interesting. Any parameter I should enable to limit those artefacts ?

jxl should be quite good by default at avoiding banding... but the decoder doesn't do any dithering atm so you should decode to 16-bit png and do your own dithering...

For sure, thank you for your answers 🙂

Sounds like a good example of economy-of-scale.

Literally as image size scales up

Also smaller images tend to have a lot more entropy per pixel

Large images tend to have a lot more blurry and smooth area

Sounds like a libjxl decoder bug, when decoding to an 8 bit buffer the output should always be dithered if it's lossy

And if that isn't the default behavior that ought to be considered faulty.

Hence the noticeable banding issues that make the perceived quality of jpegxl suffer

Harms the reputation of the format

iirc there is an additive noise you can apply to an image as an encoding option

There is a film grain option in the encoder

ISO photon noise or something like that

You can specify how much to apply

I agree, at least by default. But dithering should be applied after color conversion, so this was stuck on getting the new decoder cms api implemented...

There was an interesting paper I read the other day about a universal colored noise generation algorithm. I wonder if such a trick might be useful for generating dithering over a wide array of images

https://dl.acm.org/doi/10.1145/2185520.2185572

Just 4x4 ordered dither is fine for avoiding banding in 8-bit sRGB...

Dither can get pretty fancy and clever and awesome

And impossible to distinguish from "natural" or usual noise

And yes, technically it ought to take color space into account but that should be easy since lossy JXL is nearly always stored in the same color space in order for the lossy compression to be effective, right? So why would this be a showstopper?

Lossy JXL is nearly always stored in the same perceptual color space, isn't it? Oh, but it is often decoded to an arbitrary color space, where the dithering needs to take place...

So that's probably why it can't be done yet...

Maybe the default needs to decode to a 16 bit buffer/file instead of 8 bit

Like djxl should produce 16 bit files

Until dithering is ready

I love it when computer drawn art with solid coloring gets naively dithered 🙂

as far as I know, to dither digital signals, triangular noise is the preferred option

you can get that by summing two uniforms together

dithering with uniform noise decorrelates the first moment (the mean) of the quantisation error from the signal, but not the second moment (the variance, if the mean is zero), so you get noise modulation (the signal modulates the power of the noise)

triangular noise decorrelates the variance as well, though not necessarily higher moments AFAIU

but the higher moments are considered less important; decorrelating the first two is the point considered “good enough” for audio

as you sum more and more uniforms, the limit is Gaussian noise, which decorrelates all moments, but also spans a wider range of values, so you get less headroom (with an actual gaussian, none, since it has infinite support) and more noise

https://www.researchgate.net/publication/3317523_A_theory_of_nonsubtractive_dither https://www.recordingblogs.com/wiki/dithering#:~:text=Common%20dithers%20in%20audio%20processing,moments%20of%20the%20error%20uncorrelated%2E https://www.airwindows.com/tpdf-dither/#:~:text=TPDF%20is%20the,TPDF%20dither%20does%2E

Thank you for the readings!

Are there any high quality comparisons of photos with noise generated natively by camera, noise removed via jpeg xl, and noise synthesized by jpeg xl? I'm not really a believer of psycho visual tricks but i'd love to be converted

just as a minor clarification, most of the noise in digital photos is not generated by the camera itself, but is rather a consequence of the discrete and seemingly stochastic nature of photon arrival

fewer photons => less certainty as to the underlying “flux” (/ more variation from pixel to pixel, i.e. noise)

https://en.wikipedia.org/wiki/File:Photon-noise.jpg

this is the behaviour that the `--photon_noise` option in `cjxl` attempts to simulate

photon arrival is considered a Poisson process, which implies that the standard deviation of the number of photons arriving in a given time interval is the square root of the underlying rate

(e.g. if you photograph a grey card that is of such brightness that over a 1/100s exposure, each pixel should collect 10k electrons on average, then you’ll see a standard deviation of 100 electrons across them, i.e. an SNR of 100, or 40 dB)

there is a technique called “Photon Transfer” (https://doi.org/10.1117/3.725073) where you actually make use of that fact to reverse-engineer how many electrons each digital step in the raw files corresponds to

Bill Claff has published the result of doing that for many cameras, based on crowdsourced data (I contributed the raw data for the Canon G1 X III and Canon EOS R6 based on his instructions) https://www.photonstophotos.net/Charts/PTC.htm

so, for the iPhone 14 Pro Max https://www.photonstophotos.net/Charts/PTC.htm#Apple%20iPhone%2014%20Pro%20Max, for example, K_ADC = 4.828 e⁻/DN means that each digital step corresponds to 4.828 electrons

and you can see that photon shot noise (the second grey curve) is the dominant source of noise over much of the range

read noise (the horizontal grey line, and 1.624 DN for the iPhone 14 Pro Max) is more important at the lowest signal levels, while photo response non-uniformity (0.68% for the iPhone 14 Pro Max) takes over at very high levels

those latter two _are_ introduced by the camera

I love the photon noise -- great aesthetics -- I think we got this nearly ideally right -- if we'd do it again, perhaps it would have been nice to add an 8x8 control field to be able to modulate it locally

we don't do real photon noise but I decided that it is better to do the noise in Laplacian

that way there is a stricter impact for the noise -- it cannot accidentally create a lot of low frequency energy

and every image has an amazing accident if we let it -- dices are being thrown millions of times for every image 😄

Many noise generation systems get confused about the quantity of the noise -- Shot noise increases with intensity, but after the gamma compression curve is applied, there is more noise in the dark parts than light parts. I observe five approaches to the modeling: 1. constant noise everywhere, possibly by adding a noise image into the gamma compressed signal 2. use more noise in bright areas with some heuristic such as sqrt(I) because it is shot noise and that's the definition 3. use heuristically more noise in dark areas than the brightest areas because that is what one can observe in some real-life samples 4. psysical+psychovisual: calculate shot noise in linear space, then gamma compress 5. JPEG XL approach: calculate shot noise in linear space, high pass it (Laplacian) so that it doesn't create new low frequency features, then gamma compress

often image compression is tiled -- if one does high pass filtering or otherwise grains that relate to more than one pixel -- one needs to communicate the noise signal over the tiles in JPEG XL this is easy since it only depends on the seed of the randon number generator which is known by all the tile workers, and can be computed without waiting for neighbouring tiles in some other systems, tiles need to wait for top/left tiles to be ready so that they can absorb their film grain state machines

Just noticed speed performance is pretty bad on windows. For instance, it doesn’t seem to take advantage of every threads (same performance between 32 cores and limiting to 8 cores). Is this expected ?

Multithreading depends on the image size and how many groups the image is split into (If I recall) The bulk of processing also isn't multithreaded, so a single core will be used after the first step of encoding (From my experience, but I only have 8 cores)

Rather depends on effort. Lossless? e3 and below avoids MA tree learning. Lossy? e6 and below, with e6 probably saturating best.

There's less of a problem with noise modulation in images compared with sound

In fact, noise modulation can be a very good thing if you can mask the noise and make it less perceptible.

Unfortunately I don't think anyone has actually tried to apply such good ideas in practice

As far as dither goes anyways

Is there an official way to download a jpegli binary, or will I need to built libjxl myself?

Should find it inside these https://artifacts.lucaversari.it/libjxl/libjxl/latest/

I know the github actions builds also include it, but that's easier as a permanent link

Thank you

Hi guys, I'm trying to encode some RGBA data to jxl by using libjxl in Rust. For RGB data, everything works fine, but with RGBA data, nothing comes out and it will cause the 0x81 issue. Don't know what's wrong with my code: https://gist.github.com/qdwang/12304a925f1b0d3c1231a3b2dd0d63a6 I just use a 4x4 pure red rect sample in the code.

It seems I need to set `num_extra_channels: 1` to BasicInfo, but I don't want to keep the alpha channel information. My goal is to let it ignore the alpha channel. The `num_extra_channels: 1` will increase size in the encoded result

libjxl doesn't accept 4-byte RGB

you need to convert it to rgb24, not rgb0

alright, I get it. Thank you

By the way, it there a method to limit the memory consumption during the `JxlEncoderProcessOutput` process?

My target device has a low memory embedded and I can accept longer processing time.

Oh it seems I need to use `JxlEncoderAddChunkedFrame` to add frame

libjxl is a quick and dirty c++ codebase forked from Google PIK code and it is not very wise about memory consumption at this stage of development.

It might be wise, if you need a small, fast, easy to hack on, and memory conscious encoder, to implement a compliant encoder from scratch in a programming language that's easier to hack on, like C...

Because libjxl will only improve after you start writing and submitting patches, and who wants to get hacking on a C++ project honestly?

I dunno if that's a mean thing to say, but that's basically how I feel, but maybe it's just me...

At least with a C++ project it's possible to mix C and C++ in the same project and slowly convert all the source files away from C++ if you want them to be more readable...

I doubt anyone will argue over which language makes it easier to read someone else's code.

C++ is harder to parse, for humans as well as computers...

I am trying to understand how `JxlEncoderAddChunkedFrame` actually works to save memory usage. It calls `JxlEncoderAddImageFrame` in the soucecode. So my guess is this: Use a small tile size to be set to the BasicInfo's `xsize ysize`, and add tiles by using `JxlEncoderAddImageFrame` and `JxlEncoderProcessOutput` repeatedly. Am I right? But the problem is, how can the output compressed tiles combine together? In my test, they will overlap one by one because of the `xsize ysize` set before. And there is no `x` and `y` position to tell where they should be settled.

the answer is that it doesn't actually save memory yet

but it's not something you can achieve via API usage

I was going to implement that but vacation took priority 😛

what you do is just add a bunch of swap

Thanks for your reply. I was trying to encode a 48M photo, it will take 2 - 4GB memory if the entire image was loaded

yeah, it true

<@902036286074925096> couldn't even decode 4MP jxl without having to use swap

Instead of a callback-based approach (`JxlChunkedFrameInputSource`), has a 'push'-based tile/scanline streaming encode API also been considered? For libvips, a function similar to libtiff's `TIFFWriteTile` and/or `TIFFWriteScanline` functions would be easier to integrate (/cc <@310374889540550660>).

Cropped decode or resized decode would be nice to have in the api

I thought that was already implemented? Or maybe I'm thinking of Oxide

Maybe some kind of scale factor option. Where it decodes a blurrier version of the images without the high frequency data

With the option of decoding to abuffer with the same size as the original image or to a smaller buffer that has been scaled down by the scale factor

we had that in djxl at some point, but I guess that was before we switched it to libjxl

yes, libvips savers run in a background thread ... they are given a tile of pixels to encode and write to the output so a libtiff / libjpeg / libwebp / etc. style push API for encoding would be a lot simpler for us

we'd need to use some kind of complicated threaded buffer to work with `JxlChunkedFrameInputSource` (I think)

Well maybe the old code can be added to libjxl public API somehow...

libvips is awesome btw and a great way for jpegxl to get adopted by software that uses libvips already

thanks! we love jxl, and include libjxl support in the standard builds we distribute \o/ 8.16 ought to have improved jxl support, hopefully

libvips is also, frankly, a great way for libjxl to figure out how to improve the public API it exposes, to better provide what libvips needs and expects. Same with other consumers of libjxl, like krita and ffmpeg.

It is good if the developers that use libjxl can share critique and outline what they need and expect from libjxl

(speaking of ffmpeg, looks like it still can't decode f32 jxls without converting them to u16, rip) (except for greyscale images)

This is the primary purpose of libhydrium, fwiw

tho development has stalled a bit because of work

just checked the hydrium repo, looks good

Hi guys, I’m wondering what does setting JXL_ENC_FRAME_SETTING_EFFORT actually do in the code? It’s not mentioned in the documentation. It just says the encoder’s speed is affected.

it's the same thing as the effort setting on the CLI

--effort or -e

https://github.com/libjxl/libjxl/blob/main/doc/encode_effort.md

<@853026420792360980> <@794205442175402004> Thanks for the replies.

about compression effort, is e7 always the default unless specified otherwise?

like, the default isnt different depending on resolution. ie default being e9 for all images under 300x300 resolution and e5 default for anything larger than 2000x2000

e7 is the libjxl default, but for an application it could make sense to set different defaults depending on resolution (and maybe also depending on quality)

will effort affect the bit depth of the result?

I think effort doesn't change the bit depth of encodes

Thanks.

In my test on trying a 10bit 48m pixels Apple ProRAW image with complex color scene. Both with 0.46 distance. For effort = 1, it will consume 800mb to 2.5gb memory, results in a 10.4mb jxl file For effort = 7, it will always consume 4gb memory at least, results in a 9.1mb jxl file

Don't know why for effort=1, the memory consumption is so unstable

btw, i was running with parallel threads

For effort = 9, it will consume about 12gb memory, results in a 8.8mb jxl file

Encoding without parallel threads for effort = 1 will constantly consume 2gb memory

ok, but the size ! does it compress well ?

I didn't check the image quality by my eyes. I think for a distance=0.46, the quality=96, so it should be ok

For lossy jxl (vardct mode), bit depth is just metadata, it doesn't change the internal representation.

Thanks for the explain.

<@987371399867945100> https://github.com/libjxl/libjxl-tiny/blob/8eae18172059d54f5c734ca814f23b96eebff859/encoder/enc_frame.cc#L389 shouldn't that just be `left = 0;`?

Looking at the code: libjxl-tiny is using a fixed MA tree which for AcStrategy and Quantfield uses `West` as context and for Quantfield also as predictor. Correct?

AcStrategy and Quantfield are auxiliary images that are flattened to 1D because due to variable block sizes the number of values per row is variable. That causes `West` for the first value of a row to correspond to the rightmost value of the previous row, which is a bit annoying for row-based approaches...

What happens if you just don't use any context and no prediction for those fields? Would it make much of a difference?

The modular image containing the control fields has the ac strategy in the first row and the quant field in the second, and the LEFT predictor uses the value one row up at the first column, therefore we have to start that with the first ac strategy value for the quant field predictor.

Not sure what the impact would be size-wise, but unfortunately it would be a bit tricky to change it in the current state. I started from the very flexible full jxl code and tweaked it a bit and then I changed to the fixed MA tree by using the previous version of the code to dump all the hard-coded tables for the fixed tree. Doing any change now would require changing a lot of these things and then debugging why it does not work.

ah right! forgot that the fields are combined in rows — we did that so quant field values could use corresponding (or neighboring) ac strategy values as context

yeah I see that the tree is just a dump in the code, nothing that can be changed easily

I'd try using the Zero predictor for both control fields, and either no context or maybe North context for the quant field (`if y > 0` in the MA tree). But yes editing this tree will not be convenient.

Main thing to change is change the relevant `West` predictor to `Zero` (`Set` in jxl_from_tree terms), the rest could be done without changing the tree but by just making the cmap map everything to the same ctx so the context doesn't actually depend on `left` anymore.

maybe for debugging it would in any case be useful to have something that can decode the MA trees in a jxl file and print them out in jxl_from_tree syntax — and then we could just use that to print the current tree written by libjxl-tiny, edit it, and use some hacked version of jxl_from_tree to dump it again. WDYT?

yes, this sounds good, something like jxlinfo, but it would probably have to use the internal APIs

<@794205442175402004> is there a collection of jxl_from_tree input files somewhere?

Check <#824000991891554375> 🙂

what is the range of values for ac strategy and quant field that actually get used by libjxl-tiny?

if the set of values that are used is limited and not a continuous range of integers, perhaps it would be useful to use a modular Palette transform (also in the case of libjxl btw)

hello

will there be a new release soon ?

there should be one, yeah

do common png libraries have an interface for streaming encoding/decoding?

Is there any known issue with 0.8.2, with converting a (greyscale) PNG with an ICC profile into a lossy JXL, and then decoding back to PNG again, and getting a much darker or lighter image than the original PNG?

Because I see no reason why it should look substantially darker or lighter after simply doing `cjxl x.png y.jxl; djxl y.jxl z.png`

Is this a known issue?

Also why does jxlinfo say "(possibly) lossless" for losslessly transcoded JPEG with reconstruction data, instead of something more fitting like, I dunno, just saying "lossy"

Since it's lossy DCT encoding from a lossy JPEG...

This only happens with lossy encoding. With -d 0 I can cjxl and djxl all I want and get the same looking PNG.

Which doesn't actually make sense

Maybe there is a bug in the software I'm using to display or decode the color information.

yeah dot gain profiles and grayscale gamma profiles are often not used correctly

and just because it's jpeg doesnt mean it's lossy

it's just an image

It's using an inherently lossy algorithm so it would make the most sense to just say "lossy"

then the encoding is lossy

but say you get a jpeg out of a camera

there's nothing better than that

Since that's probably what the intended purpose of that information in jxlinfo is for

nothing lossless

so the least loss is the jpeg

i thought we all went over this

I'm just trying to guess at what the intended purpose is of the behavior and information displayed by jxlinfo.

i think it's just to say what the jxl is

I think that lossy/lossless thing is supposed to say whether it's encoded using a lossless algorithm or not.

oh

then having the word "possibly" is bad i guess

But I'm not sure if the intended behavior is actually documented somewhere.

So I wouldn't know if that's actually what it's intended for

well who is jxlinfo even supposed to be for

Well, lemme look at the source code...

lossy = VarDCT lossless = Modular "possibly" is added at the end of lossless so people don't think the contents of the image is lossless

It looks like what it's really doing is, checking to see if it's encoded using the file's original color profile or not.

PNG is inherently lossy because it maxes out at 16 bits, and what if you have 32 bit data? Now making a PNG is a lossy operation...

hell, converting a normal 8-bit JPEG to PNG is a lossy operation, you lose some of the information that existed in the JPEG

Looks like it has nothing to do with modular vs vardct

Can anyone help clear up my confusion about how ICC profiles work with jxl? I'm working to add support to Siril. My implementation is heavily based on the reference implementation, with some adaptations to suit interfacing with the internal image data structures we use. When saving an image I can provide an ICC profile, and when loading one I can retrieve the ICC profile from the jxl. However the two don't seem to be the same - working with an image in BT.2020 I save it as jxl and provide the BT.2020 ICC profile, but when I open it the ICC profile description is RGB_D65_SRG_Rel_Lin and the primaries have been changed to the sRGB primaries. The colors have been converted correctly so it looks ok when displayed but I'm not editing in my chosen colorspace any more. I would have expected that when providing an ICC profile that profile would be embedded in the file and the conversion back from XYB would convert back to the provided profile. Is this not the case (or is there a way of specifying this behaviour that I haven't spotted?)

it should not do that unless sRGB output is explicitly requested when decoding

when saving, libjxl should save a color encoding that is at least equivalent to your Rec. 2020 ICC profile

does jxlinfo confirm that it is not the case in the saved .jxl file?

Now I'm even more confused... jxlinfo returns: ``` $ jxlinfo -v M42-unstretched32.jxl JPEG XL image, 3112x2084, lossy, 32-bit float (8 exponent bits) RGB num_color_channels: 3 num_extra_channels: 0 have_preview: 0 have_animation: 0 Intrinsic dimensions: 3112x2084 Orientation: 1 (Normal) Color space: 1256-byte ICC profile, CMM type: "lcms", color space: "RGB ", rendering intent: 0 ``` and the 1256-byte size of the ICC profile it mentions matches the size of the profile I provided when saving. But when I load it and retrieve the ICC profile with JxlDecoderGetColorAsICCProfile(), and then export the ICC profile as a file, the size is 732 bytes (this is the one with the description I mentioned above).

Oh. I've just noticed - I'm retrieving the profile with JxlDecoderGetColorAsICCProfile(dec.get(), &format, JXL_COLOR_PROFILE_TARGET_DATA, icc_profile->data(), icc_profile->size()); should that be JXL_COLOR_PROFILE_TARGET_ORIGINAL instead?

ah, right

target_data is the colorspace in which libjxl returns it when decoding, which is not guaranteed to be the colorspace in which it was originally supplied (although we have been landing changes to make it possible to request it to be output in a specific colorspace)

Ah, OK. I'll wait for those changes to mature then, and for now I'll just take it in the target_data colorspace and transform it to the user's preferred working colorspace.

Is it allowed to call JxlDecoderGetColorAsICCProfile() twice, once wiht JXL_COLOR_PROFILE_TARGET_DATA and a second time with JXL_COLOR_PROFILE_TARGET_ORIGINAL (with the appropriate calls to JxlDecoderGetICCProfileSize()) in order to get both the original and the decoded data ICC profiles?

as far as I know, yes, there shouldn’t be any issue with doing that

or if there is, it’s a bug

thanks. That should let me retrieve the decoded data color profile and the user's original color profile and do the conversion myself.

so the TLDR is that jxl files are typically encoded in an absolute space called XYB. Upon decoding, you can request the pixels in a variety of RGB spaces, but it will default to sRGB unless you tell it otherwise

So if the user is working in BT2020/PQ, you can request that

something that I dont see ~~shilled~~ advertised enough is that jpegxl is an image format with good color management baked into it, I see a lot about how XYB is good for compression, but IMO what matters a lot more is that it will just look fine on even a rudimentary color managed application so long as tonemapping doesn't need to take place

also is it lowercase xyb or upper, I keep seeing it in both

uppercase

thx

I got it working in the end. I wasn't able to request the RGB data directly in the original colorspace but I was able to make the decoder provide the ICC profiles for both the decoded data and the original one provided when the data was saved and do the conversion myself, which is good enough.

Hi guys, i'm having trouble to set lossless encoding. I'm using a pure mockup sample u16 RGB image data 100x100 pure red. ``` 1. JxlEncoderSetBasicInfo 2. JxlEncoderFrameSettingsCreate 3. JxlEncoderSetFrameDistance 4. JxlEncoderAddImageFrame ``` This lossy setup progress has no error. ``` 1. JxlEncoderSetBasicInfo (with uses_original_profile changed to 1) 2. JxlEncoderFrameSettingsCreate 3. JxlEncoderSetFrameLossless 4. JxlEncoderAddImageFrame ``` This lossess setup will cause a `JXL_ENC_ERROR` on step 4. And `JxlEncoderGetError` will get `JXL_ENC_ERR_API_USAGE`.

Other settings are untouched

Oh i get it, I must set color encoding manually

Hi guys, I cannot find a way to set `Intensity` in libjxl

for C++ code, one can use `jxl::SetIntensityTarget`, but it's not exposed in c api

oh, it can be set in basic info

Will djxl PNG output use fpnge in the future? I hope so...

the PNG compression stage takes up most of the runtime of DJXL sadly

But setting intensity doesn't mean to make it a HDR image. According to my test here in jxl channel: https://discord.com/channels/794206087879852103/794206170445119489/1176554409040298025 Don't know how to make a 16bit PNG to be a HDR JXL

you most likely need to set the metadata, like primaries or transfer

untagged png typically defaults to sRGB or bt709/gamma22

how much does it affect compression? https://github.com/libjxl/libjxl/pull/2957

Very little, it does affect speed though

The author added test info to the pull request and the answer is... Looks like almost no change in compression, but at a loss of speed. 0.001% increase in file size: +1 KB for the file tested. 16% slower speed. 98% lower memory usage. This is what I expected just quickly glancing at the code. It looks like most of it is breaking things out into smaller functions and doing better memory management, but the author could explain it more.

due to smaller pre-buffering/cache and reading speed is the bottleneck? wondering how Halic can keep high speed, with also very low memory consumption, good tuning for data block sizes?

part of the slowdown is just that disk IO is measured in the MP/s now ;P

but I'll give it a pass with a profiler later

maybe add options for larger cache, memory consumption would also be higher, but still much smaller than normal mode and predictable and not depending on image size?

Who knows 😛 I have to admit I am not sure at all where the slowdown is coming from

https://discord.com/channels/794206087879852103/803645746661425173/1177006237913718944

The slowdown is only when streaming like with pipelines?

Maybe there's a pause to flush the output buffer and refill the input buffer?

streaming is "--streaming_input --streaming_output" modes and for HDD the difference is similar, so not like the drive speed is the main factor

When editing JXL files in iOS Photos App, if the transfer function in color encoding is not `HLG` or `PQ`, it will display like the screenshot. I think it's a bug caused by Photos App. But I'm wondering if there can be a tweak on libjxl options to overcome this Apple's issue..

unsure if apple is using libjxl or if they are using their own implementation

Well, isn't that peculiar?

apple will convert any image to HEIC and edit the heic file.

I think the problem is in the conversion

thats terrible lol

That's so bad. Why would they do that...

TIL iOS contains a copy of AMD's FSR

Then they can keep the original file untouched for later revert

They could use something that doesn't suck like EXR. That would be a good choice.

apple just love heic

Such a weird design choice.

oh here we go, iOS legal notices says > JPEG XL Project Authors ( libjxl ) > Copyright © the JPEG XL Project Authors. All rights reserved. > [insert license text here] Right after Joyent's events library and before jQuery.

so they probably use libjxl

it wouldn't be in the FOSS license list

im using the nightly libjxl, maybe they are using older version

but i tried with 0.8.2 cjxl before, still causing this issue

i wouldn't be surprised if the version they're using is a year old

I tried to set the ICC profile bytes exported from Apple swift code, it still has this issue.

Is the `have_preview` in basic info telling the encoder to generate a preview for jxl result or it's just indicating the src input contains a preview? I set `have_preview` to 1 and nothing happens.

Another question is for `modular` encoding mode, do i only need to set `JXL_ENC_FRAME_SETTING_RESPONSIVE` to 1 to enable progressive encoding? And for `VarDCT` encoding mode, do i need to set all the three `JXL_ENC_FRAME_SETTING_PROGRESSIVE_AC`, `JXL_ENC_FRAME_SETTING_QPROGRESSIVE_AC` and `JXL_ENC_FRAME_SETTING_PROGRESSIVE_DC` to 1 to enable progressive encoding?

It seems I only need to choose one progressive mode for VarDCT

Will progressive encoding reduce the generating size for modular mode? I tested on a 12MB image: modular with progressive => 4.2MB jxl modular without progressive => 6.9MB jxl (by setting `JXL_ENC_FRAME_SETTING_RESPONSIVE`)

if it's that smaller why isn't it default?

it tells you whether or not the image has a Preview Frame

it isn't clear what other variables are controlled here. progressive enables squeeze, which isn't exactly good at lossless

for lossy modular, squeeze is enabled by default

Actually the progressive setting is enable by default for modular mode. You can try it first.

Thanks

is there any plans to use streaming by default for modes that don't get much worse, like e1, if the slowdown issues are fixed? it would be easier for ffmpeg and other software that uses libjxl, without extra options, although I don't know if ffmpeg has any capabilities for streaming input and output for images

ffmpeg operates on a frame-by-frame basis for the most part

the decoders receive packets, which contain enough info for a full decoded picture

which for the most part is one packet per JXL still image

the encoder outputs one AVPacket per frame of input, so in that regard, streaming doesn't help

Where is it used?

this was unexpectedly very fast I didnt expect a 6s decode on this machine ```ps OS: Arch Linux i686 Host: Compaq Mini 110c-1100 0394110000001C00000300000 CPU: Intel Atom N270 (2) @ 1.600GHz GPU: Intel Mobile 945GM/GMS/GME, 943/940GML Express Memory: 215MiB / 977MiB ``````ps jxlinfo mona.jxl 11/29/2023 09:35:40 AMJPEG XL file format container (ISO/IEC 18181-2) JPEG XL image, 5221x2847, lossy, 8-bit RGB Color space: RGB, D65, sRGB primaries, sRGB transfer function, rendering intent: Relative Brotli-compressed xml metadata: 772 compressed bytes ``````ps root@archiso ~ # hyperfine --runs 5 'djxl mona.jxl --disable_output' Benchmark 1: djxl mona.jxl --disable_output Time (mean ± σ): 5.944 s ± 0.007 s [User: 10.453 s, System: 0.925 s] Range (min … max): 5.934 s … 5.954 s 5 runs hyperfine --runs 5 'djxl mona.jxl --disable_output' 53.07s user 5.03s system 188% cpu 30.790 total ```

``` JPEG XL decoder v0.8.2 954b4607 [SSE4,SSSE3,SSE2] Read 14744762 compressed bytes. Decoded to pixels. 23164 x 18480, 56.52 MP/s [56.52, 56.52], 1 reps, 1 threads. real 0m7.740s user 0m6.564s sys 0m1.157s ``` I wonder why it's spending more than a full second just waiting on the system when num_threads = 1 (and disabled output)

if you run strace -c djxl ... it will tell you

its likely futex or something taking up the brunt of it

even with num_threads = 1 though?

Seems odd

I twonder if it's like that in the library as well, and I wonder if single threaded performance can be improved. I noticed djxl decodes images much faster than software that links to libjxl

num_threads=1 is one controller and one worker

you want to set num_threads=0 to disable threading

if num_threads = N, that means N worker threads and one master thread

in that regard num_threads=1 is very much not useful

``` --num_threads=N Sets the number of threads to use. The default 0 value means the machine default. ```

they must have changed how that works then

I can confirm that it's decoding on multiple threads when set to 0

anyways it just seems like an unexpectedly large amount of sys overhead

cjxl says: ``` --num_threads=N Number of worker threads (-1 == use machine default, 0 == do not use multithreading). ``` djxl says: ``` --num_threads=N Sets the number of threads to use. The default 0 value means the machine default. ```

weirdly inconsistent

interesting

JPEG XL decoder v0.10.0 77a39356 `Number of worker threads (-1 == use machine default, 0 == do not use multithreading).`

ah yes, they fixed it in git master

...what happened to 0.9?

789

0.9 for upcoming release, 0.10 current dev git version

lol. Good answer

I hope fpnge will be integrated into djxl soon in a release. Many have already gotten a bad impression of decoding times thanks to the slow PNG encoding

And they aren’t aware that it is not the decoding that takes a long time, but the PNG encoding

send a pr 🙂

but the bigger issue is that it's x86 only...

I haven’t taken a look at djxl’s source yet, so I don’t even know why it’s so slow to begin with, or what library it’s using

I tried making a deepzoom image using JXL as the tile format http://www.rollthepotato.net/~john/deepzoom-jxl/

it actually works! you need safari on macos though (of course)

looks like this in on a mac:

it's very fast and smooth!

it's a 46,000 x 32,000 pixel image from openslide

I had to add a tiny patch to openseadragon to add jxl to the list of allowed tile formats, but that's all

so .... my feeling is that JXL needs a tiled mode to make it useful for larger images at the moment, JXL is not practical for images larger than about 2k x 2k (imo), making it unsuitable even for large cell phone pictures

Interesting that you have come to that conclusion. even my old winxp compaq only needs 5s to decode an `5221x2847` image, my s9+ with libjxl 0.8.2 it can decode a `7680x4320` in 2s on a single thread, when little libjxl go as many threads as it wants, it's 0.5s on a lossless jpeg transcode. for an image without reconstruction, it still only takes 4s, IMO that's still quite fast and iirc libjxl git is quite a bit faster then this.

the main issue is memory use, for example: ``` $ /usr/bin/time -f %M:%e vips copy nina.jpg nina.jxl 2302672:4.45 ``` 2.3gb of ram to encode a 6k x 4k jpg you need a tiled format to get memory use during encode and decode under control (imo)

we're working on that

without tiles 😉

i.e. cjxl -e1 -d0 will encode pretty much any PPM using only \~10mb of memory

doesn't it still need to have the whole input image in memory though?

that's why I said ppm

😉

it's not needed now

(with the streaming encoding API)

the streaming encoding API is push, not pull, so it's very hard to use a tiled mode would be much, much simpler for downstream libraries to integrate

> the streaming encoding API is push, not pull, so it's very hard to use did you mean pull and not push?

hehe I suppose it depends where you sit

the streaming *decoding* API is push and not pull, and I can see that being problematic

(also on the todo list)

currently streaming encoding does "please give me this rectangle of pixels"

and streaming decoding will give you some pixels somewhere without particular guarantees on where (but that can be fixed with reasonable effort)

ooop you're right, sorry, I meant the decode API

not enough coffee

hehe

btw, does alignment of the rectangles we ask for in the encoder API matter?

(it is reasonable to promise 128-alignment and 128x size, but it is also a lot simpler if we can only promise, say, 16x alignment and size)

libvips would like to give libjxl top to bottom strips, ideally imagine converting a PNG to JXL (for example) ... libpng will supply pixels top to bottom in strips, so if the saver needs something other than that, there will have to be a lot of buffering somewhere

yeah there's nothing that can be done about that

ah you mean pixel alignment? no, doesn't matter

jxl is internally tiled

so *somewhere* something needs to buffer to convert to/from formats that decode top-down

IIRC stuff newer than webp is like that

do you plan to support "please give me this rect of pixels" in the decode api? that would be very nice

plan is a strong word, but it might be the next thing we do after streaming encoding

hehe

ok, then I shall continue to wait patiently and stop asking for a tiled mode

thanks for clearing this up for me!

fwiw, the current streaming decoding API *will* give you things in 128x128 to 2048x2048 tiles, likely in top-to-bottom order

(plus or minus a few pixels here and there)

(^^ that deepzeoom JXL demo is fun btw, if you can find a mac)

(in fact, you'll get the tiles in the order that they are in the file, modulo threads, which is usually raster order)

I don't think I'll be able to use the current streaming decode API, I'll wait for a pull version

yeah that seems fair

but streaming encoding is IMO the more urgent thing, decoding is fairly well memory optimized and takes little more than the output

I'll add streaming encode though, that seems pretty easy

encoding OTOH can easily take ludicrous amounts of memory

yeah it will not really help for now (except -d0 -e1, and even that is ~alpha), but working on it...

is there any png encoders/decoders that support streaming? to also support one of the most used compressed formats libpng as far as I know?

libspng too

libheif (AVIF and HEIC) and libwebp don't have streaming, but I can't think of others offhand