JPEG XL

I prefer something like that, where the encoding and size matching is already done

(also what I like about it is that you can press SHIFT to toggle between the two images, which to me is a lot more comfortable than dragging the slider back and forth)

the above comparison page was made by people from chrome/avif/webp2, it's not exactly a fair comparison since they use the slowest possible avif encode setting vs default jxl

(so this is comparing jxl taking 0.4 seconds to encode an image to avif taking 1.5 minutes to encode an image)

The color of the yellow glasses is severely degraded compared to AVIF, and that's photographic content. DCT is definitely not a good fit for non-photo content, not sure if Squeeze would be any better.

But somehow AVIF does a passable job...

even given that it's a non ideal algorithm

What version of libjxl is in squoosh ? 0.6.1? I wonder if things are better in 0.8. There have been a few quality tweaks since that version...

guessing it's related to using an older version

libjxl had a number of bug fixes relating to distorting color recently

0.8.0

Hmm, well the glasses don't actually seem so bad anymore now that I'm comparing them again.

I find that AVIF quality settings lower than 40 on squoosh.app have pretty significant degradation

For JXL that would be about -q 78

to reach the same bitrate I mean

Not the same quality. JXL has clearly better quality at that bitrate

But there are so many different factors that effect the psychovisual performance of lossy avif

99% of digital art uses raster workflows

I wouldn't know the percentage but sure, vector is certainly not always an option

But people drawing flat illustrations in Adobe Illustrator or whatever shouldn't be rasterizing them and use jpeg or avif to compress that, they should use an svg optimizer

it might be the case that many sharing platforms simply do not support vector uploads

Even then, I'd rather use a high-res palette png than something with ringing (like jpeg) or smearing (like avif)

Anything paletted breaks appart when put against extensive gradients

At the end of the day, this is a shift of burden. Good lossy compression of synthetic productions is already here. I kinda just wish it was also in jxl.

Jxl can do delta palettes, which fixes the gradient problem

was that already implemented in the test you shared?

oh, you talking about lossless jxl?

a palette preprocessor specific to jxl that leverages delta palettes would be intetesting

no, delta palette is meant to be used for near-lossless, but it's kind of experimental still

sounds good

please be aware that a lot of artwork isn't exactly "anime style" but rather falls somewhere in the middle between synthetic and photographic content. This is one of the key areas where avif apears to excel, because it's not organic enough to have the noise and paterns of real media, yet it's easily too complex to efficiently compress using lossless algorithms and preprocessors.

However, IIRC, I've heard something about 65K color palettes. That could shift the tide.

yeah, for complex art lossy methods work better; avif does well at the lower bitrates because its artifacts are mostly blurring and smearing (and maybe some banding, but generally not nearly as bad as jpeg at low bitrate), which is not as annoying as ringing / dct noise

there can be some annoying "surprises" though, when the smoothing is just ruining a texture or when the smearing causes lines to get a bit weird (due to basically being 'rounded to the nearest available angle')

banding is effectively solved since nobody in their right mind uses less than 10 bpc in avif. and just like jxl, film grain can be used as error diffusion.

surprises are everywhere, jxl likes parkinsonian chroma blocking in edges, avif likes to schmear paticular kinds of textures

Artists often don't understand internal workings of image formats. There are some who are happily using JPEG at q100 with 2x2 chroma subsampling.

which is why having good defaults is paramount

It is the job of software developers to make the codecs easy to use and produce predictable results.

Does 10bit help much for banding when doing low quality encode? Also I am not sure if lots of people do actually use 10bit avif. It makes sense of course, even if the input is 8bit rgb (because yuv eats one bit and tv range another half bit), but there's also a cost in speed (both encode and decode), so I am not sure if it's actually very widely used. E.g. the avif team at chrome just used 8bit...

yes, it helps even in 8 bit source

specially at low bpp

10bit and low quality seems kinda contradictory.

it actually isn't

10bit will have the same quality at worst

8bit should never be used under any circumstances outside extreme, platform-specific performance considerations

By common sense, spending less on bit depth would leave more room for details.

in practice it doesn't appear to do so

actually it's only on really high bpp noisy material (well into jxl territory) where 8 bit might not matter

Most of the source data is 8bit, so it would have to be upconverted. Most display devices are 8bit and likely will be for time being.

in any instance where you perform a mayor reduction in bitrate, 10bpp will help even in 8bit sources due the noise in the source being effectivelly organic error diffusion, which can be leveraged to extract the otherwise unavailable values into the resulting 10 bit low bpp picture.

then on top of that you can optionally apply grain synthesis or film grain to dither the image for 8 bit displays

displays are 8bit RGB, which has about 75% more shades of color than 8bit YUV in most lossy codecs

IIRC

As a compensation for the smaller size of YUV space, that would make sense.

Banding is caused by dc quantization. I would assume that in 10-bit, to get the same bitrate, you apply 4x as much quantization to everything so the quantized coeffs have the same range as in 8bit, just all transforms and filters are more precise so you lose less in cumulative rounding errors.

pretty much

I didn't realize, that in AVIF, the bit depth applies to the internal color space. Where in JXL, the bit depth applies to RGB color space.

more simply, jxl below d2 is not even worth comparing, most video based formats will be better, or rather look better because they have more ways to filter and can highly compress the most visually notable parts, like edges, flat colors but at high quality it's the opposite, I mostly use jxl with d1 and for avif it's very difficult to achieve transparent quality, even at high bitrates it still loses details and smoothes the image avif is good for previews or when it's not important to match the source image and needs higher compression, but what I see in real use is very rare and most avif images even for previews have pretty high bitrate, on which jxl is also good and even better and even d1 is still very good compression and it's not some bloated images, especially for typical image resolution on web pages (images with very high resolution or source photos usually have individual links and require very high quality, having 50 megapixel photos with q40 avif compression would be very strange) i don't even think jxl is even worth the effort to optimize for low quality, that spot is taken by video based codecs and they have far more tools for that, it would be better if jxl improved at medium to high quality where there are still issues with color bleeding/shifting or ringing artifacts

very useful feature <:Poggers:805392625934663710> https://github.com/libjxl/libjxl/pull/2146

and it would be nice to have a similar one for other formats

Well I wouldn't complain about any improvements, regardless of where they are made :)

Improving libjxl at low quality is always nice, as long as it doesn't cause high quality to get worse as a side-effect.

yeah, I meant making improvements on low quality as a priority when it requires significant effort, any improvement is always nice, especially some improvements can affect on all qualities

An interesting way to improve libjxl output at lower qualities would be to integrate a form of very fast encoder psy-RDO that allows the encoder to better perform at low bitrates that targets better structure retention at low bpp and targets better frequency retention at high bpp.

I guess ssimulacra2 could work in the meantime in place of butteraugli as a shortcut in that regard.

All theoretical vomit until it's implemented in a proper framework, or just better expressed from my part.

Libjxl currently basically just tries to reach a fidelity target, without trying to do rdo. That works fine for d0.1-d3, but at lower quality you really should consider the trade-off between signaling cost and distortion. Perhaps we could start by implementing some trellis optimization like mozjpeg has.

Though I don't really see that much of a use case for d >> 3, besides making impressive examples of how nice things can still look at very low bitrates.

Just looking at the cjxl help page, I'm wondering if the `--brotli_effort` and possibly the `--resampling` options should be hidden behind the verbose flag, as they don't seem like things that would common usage. (I'm also a little surprised that brotli effort isn't implicitly derived from effort.)

It makes sense to have completely separate encoders for different bitrates and purposes, like fjxl

Although it's not good to overcomplicate things when they can be generalized

I am on Arch Linux and exiv2 seems to be build with BMFF enabled: https://github.com/archlinux/svntogit-packages/blob/packages/exiv2/trunk/PKGBUILD

And yet I don't see any metadata on my JXL files...?

are the boxes brob compressed?

if so exiv2 can't read them iirc

Hmm probably if that's the default

Well that would explain it I guess

Wonder why everyone uses exiv2 if it's so useless with any format made after 1990

Because a format made 30 years ago is still by far the most popular one 🙂

Yeah...

libjxl doesn't do trellis quantization? is there a reason why not, besides lack of engineering resources? (i.e a theoretical reason against it)

<@532010383041363969> probably has more insight on this than me. I don't know of any reason why not, but maybe some of the current vardct heuristics end up doing something similar enough for it to not help much anymore, or something like that.

Yeah, but with RDO, you can get some amazing coding gains at lower sizes, which is rather important.

Considering libjxl current doesn't have RDO, the encoder is actually very impressive in this regard.

What is RDO?

rate-distortion optimization

Ah. Thanks.

there is no trellis quantization in libjxl

I believe that trellis quantization is more interesting when there are a lot of zeros in the dct (low quality) and at higher quality it tends to slow things down, leading to somewhat unpredictable encoding performance

it's probably something we should start doing only at d > 2 or so, if possible in a gradual way so we don't introduce a discontinuity

doesn't `num_zeroes` allow us to efficiently encode zeroes though?

I'm not entirely sure what trellis quantization is, tbh

num_zeroes makes storing trailing zeroes efficient, but that's not super different from regular jpeg which also has an "end of block" symbol that makes trailing zeroes cheap

in trellis quantization, iiuc, what you basically do is turn some of the coeffs that are nonzero into zeroes, if the cost of signaling the nonzero is "not worth it" compared to the quality improvement that coeff gives. In particular, zeroing the last nonzero coeff could result in getting more than one additional zero becoming part of the trailing zeroes so if that happens then it has a large signaling cost compared to a nonzero somewhere in the beginning.

I see, so it's essentially a reduction in quality for a larger reduction in cost

yes, or if you start from a higher quality to begin with, you get better quality for the same bitrate

what you need to be able to do for this though, is estimate the signaling cost of things. This is easy for regular jpeg which has very simple signaling (just huffman coding), so it's not very hard to estimate the cost of symbols. In jxl things are a bit trickier because we can do coefficient reordering, context modeling and clustering, etc — meaning the cost of signaling depends much more on what actually ends up being signaled, so it seems a bit harder to estimate signaling costs.

that said, AV1 manages to do it so in principle it can be done... or approximated

thanks

An interesting thing to do with treillis quantization is to make it frequency weighted.

we have other anti-psnr approaches to quantization that are O(1) and don't slow down with higher quality

we could add more complexity in quantization decisions where we stay in O(1) and don't slow down with higher quality

I have ideas that can give ~2 % more density

does anyone know how to get debian to change the composition of the libjxl-devtools package? is Mathieu Malaterre on this discord?

it currently has these tools: /usr/bin/benchmark_xl /usr/bin/butteraugli_main /usr/bin/decode_and_encode /usr/bin/display_to_hlg /usr/bin/fuzzer_corpus /usr/bin/generate_lut_template /usr/bin/jxl_from_tree /usr/bin/pq_to_hlg /usr/bin/render_hlg /usr/bin/ssimulacra_main /usr/bin/texture_to_cube /usr/bin/tone_map /usr/bin/xyb_range

that's quite a mixed bag of stuff, and probably some of these are too niche to be useful to anyone, while some things are missing

also the names of these tools are not always very nice

(in particular names ending in _main look silly to me)

maybe we should change the names in the libjxl repo, and bring some more structure in the tools folder

I think it would make sense to have one cluster of tools related to benchmarking and metrics: benchmark_xl, butteraugli, ssimulacra, ssimulacra2 (*)

then another cluster related to HDR conversions: display_to_hlg, exr_to_pq (*), generate_lut_template, pq_to_hlg, render_hlg, texture_to_cube, tone_map

xyb_range is not something worth distributing. also fuzzer_corpus is not something useful to the general public imo.

decode_and_encode should be renamed and is probably useful as a general colorspace conversion tool, but it really needs some better help/man page

jxl_from_tree is just for fun and might be worth keeping but maybe as a package on its own and with a man page that can be copied from https://jxl-art.surma.technology/wtf.html

(*) are tools currently missing from libjxl-devtools and probably useful enough to be included

we should probably also turn the input format reading code that is shared between many of these tools into a library to reduce the binary sizes a bit... and allow the tools to take jxl input, which they currently don't 🙂

How do you locate updated libjxl builds on Github? I'm unable to find them. They're either very well hidden or I'm just blind.

if you meant git master builds, I don't believe there are artifacts for those automatically generated

I'm interested in up-to-date master builds. I'm getting lost in the Actions sections. I'm finding only job logs and no files.

<@688076786525143117> go here https://artifacts.lucaversari.it/libjxl/libjxl/latest/

Thank you. I usually download updated builds from this server. The repository was not updated for a few days, so I was wondering if there was a newer build on Github. I know there are builds on Github. Devs sometimes post a direct link to the artifacts, but I was never able to find them on my own.

the GIMP plugin fails to save lossless JXL files

``` file-jxl-save Info: No ICC profile. Exporting image anyway. file-jxl-save Warning: Using internal profile. file-jxl-save Warning: JxlEncoderSetColorEncoding failed file-jxl-save Error: JxlEncoderAddImageFrame failed ```

prints this to console when it fails

GIMP version 2.10.32

where do I report this? is this libjxl or somewhere downstream?

never mind, sent a PR to fix this :)

https://github.com/libjxl/libjxl/pull/2188

https://github.com/libjxl/libjxl/issues/1931

Are there really no builds after 10tf Feb?

huh I didn't realize it stopped updating, I wonder what happened? since I assume it's automated. maybe the build system for it broke or failed at some point because of a commit?

I don't know since i'm not behind the site

Thank you. So next time I need to go to Actions and Release build/Deploy. There are the builds. I was looking at the wrong place all the time.

uhhh that's odd, I'm currently horrendously busy with a programming contest but I'll hopefully fix this on Monday... maybe nag me until I do?

I don't see where the artifacts are listed

nag

No clue what happened, systemd decided it didn't like my timer

What kind of competition?

SWERC (swerc.eu)

Did you enjoyed it?

was it enabled?

ok I figured it out, the unit that was started by the timer got stuck

yup, it also went very well on the technical side for once xD

ah, it's on the summery page, not on the individual thing page

What exactly is cjpeg_hdr and how does it relate to jpegli? Does it use the same underlying code?

It's an earlier experiment by <@179701849576833024> to do accurate jpeg encoding to see if it's good enough for HDR, iirc

Jpegli was started from there IIRC

yes, and then there was a refactor to make cjpeg_hdr use the same code as jpegli

if you mean “better than q100” then yes, at least when the image is also decoded with jpegli; if you mean “better quality for the same size” then also yes AFAIK

re: the first point, from what I recall, when we tried HDR JPEG with PQ/HLG using libjpeg-turbo, q100 was not enough to prevent banding

but the approach used in jpegli was

so jpegli should unlock decent HDR in legacy JPEG

(with HLG perhaps a better fit than PQ)

libjpeg-turbo starts by converting 8-bit RGB to 8-bit YCbCr — that alone is already losing ~1 bit of precision. Then it does a DCT implemented in int16 on that, which is also not super precise. Then on decode it does the inverse things again with limited precision.

it's a number that lowers quality as it goes higher

I don't think I would be confident to say any more than that 😛

and 0 = quant table of only ones

which is as lossless as it gets for a valid de facto jpeg file, but not quite lossless

does true lossless jpeg exist?

or is it a poorly supported standard extension?

or something like that

I think you are thinking of something that libjpeg added that is not apart of the jpeg standard

I think not 😆

yes, it is part of the specification but practically no one implements it

the term may also refer to JPEG-LS which is a different, later format

see: https://en.wikipedia.org/wiki/Lossless_JPEG

also it's quite bad, it's basically encoding the whole image as if it's DC, iirc. So that's using the pixel on the left as a predictor and huffman coding the residuals, without lz77. A bit worse than PNG on photographic images, a lot worse on nonphoto.

ah, and it's not decoder conpatible, so I can see why it's not really great

I would have expected `cjxl --container=0 -e 9 0006.jpg 0006.jxl` to discard the container and jpeg reconstruction data (which was not the case)

```$ jxlinfo 0006.jxl JPEG XL file format container (ISO/IEC 18181-2) JPEG XL image, 1280x1834, (possibly) lossless, 8-bit RGB Color space: RGB, D65, sRGB primaries, sRGB transfer function, rendering intent: Relative JPEG bitstream reconstruction data available frame: full image size ```

but adding `--jpeg_store_metadata=0` worked on version `cjxl v0.9.0 423375e0`

Thanks! I understand. The - - help is a bit misleading then, I'll complete the faq

What's the easiest way to see what --downsampling=x values are available for a given .jxl file, and their byte offsets? I assumed I could see this in `jxlinfo`, but no luck. RELATED: has anyone written anything in Javascript (or compiled anything to wasm) that can read/extract `FileHeader` info, from a .jxl bitstream?

(so far the most efficient thing I have found is to run djxl a bunch of times with `--print_read_bytes` and different values of x, and manually note when the `Decoded bytes:` output changes between runs.)

there are no chats where it can be drop it, so I drop it here. my cjxl build for windows pre pull/2252 fix -- 7e5b5065c5b6f5f1d5b76fea56ec51ade81da06b -- with an appended code for 8 layers of progressive decoding

'no quantization in quality 100 jpegs' means 12-bit quantization of the dct results -- if every rounding in a 64-value block is into the wrong direction, it means an error 0.5 * 64, i.e., 32 steps -- 12 bits gave you 4096 steps, so once you divide by 32 you have 128 steps left -- it might be that quality 100 jpeg, when every step is done perfectly, gives you 7 bits of guaranteed values

If the block is smooth enough (solid color or some gradient), only the low freq coeffs are nonzero so the precision loss due to coeff quantization isn't that severe. The worst case would be for something that uses all frequencies, like noise or fine text.

correct, for smooth gradients one gets nearly full 12 bit precision

I successfully compiled the wasm32 artifacts ! youpi ! I'm not sure how to use them though.... does anybody have a clue on how to use them ? I had no issue using niutech project but I'm missing something here... it seems to be meant to be used with nodejs whereas I would like to use it in a browser

<@1051137277813870592>: What have you compiled to Wasm? And how did you compile it? If you compiled it for WASI using Clang directly (with the Wasm SDK), generally you can use an WASI implementation for the browser. I remember once using Wasmer’s implementation. If you compiled it with Emscripten, you can generally just load the JavaScript file it outputted directly in a browser.

I followed this ! https://github.com/libjxl/libjxl/blob/main/doc/building_wasm.md just to try...

I think that what I miss is: what's exactly inside those artifacts... I got a decode_oneshot.wasm, decode_progressive.wasm (the one I want most to play with) and some others

It should have generated one or more JavaScript alongside those, no? You should be able to just use them directly from an HTML page.

indeed, I'll start with this

what surprised me is that the companion .js file starts with with a shell bang to nodejs... I was wondering if the syntax could differ for a use in nodejs or in a browser...

looking for the high level symbols available in those wasm, I tried wasm-decompile and got a core dump 😅

oh ! I may have found something !

what does color quantization bring for decoding? https://github.com/libjxl/libjxl/pull/2342

<@853026420792360980>: I’m not sure if this is the right channel to talk about non‐libjxl tools like jxlatte, but I decided to take a look into its source code a bit more extensively to try to acquaint myself with it, and I came up with a patch for some small issues I found. Namely: - I replaced `sneakyThrow` with a `RuntimeException` subclass that is skipped when searching for the underlying cause of a throwable in `JXLatte.java`. It feels like a cleaner solution than throwing a checked exception by an unchecked cast. - I fixed some warnings that I got with `-Xlint:all`. - I replaced a comment with a more accurate explanation of type parameter erasure in Java, and of why an unchecked cast was necessary. I will say, for some more cosmetic changes, now that JXLatte requires Java 11, maybe it’d make sense to use local type inference more actively. I feel like it can help improve readability sometimes! Also, something else: I have been trying to get jxlatte to compile to Wasm with tools such as Bytecoder. (Though I don’t think the changes are appropriate for being merged upstream.) If I succeed, maybe I ought to consider replacing libjxl with it for my extension! 🙂

you need sneakyThrow to throw exceptions from inside stream reductions and other thinks like that

it lets you maintain the checked exception nature

Is there any reason to want to “maintain the checked exception nature”?

if you don't it defeats the entire reason checked exceptions exist

also you can always send pull requests to the github repo it's hosted on

But wasn’t its purpose to circumvent the checks? Otherwise you could just `throw` the exceptions normally.

Yes, but then everyone can see it! 😳

only for functional interfaces. so if I have a function which creates a stream and iterates on it, then I still want the function to have a meaningful Throws clause

Ah, I see. I feel like a sensible solution might be to use type parameters for functions. It feels strange to just throw checked exceptions in an unchecked way, even for functions and futures, because the function can end up being called somewhere else altogether, and it’s up to you to make sure that they are called in the correct place.

api compatibility -- someone added it in libjpeg and now if you want to build a compatible library you need to support such things

discretizing to 8 bit color might have reduced memory needs at some stage (1995 or so)

it may still be used by some jpeg-to-gif conversion tool

cjxl cannot handle a 42458x38011 pixel PNG? `JPEG XL encoder v0.9.0 6d38e955 [AVX2,SSE4,SSSE3,SSE2] lib/extras/dec/apng.cc:467: JXL_CHECK: frame->rows[row_num] < frame->pixels.data() + frame->pixels.size() Illegal instruction (core dumped)`

Weird. Is it using libpng? Maybe it is hitting some libpng limit and not checking for an error from libpng in the right way?

Yes, libpng

turns out 32 GB is not enough RAM anyway

out of curiosity, does it work with hydrium?

I'm wondering if lodepng can fit that PNG buffer inside your RAM

Well, I ran out of RAM after reverting to 8bb9d6ec. Seemed decode was going fine except for libjxls huge buffers.

`libhydrium version 0.2.2 ./hydrium: error reading PNG file Total libhydrium heap memory: 0 bytes Max libhydrium heap memory: 0 bytes`

Used less than 40% RAM before bailing.

that means lodepng failed

unfortunate

I should roll my own PNG decoder that can do one Nx256 thing at once

Maybe it could be worthwhile to try `stb-image`.

what's stb-image?

See: <https://github.com/nothings/stb>

8k LOC?

seems like I could do my own zlib-based decoder for less

I see! Well, go for it, then! Though I will note that it includes its own gzip decoder, as well as support for image formats other than PNG.

I have no need for that tho

That’s fair. I was just justifying the number of lines, is all. Besides, I was just wondering whether it would be enough to overcome the image size issues. (I think it will intentionally limit the image dimensions by default, but I think you can customise it with macros.)

It might be worth it to use lodepng for zlib and just brew a png decoder that decodes Nx256 at a time

One potential issue is one of colorspaces

id rather not have a whole cms library all over again

and how much of this code should be in the library and how much should be in the frontend

I'm not a fan of stb image personally

I would suggest spng, but it can't do animated pngs

oh, that's a good suggestion, this is a nice API

try now. I switched to using spng, which loads PNGs progressively

worked

only problem is nothing can decode the jxl, and there's probably not enough time in the lifespan of the universe anyway

(decode with my memory constraint)

what's your memory constraint?

32 GB overall

I feel like libjxl *should* be able to do that

what about jxlatte?

nope

jxlatte isn't very ram-friendly so that's not really a surprise

In my experience the JXL decode wasn't any bigger than the original PNG, but maybe that changes at such large sizes

libjxl is quite memory-hungry. But decoding a hydrium encoding takes like 2x as much memory as a default cjxl encoding.

hm, that's intetesting

Probably the frame blending being implemented in a naive way

I wonder how libjxl tiny handles it

why did you make a tree that big if you were just going to complain ```JPEG XL encoder v0.9.0 6d38e955 [AVX2,SSE4,SSSE3,SSE2] Read 21888x14592 image, 394324866 bytes, 184.2 MP/s Encoding [Modular, lossless, effort: 7], lib/jxl/modular/encoding/enc_ma.cc:987: JXL_ASSERT: tree.size() <= kMaxTreeSize Command terminated by signal 4 1068.98user 1808.18system 47:02.95elapsed 101%CPU (0avgtext+0avgdata 30146864maxresident)k 772592inputs+0outputs (17major+2340192864minor)pagefaults 0swaps```

101%CPU?

was trying out one of those multi-core processors

apparently useless

320 megapixel 🙂

market's dream

320 megapixel 🙂 effort: 7

idea: announce JXL has built in AI that doubles megapixels and makes iPhone shoot better photos then see how AVIF dies

try other lower effort options, start with e 1 and see if it works

Could you open an issue for that? Indeed silly that it creates a tree that is too large.

Surely this error is only possible at e4+ where there is MA tree learning. Anyway, e5 worked, just didn't compress better than the PNG.

djxl writes cICP and iCCP for srgb output pngs, instead of sRGB

is there a reason for this?

Probably it just does that for all enum spaces that can be represented with cICP? Writing sRGB is probably better for interop though...

instead of? I’d have thought we’d write both

W3C’s version of the PNG specification (which I think introduced the `cICP` chunk), recommends against including both `cICP` and `sRGB` chunks in the same file. Also, it states that if a decoder encounters a file with both `cICP` and `sRGB`/`iCCP`/`gAMA`/`cHRM` chunks, it only supposed to take the `cICP` chunk into account.

I don't see any such recommendation to not include sRGB if cICP is present in the cICP section 🙂

(then again, that specification also allows you to do horrible things such as limited range RGB images, which I do not want to see ever)

in general having cICP and also other stuff is a good idea, b/c as far as I can tell only Chrome supports cICP in PNG for real

True, I guess they forgot to write it there. After all both the `sRGB` section (https://w3c.github.io/PNG-spec/#srgb-standard-colour-space) and the chunk ordering constraints (https://w3c.github.io/PNG-spec/#table53) do include notes about this.

it only talks about iCCP and sRGB 🙂

although arguably if you have cICP it's not too different from if you have iCCP

Ohhh... I misread it as `cICP`. Sorry.

I guess an srgb ICC Profile inside iCCP is just as good as sRGB?

as good for what? likely not size

I suppose there exists software that: - ignores any colorspace info - only understands gAMA/cHRM - applied gAMA but not cHRM - only understands sRGB/gAMA/cHRM - applies iCCP but not gAMA/cHRM - understands anything except cICP - understands everything

I'm just wondering what software respects the iCCP tag but not the sRGB tag

other than imagemagick, before that bug was fixed

The same source code but in 1 executable file and works 10% faster

Minor fixes of the build above 👆 The build was without a jpeg decoder, now it is available for transcoding. Fixed the shortcomings of which version it is, due to the fact that it was decided to take all the sources from scratch and make One file change, everything else untouched. Uses latest versions of libpng 1.6.39 zlib 1.2.13

speed could be slightly better maybe if zlib-ng was used instead of normal zlib, zlib-ng has a compatibility mode to work with programs that just use the normal zlib api

not sure how much work to switch it out even with zlib-ng having a compatibility mode

I tried to jpegli a scan I did ... and it fails

```Read 2435x2768 image, 13813083 bytes. Encoding [YUV d1.000 AQ p2 OPT] jpegli encoding failed ```

I would like to know if I can ask jpeli to tell me more on its failure ...

here is a link https://drive.google.com/file/d/1Wa41uJsW-EATWUoJGrmerYLqcDCjyDG7/view?usp=sharing if someone has an idea

iirc cjpegli fails if input png has alpha channel

oh really ! I check that

well well, I'll have to batch strip the alpha channel then

hi i want to reproduce results of the FastGaussian from jxl/gauss_blur.h in ssimulacra2 the closest i currently get is: ```python from scipy.ndimage.filters import gaussian_filter blurred = gaussian_filter(image, sigma=1.5, mode="constant",cval=0)``` but there are cases in which the results are not the same. How can that be?

No idea but it probably doesn't matter that much as long as it's a reasonable approximation of gaussian blurring. Maybe you treat the edges differently?

I treat the edges as 0 padding. Somewhere in the code i saw a comment which says padding is "reflect" but this is certainly not the case

for my testing image the different gauß blurs change the core from 18 (original) to 49(my perfect gauß blur)

Maybe try an image with just a single white pixel on a black background as input. Could be the sigma is not scaled the same way or something...

mhm it has to be something different....

I used blurred a image like this (but bigger) ``` 0 0 0 0 1 0 0 0 0 ``` and used the output as kernel for 2d convolution and the results still differ<:PepeHands:808829977608323112>

I got inspired by Apple's jxl announcement and updated my JPEG XL PHP library with the latest binaries and added a new option to use the jxl system binary from the system PATH instead of the bundled static binaries 🙂 https://github.com/joppuyo/jpeg-xl-encode

my aim is to provide a JPEG XL library that works out of the box in any PHP project and provides a simplified API for the most common use cases

can you show the difference ?

mu1 is unblurred. sigma1_sq is blurred with the fast gaussian. diff ist the reshaped 1st plane of the difference between that scipy blur and sigma1_sq. In the first to file the format is [XXXX.... YYYY.... BBBB...]

I though of something more... visual, like a picture

Oh then no

looks to me like fast gaussian is doing mirroring at the edges while you're padding with zeroes?

no the fast gaussian is 100% padding with zeros. But now i think my error is somewhere else in my processing pipeline.

because the first plane is acually correct. but the second and third is not<:monkaMega:809252622900789269>

okay looks like my gaußblur is fine. Sorry

if it's padding with zeroes how can the top left corner of the blurred image not be closer to zero?

The files i sent are wrong. I made a mistake in exporting them

padding is with zeros

Someone please help me understand how it's user friendly to turn djxl help into an RPG. I have to level up three times to access all my abilities, and between level two and three I gain a whopping three perks.

I truly can't imagine a situation where I'd want to see more than the basic options, but not all options, and this just makes me type more.

indeed, it could be essential (level 1) and all (level 2)

I guess 4 levels is indeed unnecessary. I think 3 is useful though: basic, advanced, and "only useful for stuff like conformance testing and benchmarking"

Yeah, 3 and maybe eventual reordering based on what's used most

Currently, it's a mystery what's behind those extra levels, so if your goal is understanding what the tool offers, you simply must pay the typing tax. While it's very common to have a short help and long help, the way the libjxl tools make you work for the documentation is exceedingly rare (no comparable tool comes to mind). The mental model required to associate a certain verbosity level with some granular level of utility is not straightforward (Do I need two or three `-v`s to see everything? I'll just add a bunch.) If you step through each level one by one, the latest options are mixed in with the prior options, so you have to parse through everything again anyway. There aren't enough options that presenting everything at once should be overwhelming, especially for djxl.

If the distinction between the kind of utility behind each level can be efficiently and effectively communicated, it may mitigate some frustration. IMO, such distinction between options would be made much more accessible as separate sections with headers in one long help, assuming there's flexibility to do that.

**The Quest for djxl mastery: An RPG adventure** You find yourself in the mystical land of JXL, where powerful mages and warriors roam. Rumors spread about a legendary artifact known as the djxl Decoder, said to possess unimaginable abilities. Determined to unlock its secrets, you embark on a perilous journey as a novice adventurer. Your goal is to become a master of djxl, harnessing its power to bring balance to the realm. **Chapter 1: The call to adventure** As the sun rises, you awaken in a humble cottage on the outskirts of the peaceful village of Pixelfield. A mysterious letter arrives, summoning you to the Grand Academy of JXL. Eager to discover your destiny, you set out on a path that will lead you to unimaginable challenges and triumphs. _What is your character's name?_

Let's make the interface so you first have to slay a few goblins before it will write the output to file

For every kill you gain 1 bit depth

one comparable tool does come to mind – ffmpeg: ``` usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}... Getting help: -h -- print basic options -h long -- print more options -h full -- print all options (including all format and codec specific options, very long) ```

Maybe it would be comparable if ffmpeg required about 1000 -h flags to see everything.

April release <:PepeOK:805388754545934396>

tried to do something like that: https://github.com/libjxl/libjxl/pull/2593

I see where this is going, so I made my own solution.

-v / -vv would be much nicer

iirc -vv doesn't work for cjxl and djxl

it would be nice if it did since it's fairly standard to support both

`$ cjxlh | head -n 20`

Have a separate executable per option that opens a window to display the text

"Expert options:" "--allow_expert_options" Hmmmmm

1.0 version milestone

and half derivative https://www.youtube.com/watch?v=2dwQUUDt5Is

suddenly made myself a glorified batch command~ https://github.com/kampidh/jxl-batch-converter

Add you to Wikipedia En

https://media.discordapp.net/attachments/808741974596517928/1124417739340128277/Screenshot_from_2023-06-30_12-14-19.png

progress

Then it turns out it was just a coincidental hash clash

three months later: > In this article, we present a method to efficiently generate SHA-1 collisions with the JPEG files that should be reconstructed from JPEG XL codestreams.

> To improve adoption of JpegXL, we now use the .jpg extension and embed a decoder inside the header data of a normal jpeg file

if only OG jpeg was turning complete

Like a U turn?

XD yeah typo

Ooooh, I was going to ask if there's a PHP solution for compressing JPEG/PNG/GIF and decompressing on-the-fly JPEG XL images, and looks like there's one for the compress stage, very nice!

I wonder if there's one for decoding to JPEG/PNG though. My usecase is a reddit-like link aggregator, and we want to store all our user-submitted images as JPEG-XL, and still serve them as JPEG or PNG, whichever is better

This is the project if you're curious: https://gitlab.com/postmill/Postmill

(Ignore the archived repo warning btw, not important)

There’s no reason why the code could not be adapted to do that

libjxl only decodes to jpeg (if supported) and pixels

you would want to encode it separately

The library doesn’t use libjxl directly, it wraps cjxl binary (a static one or one from the system PATH), imagemagick and vips into one interface

if you want to serve images you can always use a cdn

like cloudinary

on the fly server side decoding and encoding is just so computationally expensive

I see, thanks for the responses! I might take a look at the code later (tho I don't really work on PHP, I'm just here because I'm looking for options the developer of the link aggregator I participate in could make use of). I will also see if Cloudinary's CDN could be an option for the Raddle admin, though I suspect they will be going with AWS S3, IIRC

there's also my jxl proxy <https://embed.moe/> which decodes and encodes to supported requested format

Yeah, but I don't want to freeload off of it :P

oh I mean just as an example to see how it's done

ah ok lol

Found this at the bottom of a reddit thread, supports JXL oddly enough https://github.com/ermig1979/AntiDupl CLI Version https://github.com/ermig1979/AntiDuplX

huh I wonder if there's a CLI equivalent

At the end of Readme: Command line tool AntiDuplX There is a command line tool AntiDuplX to search simular and damaged images. This is standalon project which uses similar algorithm of image comparison. It works for Linux and Windows.

huh

Added the CLI link to my message if anyone wants it

if anyone wants to use or take a look at the PNG thingy I threw together, it's here

https://github.com/Traneptora/umbrielpng/

Hi. cjxl claims to support EXR, but so far I've gotten "Getting pixel data failed." and couldn't fish out any specifics

cjxl v0.8.2 from fedora rawhide/39

it claims to support EXR when listing with --help

oh. bummer

thank you, nevertheless

Hi. Can anyone modify this conversion script so that it exits after the loop ends? The first version of this script was written by Traneptora, then I used ChatGPT to improve and add two new features. But I ran into a problem that I could not solve, and AI only offers some nonsense that does not work. `@echo off & for /R %i in (*.jpg) do (C:\My\Programs\libjxl\cjxl.exe "%i" --lossless_jpeg=1 --brotli_effort=11 --quiet -d 0 -e 9 "%~dpi%~ni.jxl" & if exist "%~dpi%~ni.jxl" (echo SUCCESS "%~dpi%~ni.jxl" & del /f /q "%i") else (echo FAIL "%~dpi%~ni.jxl"))`

Is python not an option for you? I mean - only bat file, or python is ok too?

Perhaps this option is also suitable. However, I do not know this programming language. I need the simplest and most convenient way so that I don't have to modify the script file every time. At the moment, I open a command prompt in the desired folder and copy this script there, after which I just press Enter.

I mean I can write it if you tell me what it should do

Oh thanks. 1. Search for all files with the JPG extension, starting from the current folder from which the script is launched and in all subfolders. 2. When a file is found, check if a file with the same name and extension JXL already exists. If it exists, then it should be skipped. 3. Otherwise, need to try to convert. 4. One more check if there is a file with this name and JXL extension. This is necessary to check if the conversion was successful. If yes, show the inscription "SUCCESS", as well as the path to the file and the name. The original file must be deleted (preferably in the Recycle Bin), even despite the "read-only" attribute. (By the way, the AI was unable to advise how to remove to Recycle Bin). 5. If not, show the inscription "FAIL", as well as the path to the file and the name. 6. When the cycle is completed, need to give a sound signal and show a list of files that could not be converted.

I don't write batch files

so I don't recall writing anything like that

```bash while read -r -d '' line; do jxl="${line%.jpg}.jxl" if [ -e "$jxl" ] ; then continue; fi cjxl --lossless_jpeg=1 --brotli_effort=11 --quiet -d 0 -e 9 "$line" "$jxl" if ! [ "$?" = "0" ] ; then printf 'Failure: %s\n' "$line" else cp -a --attributes-only -- "$line" "$jxl" rm -vf -- "$line" fi done <(find . -name '*.jpg' -print0) ```

this is in *bash*

Oops, indeed, I confused you with jonnyawsom3, thanks anyway. Hmm, it looks like this script is for Linux, I'll try to get ChatGPT to convert it to Windows.

Don't do that

Why? Do you suggest installing Linux specifically for this task? But this contradicts the idea of maximally simplifying the entire process.

No, I suggest you don't run code that's generated by a transformer

Ah, this. Of course, I understand and always test on a separate folder first, and also carefully read the description of the commands. I would not have been able to achieve the previous result if I had not been attentive.

If you know what things do just write teh code yourself

if you don't know what things do don't run them

using gpt scripts is sketchy, sometimes ill use it to generate boilerplate, but thats about it

or when you ask a chatgpt how to clean .trash and it gives you `rm -rf /home/$USER`

Tell me, why do you think that the encoder is capable of issuing any return codes? When I and the AI previously tried to check the success of the conversion, it did not issue any codes at all. `if ! [ "$?" = "0" ] ; then printf 'Failure: %s\n' "$line"`

because every process that exits always has a return code

whether or not it exits gracefully

I don't actually care what ChatGPT thinks, since ChatGPT is not a programmer, doesn't think, or understand code.

If you're asking me why I'm disagreeing with ChatGPT a better question is why ChatGPT is disagreeing with a programmer.

Here is the script that uses the return codes, but even if I specifically gave the wrong files (with the JPG extension), it always showed the success of the conversion. `for /R %i in (*.jpg) do (C:\My\Programs\libjxl\cjxl.exe "%i" -d 0 -e 9 "%~dpi%~ni.jxl" & if %errorlevel% == 0 echo Conversion successful! & if not %errorlevel% == 0 echo Conversion failed!)`

I have no idea how this works in Batch

well, I can gander a guess

You're not using *else* you're just issuing another if command

so if I had to guess, `if %errorlevel == 0` is its own command

and the comparison succeeded, even if `%errorlevel%` was nonzero.

since the test condition was false, but the == check itself succeeded.

So when you check it again immediately, it'll be 0, even if it was nonzero from cjxl.

this is just a guess, as again I'm unfamiliar with Batch

you can use bash on windows, if you install either git or msys2

To delete to RecycleBin you need to *pip install send2trash*, otherwise it will delete jpgs permanently. Please test thoroughly before launching on real files

Many thanks to everyone for the help.

I stumbled upon https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8 again which encodes with -I 1 and -E 3 but looking at the --help -v -v -v (etc.) I couldn't find the meaning of these short options

-I 1 is now -I 100 and means 100% of the pixels is used for MA tree training (default is 50%, using more means using more memory and time but should give better results)

-E 3 means using up to 3 previous channels in the MA tree properties, so e.g. for RGBA, the A channels can use R,G,B as context

Thanks!

You have to use four `-v`s to see the useful options.

indeed ! I only used 3, my bad

the one-liner i use to convert files. with parallel, all your cores are used `parallel ffmpeg -i {1} {1.}.{2} ::: *.png *.jpg *.jpeg ::: jxl`

the filetypes are arbitrary, this turns the png and jpg files in a directory into jxl

This won't losslessly transcode jpg to jxl

it's going to decode them to pixels and re-encode the pixels to lossy jxl

it's better to use cjxl for this as it automatically will work the way you want it to here

thanks

im not sure how ffmpeg sets the parameters by default

you can view the defaults by doing `ffmpeg -h encoder=libjxl`

although FFmpeg cannot losslessly transcode jpeg->jxl due to the nature of the pipeline

you'd need a bitstream filter for that, and that hasn't been added (and likely will never be added)

To be more clear, in order to losslessly recompress, cjxl has to have the actual JPEG file it's working with FFmpeg whan handling images decodes the file to pixels and passes that to the encoder (cjxl). cjxl is missing vital bitstream information to add in the reconstruction information

Yes I know this

Accidentally pinged, was clarifying for []

FLIF is better for NPR then JXL effort 7? I wonder how it would work on images from derpibooru.

NPR?

non-photo realistic, I assume

Art and FanArt categories

yes, but JXL -e 9 -E 3 -I 100 still perform better than FLIF most of the time, if you want "encode once, serve many" type of archive, jpegxl rocks (and it is supported by safari :D)

I wonder how much these results change after the encoder tweaks that have been done since then.

I hope it only gets better everywhere

do note that synthetic images that are designed to look realistic, like art that's got good lighting, are closer to photorealistic than nonphotorealistic

Yeah, synthetic images are more based on content than purely being artificial

FLIF is slower to encode, slower to decode and not parallel -- shouldn't be compared with jpeg xl effort 7

FLIF compared to effort 9. Or 8.

still it should be slower to decode, and not parallel -- if made to tiles, it will compress less

there are also some differences like how it detects/orders palettes or how it learns MA trees where the FLIF methods are different from the libjxl methods — and for some types of images maybe some of those encoder decisions are still better in FLIF

we should probably port the palette ordering methods from lossless webp to jxl, I suspect they are the best

for MA tree learning there are still a lot of things to try

could the in-place butteraugli be used for butteraugli_main? i don't think it needs the original images in memory for anything after getting the score

although that would be useful for losslessly converting motion jpeg to motion jpeg xl

Not really, motion jxl has basically no use

Medical archives maybe...

I thought they meant animated JXL already fills that gap, although not being able to seek is an issue

There is a seek option if you use the optional frame index box. But it's still an intra only codec of course...

People use infra only codecs like FFV1, so I don't see why MJXL wouldn't have any use.

im curious as to whether or not there is a place for jxl in hardware recording things, it could be interesting I suppose, better then raw yuv422 capture anyway

as for a mezzanine I think jxl has a lot of potential there

I believe Jon has mentioned that a company is developing a JXL HW encoder

Yeah, it will not be for anytime soon though, but yes this is one of the topics in the JPEG XL adhoc group within JPEG.

Writing Netpbm formats by djxl was recently improved but it's still somehow quirky. Specifically, the particular file extensions must be used for certain pixel formats. PPM accepts only RGB images, grayscale images are rejected and PGM must be used instead. PNM extension is not supported by djxl although that would cover both RGB and grayscale images. PAM accepts only images with alpha, images without alpha are rejected. That might be a bug because PAM can support both RGB and grayscale, with alpha or without alpha.

Netpbm formats are frankly trash outside of legacy use cases.

They are the closest to raw pixels. As such they have their use. Once implemented in djxl, Netpbm formats should work reliably, not requiring trying all possible file extensions.

Zoltan created libjxltiny for this purpose -- to make it easier for hardware encoders. I'd recommend HW encoders to follow the approach implemented in libjxltiny.

FLIF is still slower to decode than JPEG XL at -e 9 (or any), more dynamic probabilities/context modeling

file an issue to libjxl ?

Can you use `.pnm` ?

djxl does not accept .pnm >> can't decode to the file extension '.pnm' Attempting to decode RGB image to .pam gives a different error >> Decoded to pixels. >> Encode failed

Maybe open an issue for this, I think it would make sense to have .pnm as an output option for both color and grayscale. And also .pam should always work, not just when there is alpha...

Alright

.pfm works for both color and grayscale, so .pnm should work as well.

Exactly.

Tbh we should add Luca's fast PNG encoder and use that for djxl PNG output — then there is no real reason anymore to use pnm imo...

PNM having no colorspace info makes it not that practical imo

I won't object 😛

Hello - I'm interested in updating the ruby FastImage (https://github.com/sdsykes/fastimage) gem to support JXL. I can match the first few bytes ( "\xFF\x0A" )to detect the image type, but I was thinking I'd ask how you'd go about plucking out the width and height from a binary stream? Easy or hard?

Also, can someone point me to a JXL file in the ISOBMFF format? I see that FastImage can identify AVIF and HEIC files with that format.

image dimension info comes just after the signature (`ff 0a`), and it's in relatively simple format

(if not, in the simplest format within jxl spec)

this function in libjxl reads `SizeHeader`, which is image dimesion stored in compact format https://github.com/libjxl/libjxl/blob/891c193febca5b5c2aa19b4a9d5dabb5766dc350/lib/jxl/headers.cc#L120

Here's an example that uses the isobmff container format: https://github.com/libjxl/conformance/blob/master/testcases/cafe/input.jxl

Thanks!

I'll have a shot at it, but It's been 20 years since I wrote C!

you basically have to find the `jxlc` box or the first `jxlp` box to find the actual codestream (say if you want to get the image dimensions)

I've got a simple parser that gets the width and height. https://github.com/Fraetor/jxl_decode/blob/main/src/jxl_decode/jxl.py You also need to check for the following bytes, for the case of an image in a container. `0000 000C 4A58 4C20 0D0A 870A`

Can you retrieve the animated frames count this way too?

For frame count you have to decode all frame headers, there is no count in the image header (to make it possible to create bitstreams where the number of frames is not known in advance)

At least can you detect easily if the image is animated in advance?

I don't think it is possible to distinguish between images with multiple zero duration frames (such as those used by patches), and animations without reading the frame headers.

No you can distinguish that: animations will have timing info in the image header

While composite-still images with multiple layers will have have_animation false in the image header, so no timing info gets signaled

I stand corrected.

At least being able to check `have_animation` in advance is a good thing, because it would be bad to start counting images just to check for animated ones.

Looking at the spec I think you should be able to tell within at worse 23 bytes of codestream.

Found a JPEG XL HDR sample image. * Web page containing multiple images: https://people.csail.mit.edu/ericchan/hdr/hdr-jxl.php * Image in question: https://people.csail.mit.edu/ericchan/hdr/jxl_images/20140606_102418_IMGP0297.jxl So far I've tested it on a non-HDR-cabable Windows 11 system. ## Seems to work * PhotoQt v3.4 * Thorium browser M117.0.5938.157 ( https://github.com/Alex313031/Thorium-Win-AVX2/releases/tag/M117.0.5938.157 ) ## Dark and dull (I've sent feedback about this to appropriate places) * ImageGlass v8.10.9.27 * IrfanView v4.62 with plugins v4.62 * JPEGView v1.2.45.0 * Paint.NET v5.0.11 with pdn-jpegxl v1.0.3 * XnViewMP v1.6.1 ## Questions 1. The sample image seems legit/correct, nothing funky going on with it? 2. Thorium seems to handle it correctly, right? Attached is a screenshot of Thorium vs. JPEGView. 3. Do we know of a Windows compatible image viewer with known good JPEG XL HDR decoding?

btw paint.net will not do any color management (by choice)

isn't paint.net supposed to be an image editor? that seems like a bold choice

i think it's an edit pixels vs edit photo angle

like old mspaint

I wish people would stop thinking that images can be displayed without colorspaces...

Krita

nomacs

Speaking of Irfanview. I like this program, I'm using it all the time. It has many limitations due to it's old architecture, but if you know how to work around these limitations, it's still very useful program. It can do color management but it's also limited. The interesting quirk of IrfanView is that it's using Windows bitmap format to do all image processing. Upon loading, any image format is converted to color unmanaged Windows bitmap, either 24 bpp or paletted. It even preserves junk data in padding bytes inside BMP files.

Here's what I get with Mercury, vs with Thorium

first thing to mention: There's no "correct" way to render an HDR image on an SDR display

you *have* to make some compromise

there are incorrect ways, but nothing is "correct"

do note: if you see something very dark like this, that's because this is what libjxl produces

libjxl does no peak detection

which is necessary for viewing HDR on SDR monitors most of the time. It's something that libjxl relies on clients to do themselves

other things of note

if you ask libjxl to output PQ images, it will work fine with color-managed software

`djxl 20140606_102418_IMGP0297.jxl --color_space=RGB_D65_202_Rel_PeQ djxl.png`