JPEG XL

test fonts, for devs of jpeg xl

D 1 s 9 e 2 1.443 gaborish 0 dots 1 epf 1 faster decoding 4 use new heuristics 22082021 2 build

Is quality the only important thing

Jxl will evolve

Will become more affordable

And worthy to reduce x4 x5 the sizes of screenshots compared to png

People that dont like it will be the same

People need to have patience

The encoder still is in Early ISO stage

ISO has nothing to do with encoders.

Standards only define what a decoder does, for encoders the only spec is "produce something that decodes".

Now intensity target 872 s 6 use new heuristics d 0.213

Produce better quality than before

So HDR is starting to make sense

It already made sense before. Just default settings should be fine, if you give the input image with a PQ or HLG color profile.

-s 2 -d 0.79 -I 0.49 --use_new_heuristics --faster_decoding=2

benchmarks on five images

359 KB (368.175 byte)

old s9 322 KB (329.762 byte)

old s7 336 KB (344.438 byte)

for %i in (D:\Images\INT2\Insta\2021d\*.jpg) do cjxl -j -s 3 -d 3.5 -I 0.549 --use_new_heuristics --faster_decoding=3 --gaborish=0 %i %i.jxl

For a simple case with lossless compression, is there any way of beating PNG at compressing this QR code?

That PNG is 670 bytes. `cjxl -q 0 -e 9` gives 4,547 bytes. I assume PNG is winning here because it is indexed.

``-d 0 -E 3 -g 2 -I 1 -e 9 --patches=0`` 637 bytes

can probably beat with less convoluted command but i just tried stuff to get lower slowly 👍

``--patches=0 --override_bitdepth=1 -d 0 -e 9 -g 2`` this gets 610 bytes (why does override bitdepth matter here?)

``` PS > cjxl qr.png qr.jxl -m -e 9 -g 2 --patches=0 PS > cwebp -m 6 -mt -z 9 -lossless -o qr.webp qr.png Name Length ---- ------ qr.png 670 qr.jxl 624 qr.webp 550 ```

this kind of images is a best-case scenario for lz77

`-m -g 2 -I 0 -e 9 -P 0 --override_bitdepth 1 --patches=0`

is 540 bytes

png has the advantage of doing bit packing: for 1-bit images, it stuffs 8 pixels in a byte and gives that to lz77

it matters because the png input only does 8-bit or 16-bit, it doesn't do 1-bit. So it passes the image as an 8-bit image to the encoder, who will then use channel palette to encode it effectively as 1-bit, but it takes some extra bytes to signal the channel palette

So, some kind of automatic bit depth reduction would be useful and as I said before something similar for lossy palette, but for grayscale and b/w images

Also, is it possible to select a limited set of predictors, not just the only one, or combinations like 13, 14, 15 already defined in the options? And it would be interesting to find some optimal encoding options for lossless grayscale manga content that would be more often efficient than png and webp, but not as slow as for general content like `-s 9 -E 3 -I 1`

The complication there is that strictly speaking bit depth reduction is never really lossless, e.g. if you have a 3-bit image, the pixel values are 0, 1/7, 2/7, 3/7, 4/7, 5/7, 6/7, 1, but you cannot express that exactly in 8-bit where values have a denominator of 255.

E.g. it is common to encode 10-bit or 12-bit images as 16-bit PNG, but strictly speaking there's a difference between the 16-bit image and the 10/12-bit image you get by ignoring the padding bits and treating it as 10/12-bit data

😱

I always wondered why 8 Bit --> 16 --> 8 wasn't lossless in GIMP. But this seems kinda unavoidable when wanting exactly 0 as well as exactly 1. <@!794205442175402004> This also means however, that libjxl will treat 2 and 4 Bit grayscale PNGs wrongly?

libjxl does not treat png files

cjxl does read them as 8-bit, yes

I think

Should probably change that

for what it’s worth, if you *255/15 or *255/3 the values, round them, then do the inverse and round them again, they seem to roundtrip properly

Well 255 is divisible by 15 and by 3

So 2-bit and 4-bit are fine

And 1-bit

3-bit isn't though

Also 1023 and 4095 do not divide 65535, so 10 or 12 bit does not exactly map to 16 bit

Fun fact btw: you cannot represent exactly 50% gray in any bitdepth

Since 2^n - 1 is always odd

Always something annoyingly unsatisfying about having to choose between 127 and 128

Oh well, at least in jxl you can get real 50% by using lossless float.

sorry, I meant 7 (2³ − 1), not 3

so it does seem to work for 3 bits

also 10 and 12 for 16

of course, you have to actually do the proper scaling in both directions, not just truncate

```pycon >>> all(k == round((2**3 - 1) / (2**8 - 1) * round((2**8 - 1) / (2**3 - 1) * k)) for k in range(2**3)) True ```

ugh so many thinkos in there

but I think I’ve got it at last

(long day)

Good to know that 12 to 16 to 12 rounds back correctly

But it's still not technically the same to have 16/65535 instead of 1/4095.

<@!794205442175402004> Is there any bit depth format that goes from -128/65280 to 65407/65280 or -2/1020 to 1021/1020 or -8/4080 to 4087/4080 with negative numbers behaving like full black and values above 1 as white? This would waste a minor bit of precision range but conversions from 8 Bit to 10 Bit to 12 Bit and then to 16 Bit would remain perfectly accurate.

Not that I know of

I suppose there can be metadata saying what is the black level, or something

-q 93.182 -s 6 -I 1.221 --patches=0 --epf=2 --gaborish=0 --intensity_target=255 --photon_noise=ISO4917 --faster_decoding=4 --dots=1

like seriously i have watched a link posted in other codecs

https://discord.com/channels/794206087879852103/805176455658733570/889861035958366208

and this remind me of jpeg xl

it looks like a better jpeg xl

the concept of patches is not new i know

but is impressive

how they can get right balance of quality and fidelity

like is not incredible or fine in terms of image fidelity

it tries to be good at all miniscule difference

from 0.588 0.600 bpp to 0.384 bpp and even lower if there is right

is not that simple to know if an image is worth it

is optimized for 400 nits display

for all

it should look very similar in many devices

not simple

have decent speeds

from a 1.200 bpp an 1.800 bpp image and this

simplying the delta

is not that simple

while good fidelity

simplying the pixels

quantization and squeeze

noise

saliency based compression

detail

<@456226577798135808> this is the point of lossy, i don't think wb make encoders just to encode some images at q 95 and some in lossless. many users in jxl or av1 server do that but that's not what i want

i want to lose colour

to me a 19:9 photo that weight 854 kb is perfect to post in the internet

for manga though is not fb with 4 mb limitation per image so lossless original is better

do not know all people say lossless original is always more than 40% better

is really true or is it stupid?

i do not have the experience for that

and neither the abilities

Sorry for bad English

<:linusretire:707375442226708542>

Are the comments encoding parameters? <:HaDog:805390049033191445>

What else can I tune up to more reduce the file size of such an image? ``` cjxl -m -e 9 -P 0 -Y 0 -I 1 -g 3 --palette=8 Compressed to 583578 bytes (0.191 bpp) ```

Can try --patches 0 and/or -I 0

`--patches=0` has no effect. `-I 0` increases the size. `-g 2` slightly reduces the size.

``` cjxl -m -e 9 -P 0 -Y 0 -I 1 -g 2 --palette=8 Compressed to 583411 bytes (0.191 bpp) ```

Does --palette=8 do anything?

Can try --palette=-8 too, iirc

Negative number for palette means it makes the palette in (sub)image order, not sorted lexicographically (i.e. by luma in case of YCoCg) which is the default behavior.

`magick identify` reports that the image has 8 colors, so I tried `--palette=8` and that reduced the size a bit. `--palette=-8` slightly increase the size.

What's interesting, in this case `cjxl` with the default settings easily beats the tuned up `cwebp`: ``` cjxl -m 0.gif 1.jxl Compressed to 738165 bytes (0.242 bpp) cwebp -mt -m 6 -lossless -z 9 -o 2.webp 0.gif Output: 1203588 bytes (0.39 bpp) ```

lossless is a weird game

Nausica!

An updated settings guide: `avifenc -s X -j X --min 0 --max 63 -a end-usage=q -a cq-level=XX -a color:sharpness=2 -a tune=butteraugli -a color:enable-chroma-deltaq=1 -a color:enable-qm=1 -a color:deltaq-mode=4`

Also, if you're on the absolute bleeding edge(like, unofficial patched aomenc...) you can try this thing out for 10bpc: `avifenc -s X -j X -d 10 --min 0 --max 63 -a end-usage=q -a cq-level=XX -a color:sharpness=2 -a tune=image_perceptual_quality -a color:enable-chroma-deltaq=1 -a color:enable-qm=1 -a color:deltaq-mode=4`

Thank you very much BlueSwordM 🙂

no enable_qm? 😛

Oh.

do you know what is the "image_perceptual_quality" tuning mode exactly?

by default I'd assume it's the mode that was added somewhere around end of June

but I'm not sure it is

No, it's very very new. In fact, it's a WIP patch. That is why I mentioned "unofficial patched aomenc".

link to the commit? 😄

I don't understand it 100% , so you are better off reading the commit itself directly so I don't make any mistakes: https://aomedia-review.googlesource.com/c/aom/+/147022 It's actually similar to `--tune=ssim` in the way it calculates and redistributes bits, but without hurting color performance or flat shading(animated).

deltaq-mode=4 <- why are you not trying that? 😛

What do you mean Veluca?

huh

either I'm blind or you changed it and it used to be 2

(probably I am blind)

2 is currently borked and a WIP, so never touched that 😛

does enable_qm not work with this patch?

No, I just forgot to enable it in the 1st place.

ok 😄

admittedly the cli options are nontrivial

In fact, in my original "guide", I forgot to enable the default quantization matrices in the 1st place.

from what I can see of the patch it's probably a nice improvement, although admittedly I'd say the perceptual optimization still leaves a bit to be desired

Butteraugli is still better indeed as an RD tune, but it doesn't work in 10-12bpc, so yeah...

It's our best option for now 🙂

tbh from what I understand of how butteraugli optimization is implemented, it's not great either

(better than function-of-variance, but still there's some corners being cut)

Indeed. I don't get why some AOM members don't want JXL to be really a thing.

AV1 encoders has already benefited nicely from working with the JXL folks 🙂

We just need JXL to become an AOM standard, and we are all set.

ah that would be nice! 😄

one would need to show that it solves a problem that avif doesn't solve, I think

but I would be happy if that happened

Me too

We could even define a new superformat, .jaxif or whatever, which would be compact headers with a jxl or av1 payload

Btw, how compatible are the AOM rules with JPEG/ISO or is the format just enough to be open and royalty-free?

No idea tbh

But if ISO is a problem, we can do it like so many other standards do it (e.g. Unicode): let ISO just be a mirror

I'd expect the difficulties to be mostly with AOM rather than with ISO for this

do you have a good idea of what's the gap between best-avif and jxl btw? 😄

(since you certainly explored more encode options than most people...)

Lossy: It's now actually decently competitive at all qualities, even at higher bpp. 8b performance has been improved, but 8b YUV being what it is, still is a problem in some images. 10b is where it's at, at a cost to encode speed. Lossless: Absolute demolition by JPEG-XL. Never use AV1 lossless for images. JPEG compression: JXL obviously wins, but now, if you have libyuv installed on your system/binary is built with avifenc, you get no generational chroma subsampling loss from aomenc, which is good. Speed: At S6 and above with JXL, aomenc is sort of competitive on average in terms of CPU time. At S5 and below however, all open AV1 encoders get absolutely destroyed. Memory use: Now that's where aomenc has improved massively with lots of recent work: its memory usage in intra-only/all intra is now much smaller than before, and in most cases, is superior than JXL. Decode: dav1d is very fast, usually on par with DJXL per CPU-time, and scales quite well.

I'm not surprised by memory use - JXL uses floats, avif doesn't

I was mostly wondering about lossy 😄 what does "decently competitive" mean?

It means that in images where aomenc had trouble in at higher qualities, it means aomenc can now get decently close to what cjxl produces.

no I get that, but if you had to give a % to "decently close", what would that % be? 1%? 5%? 10%? 25%?

No objective evidence for now. I'm still gathering a much wider image corpus for my article and as such, I haven't tested anything objectively speaking 😛 I've also modified my analysis metrics: 👀, single frame VMAF, butteraugli, SSIMULACRA, and MS-SSIM just in case.

Any more metrics that I could use?

fair enough!

do you have a "gut feeling" number?

you could also use dssim, I think (assuming you're OK with AGPL)

also, is it max-butteraugli or 3-norm-butteraugli?

Do you want some images for your corpus ? I have some good fresh photos (Assume CC0)

If I would give CJXL an output score of 4.5/5, I'd give old aomenc 3.5/5, and current aomenc 4.0-4.2(per image performance is still a problem...). As for butteraugli, I'll be posting both scores, since a higher average picture quality is important, but having higher low-percentile quality is even more so.

so it cut the gap in half in some sense 😄 ok, thanks for your thoughts!

Did you know that libavif decoder is fastest in 8bit when it use libyuv (and of course with dav1d too)? Many people use libavif without libyuv so they use slow YUV->RGB conversion. On my system, 8bit AVIF decodes approximately 30% faster than similar JXL image.

How is 10bit?

I don't have recent data. However in the past libaom decoded 10bit AV1 faster than dav1d. YUV decoding is probably still quite fast, it depends if quality or fast conversion to RGB is used.

8-bit ycbcr has about 4 million different colors

8-bit rgb has 16 million colors

from what I can see on openbenchmarking, the difference is about 20%

(on x86)

and similar on arm, perhaps a bit more

although which arm cpu it actually is is kinda hard to say 😛

either way 4-core A72 decode seems to be able to do ~60MP/s or thereabouts with 10-bit, which is I think rather close to what djxl does (IIRC on a samsung a51 it is ~20MP/s without filters on one core, and 10~15MP/s with filters)

PSNR

3-norm is more of optimization tactics (smoother surface for optimization algorithms) and 6-norm would be a better default for humans

What range is all qualities?

WebP lossless has great single frame compression. The animation however doesn't use the same compression engine, it was added by other people in a way that consider WebP lossy and lossless techniques a black box.

As a consequence, WebP lossless wins ~30 % over PNG, but often slightly loses against APNG.

JPEG XL has a lot more expressive methods for using the previous frame(s) in animations, and we haven't spend much time figuring out how to use them in encoding

If we think of high quality pixels vs. high quality photography compression, I consider the delta palettization a 'dark horse' for offering a 50 % improvement for classic palette reduction approach that makes GIF, PNG and WebP lossless practical in many cases

Delta palettization has not been implemented in other formats, it is a unique value brought by JPEG XL. Its encoder in JPEG XL is somewhat experimental still (single speed, and slower than what is practical).

More guarantees on degradation in quality can be given for pixel based compression (PNG, WebP lossless, JXL lossless, JXL delta palettization) than for block based (JPEG, JXL, AVIF, WebP lossy) compression.

when comparing butteraugli, some people have added the opensourced butteraugli from github.com/google/butteraugli that had least compromises to assist compression, some have even had their favorite release on that (often the first one)

the butterauglis that are integrated in JPEG and JPEG XL compression (guetzli and libjxl) have adjustments that make them perform slightly better in that context, and JPEG XL's butteraugli includes changes that make it perform better when used as a differential version in neural compression (that make both image quality use and JPEG XL use slightly worse)

In my case, the image isn't animated, it's a static gif made from a grayscale scan. 😄

https://discord.com/channels/794206087879852103/840831132009365514/892277140895182859

I'm used to lossless webp always being good, but in this particular case it is noticeably losing to jxl.

This loss can be explained by lack of pixel-based context modeling in WebP

in WebP the entropy code is chosen by a subresolution image and is the same for a say, 16x16 or 32x32 area

in JPEG XL the entropy code can depend on the neighboring pixels

for example if T and L pixels are white, then use one entropy code, if T and L are black then another, otherwise a third

when there are very high pixel-to-pixel possible decorrelations like such high entropy line drawings and some fractals, then the pixel based context is far more dense

but it is always necessarily slower to decode

Do I understand correct that some flags are dropped in latest version like choosing threads number ?

Irfanview has a jpeg-xl plugin: https://www.irfanview.com/plugins.htm

<@!285240447922733056> No flags dropped between v6 and current. According to help, cjxl accepts "PNG, APNG, GIF, JPEG, EXR, PPM, PFM, or PGX" (it can also accept PSD) and djxl outputs "PNG with ICC, JPG, or PPM/PFM" (it can also output APNG).

For testing lossless, you can do `compare -metric pae <original> <decoded> null:`

although that will return non-zero for (almost all) jpeg reconstructed images

if "count pic colors" means counting palette size, that seems... unreliable at best

For testing jpeg recompression, you can just reconstruct the jpeg and use `cmp` to check that it's the exact same file

<@!794205442175402004>for now I stick to packjpg cause it`s smaller, small difference but still (in my sample image 2 234 kb vs jxl 2 377 kb) wish it will change in the future

<@!263300458888691714>in latest version I don`t see multi thread option in-built in help so my question arose

<@!411361798696992798>thanx

`--num_threads` is hidden behind one level of verbosity (`cjxl -h -v`)

But that was the same as before

ok thanx, missed

Hi, any way to losslessly convert webp to jxl?

Lossy webp? I doubt it.

lossless webp is same picture as png

but if you are talking about lossy webp, then no, it's impossible, the algorithms are too different

Well vp8 is doing DCT, in block sizes that are probably a subset of what jxl can do. But jxl doesn't have directional prediction, so that's a problem. Also webp always uses tv range YCbCr while jxl can only do full range (like it is in jpeg)

Ah and jxl doesn't actually support variable block sizes when it's 4:2:0

Yeah no, it's not possible

This is interesting, would it be possible to losslessly recompress (doesn't need to be reversible) other JPEG "X" flavors?

I'm talking about JPEG XR, XS, XT, if anybody actually use them.

Only lossless for the original JPEG.

At least with the existing encoder.

In principle other lossless format conversion could be done if the internal structure of the codecs are similar enough, but that would require significant development effort.

XT is basically old jpeg plus extra stuff, the old jpeg can be done with jxl but the extra stuff I dunno

XR and 2000 use different transforms, they cannot be represented in jxl

XS is not really meant as a file format so I don't think it would be useful. But also probably cannot be done.

These formats don't seem to be popular very much. I discovered them only while I was researching about JXL. So the actual question was if these formats would "fit" into the JXL codestream. JPEG2000 seem to be somewhat popular, but as far as I know it doesn't use DCT, so that wouldn't work at all.

One of the use cases for j2k is lossless for medical, that of course can be converted to jxl losslessly

That's right. Lossless formats are not an issue. Lossless formats can be converted freely.

The only way to effectively do lossless recompression of a lossy codec, is to be a superset of it (as far as the lossy coding tools go, the bitstream organization, entropy coding, and lossless parts can be different).

For jpeg, it made sense to do that, because jpeg is relatively simple, its coding tools are very battle-tested and known to be useful, and there are a huge amount of existing jpeg files out there.

For any other lossy codec, it makes a lot less sense, because it's more spec complication for less practical value.

doesn't it depend on what exactly you're missing? like, if your codec is much much better but doesn't support some feature, then you can code it in a "residual error" block

the problem with this theory i think is that image compression hasn't gotten *that* much better over the last decade

If lossless pixel values is all you need, then "residual error" works, but it's likely not going to be more effective than just encoding the decoding image losslessly (so will end up larger than the other format, probably)

mmhmm

Hello, I just started using JPEG XL. Trying to compress some uint16 image with JPEG XL lossless. I could achieve lossless compression with cjxl. But I can not achieve lossless compression with interface used in encode_oneshot example. I tried all the lossless frame controls, it didn't work for me. It seems that cjxl_ng in progress uses same interface as used in encode_oneshot. But cjxl_ng crashed on my single channel 4001x4001 uint16 image. Any suggestions? Thanks a lot first.

The lossless frame settings that I have used so far.

float distance = 0.0; // mathematically lossless if (JXL_ENC_SUCCESS != JxlEncoderSetFrameDistance(frame_settings, distance)) { fprintf(stderr, "JxlEncoderSetFrameDistance failed\n"); return false; } if (JXL_ENC_SUCCESS != JxlEncoderFrameSettingsSetOption(frame_settings, JXL_ENC_FRAME_SETTING_EFFORT, 7)) { fprintf(stderr, "JxlEncoderFrameSettingsSetOption JXL_ENC_FRAME_SETTING_EFFORT failed\n"); return false; } if (JXL_ENC_SUCCESS != JxlEncoderFrameSettingsSetOption(frame_settings, JXL_ENC_FRAME_SETTING_MODULAR,1)) { fprintf(stderr, "JxlEncoderFrameSettingsSetOption JXL_ENC_FRAME_SETTING_MODULAR failed\n"); return false; } if (JXL_ENC_SUCCESS != JxlEncoderFrameSettingsSetOption(frame_settings, JXL_ENC_FRAME_SETTING_KEEP_INVISIBLE, 1)) { fprintf(stderr, "JxlEncoderFrameSettingsSetOption JXL_ENC_FRAME_SETTING_KEEP_INVISIBLE failed\n"); return false; } if (JXL_ENC_SUCCESS != JxlEncoderSetFrameLossless(frame_settings, JXL_TRUE)) { fprintf(stderr, "JxlEncoderSetFrameLossless failed\n"); return false; }

You have to set `basic_info.uses_original_profile = JXL_TRUE`

Otherwise it converts to XYB, which is not lossless and it's also a bad idea for compression

This is such a common api usage pitfall that I feel like changing JxlEncoderSetFrameLossless to fail if the basic info is not compatible with lossless

It works perfectly now. Thanks a lot. It bothered me for almost two days.

are there any suggested parameters for lossy encoding of lined, flat-colored digital art? i.e. there's a small color palette and all color transitions are sharp, especially along edges

unless i drop the quality level *really* low, lossy encoding does way worse than lossless in terms of file size, and of course in terms of quality

<@!461421345302118401> has been studying drawing/manga content encoding, but targeting distances 1 to 0.5. If lossless just performs much better, then I'm not sure current JXL parameter tweaking will be enough. Based on description, smart lossy palette+dither might be an option (e.g. pngquant). Maybe Lee has some insights anyway.

thanks! looking forward to their insights, if they have any, then 😛

In my opinion, I don't recommend apply current jxl lossy(vardct) for non-photo content, for some type drawing content, vardct work very well on `-d 0.5~1.0 -e 8`, but for some type drawing content, I think still need wait next quality and features improvement. (non-photo quality improve and auto palette features) For now you can try different lossy mode in jxl, lossy Modular(XYB, Haar like(squeeze)transform, PSNR), lossy palette(--lossy-palette, still very experiment). but I guess the best practices is wait jxl complete next quality and features improvement for non-photo. lossy palette + dither(pngquant) also a good method, but sometime will happen very noticeable error. (you also can try av1 palette prediction, but I can't talk this too much in this channel.)

sorry for the delayed reply, but I wanted to say thanks!

that's a very helpful reply and in a lot more depth than I could've asked for, so I really appreciate it

As expected from Lee 🙂

I open a new libjxl issues about lossy palette(delta palette), I think if can switch varDCT mode and lossy palette mode for different non-photo content, that will be very great.

are these correct params for lossless convertion of jpg to jxl? "-d 0.0 -e 9 "

Yes.

Strange but Irfanview shows result file have doubled number of unique colors in picture, plugin bug?

cjxl does lossless JPG conversion by default, you only need `-d 0` to get lossless encoding of PNG (or such) input

No, just different results you get with different jpeg decoders (libjxl and I assume libjpeg-turbo). The jpeg format allows some decoder freedom so it is not guaranteed to produce identical pixels when different decoders are used.

<@794205442175402004> I wonder if that is possible with Cloudinary (and if you know if that is possible): Assume a lossless source image uploaded to the media library. Can I deliver it as - JXL (q93) if the browser supports it - JPEG (q93) if the image has no transparancy - WebP (q95) otherwise

yes, `f_auto,q_93` would have that behavior (more or less; I'm not sure what cwebp quality q_93 maps to but probably something like 95) but you'd need your f_auto configuration to be set to include jxl and webp but not avif, and we don't have a self-service way of doing f_auto configuration yet, I think, so you'd need to open a ticket with customer support to do that at the moment.

ok, thx! I'll open a ticket once I actually need that. ^^

Is there a maximum distance that can be used? I'm getting an error about an invalidly freed pointer.

I get the same error with any distance of 20 or more.

OK, so it encodes if the effort is 7 or less, and a different image encodes fine at e 9.

Hmm, I think I'll open an issue about this.

The issue is that e>7 and resampling doesn't work, and at very high distance it does 2x resampling by default

Ah

So it is this issue then? https://github.com/libjxl/libjxl/issues/330

Is there any other heuristic other than distance to decide if it does respampling? I've got some images encoding, and some not, with the same settings.

The one that works is much higher resolution though.

Interesting. Could be an indication of where the bug is

I'm seeing if I can produce a simpler image that encodes, then I'll attach it to the bug report.

I guess Cloudinary cannot do something like: if supported use /image/a.jxl else /image/b.jpg as one URL?

it can, that's what f_auto does: send different responses based on accept header and UA

yes, but I mean you cannot manually specify two images like how it is possible with a picture tag, right?

ah, no, f_auto is not meant for that

I think we will soon expose a way to do two (or more) really different images behind the same url, where what you get can depend on request headers including e.g. geolocation estimated from ip

so you can have an image that shows someone with a thick coat when you view it in Alaska, but someone in a swimsuit when you view it in Hawaii

(or some other kinds of funky localization / personalization)

main use case for that is A/B testing where marketeers want to see which image "works best" so one way to do that would be to use such a url that can end up showing different images (e.g. randomly based on ip) and then keep track of conversion metrics for each variant

sounds interesting. Then I hope it will support the accept header. Please ping me if this releases and you still remember.

Will do - if I remember :)

``-e 9 --brotli_effort 11`` is it really the maximum possible power parameters that I can enter?

No

Add -E 3 -I 10 for more

-I is what? I don't see it in the help -v

How much data it looks at for MA tree learning

surprisingly it is not so much slower than I expected, alas, can not mining the picture

does `--brotli_effort` do anything to the codestream or only the brob boxes

Iirc it's brob and jbrd

I have some RGB24 images. If I use cjxl to do lossy compression, it works fine. But if I call JPEG XL api to do the compression, the color changed a little bit with the same distance control. The color controls that I used was from the example encode_oneshot.cc.

JxlColorEncoding color_encoding = {}; JxlColorEncodingSetToSRGB(&color_encoding, /*is_gray=*/pixel_format.num_channels < 3); if (JXL_ENC_SUCCESS != JxlEncoderSetColorEncoding(enc.get(), &color_encoding)) { fprintf(stderr, "JxlEncoderSetColorEncoding failed\n"); return false; }

I also tried the cjxl_ng which was cloned and built a few months ago. cjxl_ng can encode the image without changing color but scale the color to RGB48 (each channel is 16bit).

Any suggestions for the color encoding controls for RGB24 lossy compression? Thanks a lot for any suggestions.

What colorspace are your images in?

Probably not sRGB then

i have problems compressing this image, it has 3 colors, i use -m -g 3 -P 15 --palette 3 but PNG turns out to Size 2 times less

there is any function that I don’t know about that would brute force modular compression

I think you are missing `-d 0`

in fact `-e 9 -d 0` is enough for me to get 5096 bytes (which is not much different from 5089 bytes for `-e 9 -d 0 -m -g 3 -P 15 --palette 3`)

thanks, and i used -I 1 and got 5060 bytes

`-d 0 -e 9 -I 1 -P 0` : 4977B

It turned out to be my own issue. I used opencv to write the RGB24 image extracted from points cloud to be used for cjxl and forgot that opencv imwrite always assumes BGR. There is nothing wrong with JPEG XL encoder control. Sorry for the false alarm.

Yeah opencv assumed bgr

this is one of those use cases where it would be nice to request planar output in any order

so you could request planar GBR or BGR or RGB

You mean interleaved?

no, I mean planar

I figure it would be part of the patchset

In planar you'd probably just get one buffer per plane, order doesn't really matter then

I guess, yea

order wouldn't matter since you'd just get a pointer

-e 9 is maximum for varDHT ? can't I bruteforce the compression? maybe there is some debug function for this?

It looks like more effort in VarDCT improves quality at the same file size.

The encoder always targets the quality/distance you request. With higher effort it can be more accurate. The relationship between effort and file size is not guaranteed.

`-E 3` can also benefit VarDCT mode.

Keep in mind that higher effort also means more *consistent* quality to bpp

that is, you're more likely to achieve the target you desired

when low efforts produce smaller files at the same distance it typically means that the achieved quality is lower than the requested one

👋 for encoding progressive JXL images, do I only need to set the `JXL_ENC_FRAME_SETTING_RESPONSIVE` param to `1`? There are other params `JXL_ENC_FRAME_SETTING_PROGRESSIVE_AC` , `JXL_ENC_FRAME_SETTING_QPROGRESSIVE_AC`.... but I am not sure what they are for! I am a newbie to this and still gaining context, so thanks in advance 🙏 I did notice that when `cjxl --progressive` param is passed, it sets multiple progressive encoding. params https://github.com/libjxl/libjxl/blob/0103e5a901b776c46b55081fd6b647fbdd618472/tools/cjxl_main.cc#L891-L894

technically speaking there is no single "progressive" mode in jxl, it provides a way to chop different bits of data and merge them at the decoding end. those options determine how to chop them.

I guess options set by `--progressive` are fine defaults.

even just the options set by doing nothing at all are not too bad — you always get DC first, which for most use cases I think is actually "progressive enough"

so setting `JXL_ENC_FRAME_SETTING_RESPONSIVE` to `1` should be good defaults then?

uh no, that forces modular to use squeeze

the best default is not setting anything

Is there a good way to view progressive images at different load amounts? I tried just truncating the files then viewing with a gdk-pixbuf viewer, but they only show black with in incomplete progressive file.

(Interestingly a non-progressive file does show the blocks loading sequentially.)

https://github.com/libjxl/libjxl/blob/main/examples/decode_progressive.cc

Thanks.

Should also work in the very latest chrome when you pass a truncated file. The patch for that was merged yesterday

does this have anything to do with whether chrome decided to go with JXL or not?

Afaik that decision still has not been communicated.

so, six months have passed and I want to ask again -- the *-e 9 -E 3* parameter is the maximum possible for VarDCT compression?

For VarDCT -E 3 does not really make a difference

Also -e 9 does not seem to be better than -e 8 for vardct mode, just slower, according to my testing

We don't really have an exhaustive and very slow encoder for jxl, like the slow settings of avif and heic reference software.

0.5 %

Really? I wonder how, that would be just on DC then I guess...

I would like some kind of algorithm bruteforce quality

Then you can also try -I 100, I suppose

I just wanted to do a little test for the forum, so I asked <#805176455658733570>

Bruteforce full exhaustive would be extremely slow, even slower than slowest avif. The bitstream is very expressive so the search space is huge.

how many days per 512x512 image?

Nobody tried making something like that but it could be more than the lifetime of the universe to really find the optimal jxl encoding (according to some metric, obviously) for a single image

it would be very interesting for me to use a search of all the options for the lowest quality option, because for AVIF it really works well at extremely low quality, when the codec can spend tens of minutes searching for the right angle option, guardian, and the like

I just would like to compress some parameter like "--compress much possible" so that there is warmth in the soul

This would be interesting indeed, though maybe not a priority for practical use cases

but it gives false confidence that you are *getting the best you could get*

For lossy, that is kind of inevitable, since the perfect metric to optimize for does not exist

it makes more sense for lossless as it's literally only a file-size encode-time tradeoff

the metric is the file size

for lossless it is not interesting

Lossless is interesting. Now I have to use PNG because I have no choice. Letting PNGOUT chewing on bunch of PNGs overnight is silly.

just use pingo™

same thing applies, letting a png optimizer squeeze the last few bits out of a PNG is kind of wasteful when you could just use lossless modular

is it true that limiting the Modular palette to 64 colors, it will be 6 bits long? or 256 as 8 bits? is there any additional efficiency in the fact that I choose exactly 256 colors and not 258 colors? any overhead?

You don't need palette sizes to be a power of two to make entropy coding efficient. It is not based on bit packing, so the cost of more colors smoothly increases, not in a step function where there's a bigger penalty to expand the range by one bit than to expand it within the same bit width.

Sounds great

Note that libjxl does not perform any color reduction itself when using the regular palette, it only applies it if it can be done in a lossless way. With delta palette it does apply loss, but that kind of palette works differently than a simple gif/png palette (palette entries can be either a color or a set of deltas w.r.t. a predictor, so gradients can be done cheaply without needing dithering or many palette entries).

according to my tests, cheetach -e 3 is now the best choice for compression from an image with a quality of 90-95, mainly due to the lack of complex anti-aliasing filters that spoil and blur the image more at such high qualities, but at the same time there is already effective separation by varDCT Blocks

For pixel art, it's worth trying to increase the number of palette colors, so the whole image is encoded just using indexed color. It may help a lot.

the number of colors is automatically determined by the encoder, based on all the colors in the picture

By default, the number of colors in palette is limited. I think the default limit was 1024.

on the quality of e5, some formulas are beginning to be used to generate color channels, and this is noticeable in some of my examples Not for the better, so on the qualities of 95+ it is better to use e 3, completely without any transformations, but I would not refuse variable blocks... in fact, I will have to start testing Jpeg XL at speed 3 and JPEG XR. it is important.

no limit

There used to be a limit. It made a difference when I tested it.

Anyway, I suppose there has to be some limit. It doesn't make much sense to create a palette of millions of colors.

before is before. the codec is in active development. as far as I know, the lossless compression module is now completely ready. and I'm already using it. but the more massive VarDCT module is still of course.... well, you can use it. But I don't use

the limit depends on how many colors were in the source

Iirc default limit is still 1024. Maybe it was lowered to 512. Bigger palettes can be useful, but the current heuristic just applies the palette when below the limit, so we need a conservative bound to avoid cases where palette is counterproductive.

you are now about VarDCT ?

No, palette is a modular thing

What if I have lossless colors more than 1024?

So I recall it correctly. The default limit is not listed in the help at the moment.

I don't understand how a default limit can exist at all

I suppose other compression strategies are usually more efficient in pictures with more colors.

I tried VarDCT mostly on scanned documents and analog video captures. Near lossless is okay for such content. It's not worth wasting bits in lossless on the noise introduced by analog media.

I used JPEGXR with the same result

now I would most likely use AVIF for documents. because of all its vector-like functions

The JXL images are indistinguishable from the original, so it worked well.

👌

and all such e6 turned out to be the best compromise between speed and quality for me. better than e7 due to the lack of more complex functions that give almost zero result...but the speed of work at e3 is really amazing. that's fast...

To compress this pic, will the ```-q 100 -e 9 -g 3``` command be the most efficient possible? or any hidden keys I missed?

no wait, I found that -g 2 compressed much better

and new bug

Or I didn't understand what it was.... for what reason does it work like this cjxl.exe -m 1 -e 9 JPEG XL encoder v0.8.0 72af6c7 [Unknown] Read 698x365 image, 60449 bytes, 107.6 MP/s Encoding [Modular, d1.000, effort: 9], Compressed to 25337 bytes (0.796 bpp). 698 x 365, 0.12 MP/s [0.12, 0.12], 1 reps, 4 threads. cjxl.exe -q 100 -e 9 JPEG XL encoder v0.8.0 72af6c7 [Unknown] Read 698x365 image, 60449 bytes, 127.5 MP/s Encoding [Modular, lossless, effort: 9], Compressed to 22037 bytes (0.692 bpp). 698 x 365, 0.03 MP/s [0.03, 0.03], 1 reps, 4 threads.

Is there a need to specify group size with -g flag?

The default should be reasonable, sometimes different values give better compression (but bigger group sizes also means less opportunity for parallel decode)

-g 2 in my cases compresses better by 5-10%, this is extremely much

<@794205442175402004>I wanted to ask why -m 1 does not mean -q 100

and I would like to ask - why there is no algorithm for finding the best group size (not bruteforce)

Would it make more sense if modular implied lossless by default?

I haven't found a heuristic yet for that, but also changing it impacts parallelization potential so maybe shouldn't be done automatically...

Of course, that's what it was created for.... in fact, in my particular case, the problem is that it is worse in size than the lossless

okay, for my specific pictures, I have deduced my value -g 2, but in theory, can compress even better if you choose the exact value

on what qualities is recommended to use lossy modular?

in lossy modular mode there are such big artifacts on q 85

I don't know if lossy modular should be recommended at all, tbh.

modular is designed for lossless while vardct is designed for lossy. trying to use lossy modular is like trying to use a png to store an image in a lossy matter

In fact, lossy modular always loses to varDCT on any quality parameter

Lossy modular effectively just optimizes for psnr atm (in xyb space). Vardct has better heuristics, and also dct is just better for photographic images.

I don't think there is any situation where it performs well either, at least in the encoder's current state.

It should never be used until it's improved

It would be nice if there was a lossy mode that did this. Making smart little edits to make the codestream more predictable and compressible.

Right now lossy modular doesn't do anything like that at all. From what I understand, and I don't know that much about the encoder, really... but it seems that it basically does some sort of wavelet-based transform called Squeeze or something that doesn't seem that different in principle from varDCT mode?

looks like blue quant needs to be tuned

I don't think that's the only problem. modular itself just looks disgusting at this quality and below. Much worse than the vardct. In fact, I'm starting to doubt why lossy even exists for modular. But until the vardct has solved all his problems, modular is an option.

Lossy modular is not really designed for perception, at least the current encoder basically just optimizes psnr, the only perceptual thing being that it does that in XYB with channel weights that make some amount of sense.

It exists because it's the only way to do lossy extra channels — vardct is only defined for the 3 color channels.

we don't know the full potential of lossy modular -- it is likely that it will not be that competitive ever in the high loss area -- it's pretty good for lossless and it may have more potential in near-lossless and the delta-palettization approaches

Unfortunately, even at 99 quality, it shows results worse than VARDCT, both in visual and relative size. And only if the task is to leave all the information unchanged, every pixel, do lossless, modular works

Just don't use lossy modular. Pretend it doesn't exist.

you are correct -- that is the current state, VarDCT is our workhorse for lossy 🙂

I want to note that the key -m for activating modular mode encodes exactly in lossy

honestly it should be harder for people to use bad settings like this so that people who don't know what they are doing (commonly referred to as idiots) don't go around creating and distributing poorly encoded images

Maybe at the very least some sort of warning to alert any potential idiots that they are needlessly and preventably degrading image quality.

don't conflate uneducated people with stupid people

someone isn't an idiot cause they don't know how to encode a jxl

That doesn't stop intelligent yet uninformed people from being perceived as and called idiots by others. Likewise nor does it doesn't stop unintelligent people from being perceived as smarter than they really are due to their acquired wisdom.

Intelligence and wisdom are two separate things and intelligence is more subtle and wisdom is more obvious and practical. Unwise people will still be called fools no matter how capable of learning they are.

Those people are not going to use cjxl/djxl. So that's not a problem.

They will.

If you use cjxl, just use the options you see in the default help screen

If you use options hidden behind one or more --verbose --help, it's to be expected that many choices will be worse than default

This is the same in any tool that has advanced settings

I just noticed yesterday while at a friend's house, he was using irfanview, and there was a checkbox "use modular"

For some reason that option is extremely inexplicably attractive to a certain kind of person

Even though it should never be used, CERTAIN people are somehow inexplicably drawn to it

we could perhaps use larger context trees for very small color counts PNG does that through pixel packing, 3 colors would be packed in PNG to 2 bits, i.e., entropy coding would compress a joint-distribution of 4 pixels -- a competing entry would need a context tree of 3 neighbouring pixels

Good news, the modular option was removed in the recent version of IrfanView. It was replaced by progressive encoding.

Haha, wow, I wonder if it's somehow related to how I complain way too much about little things like that...

But that's definitely a nice improvement!

yes my msgs eads bad like gsehewuwebufbuy because i'm not native. anyway i finished conrtibutions. bye all

also wods in talk aent in ode like byefwhfqbu

I discovered that mental brain encoding in some disability is a disability also

https://www.threads.net/@rehabscience/post/C5OH45oruJg/

Effort 9 fast speed start to be good

Vmaf 66 amazingly high

re all , can you suggest command line to convert heic files to visually lossless jxl or avif?

visually lossless is kinda a bad term since it depends on the image and the viewer of said image, I typically encode using `-e 7 -d 0.5 --progressive_dc=1 --brotli_effort=10`

but first you need to decode the heic file to an intermediate, or just use ffmpeg and loose the `progressive_dc` and `brotli_effort`

You can set effort and distance in ffmpeg

With `-effort` and `-distance` surprisingly

Everything depends on how lossy the HEIC files are to begin with. Transcoding in general is tricky. It only makes sense if you pick a quality setting that produces a smaller file and doesn't waste bits on getting the compression artifacts of the first codec exactly replicated. But you probably also don't want to set it too low and introduce too much generation loss.

Since when has this channel existed? I’ve never noticed it

https://kb.ltgc.cc/ltgc/encode.html#lossy-still might prove helpful to you, though I forgot to update with the "fake lossless" portion depends on the input, a distance between `0.1` and `0.5` should work, and modular mode could be applied for lower distance values

the mental deterioration of fab "coincidentally" almost perfectly lines up with his apparent addiction to social media in the av1 server

ppl may find this useful I guess

Lossy webp should NOT be used for archival purposes. TV range 4:2:0 images should be delivery only.

Unless you want to archive screenshots from VHS tapes 😄

on that front it's better just go with near-lossless I just didn't have which word to use for that balance point

though with near lossless WebP is, again, out of the question, even with dedicated near-lossless mode as it's too inefficient

Lossless webp is sometimes better than lossless jxl, especially on screenshots of old games 😅

it excels at stuff with low entropy I guess, I'm observing the same trend with a lot of my screenshots

Lossless WebP gives good and consistent results.

Lossy WebP is lacking fidelity, but if the source material is already in 4:2:0 TV range, lossy WebP actually wouldn't do any harm. It's quite niche use case but it's possible.

Movie screenshots?

Possibly. DVD is using TV range AFAIK. Not sure about Bluray. However it only makes sense if the source material also uses the same conversion matrix as WebP.

It's another thing I feel like I say every week, but I know WebP has a much more advanced palette sorting method which was going to be ported to libjxl at some point I can't remember the exact details, but this is what's currently used https://github.com/libjxl/libjxl/pull/2523 https://github.com/libjxl/libjxl/pull/3420

Since [AllRGB.com](https://AllRGB.com) has been mentioned before, has anyone looked into recreating any of the high-compressed images there using `tree_to_jxl`? _(Reposting from https://discord.com/channels/794206087879852103/824000991891554375/1269696743654686763 since maybe a better fit here?)_

Turns out FileOptimizer doesn't optmize JXL very well... https://www.reddit.com/r/jpegxl/comments/1efrls4/comment/lh6ep6s/

Seems to just set ImageMagik to effort 9 and that's it, not even sure if it's lossless as the optimizer claims

Last time I checked it was lossy. But it has some other issues too lile removing AV1 video streams when optimizing mkv ~~I've been too lazy to open an issue~~

I guess the best file optimizer is `rm` then

Why only delete crucial data when you could just delete the whole file?

mkfs.ext4 for the win

that's the hard drive optimizer/defragmenter

`blkdiscard` is also good if you want it really optimized

Best optimisation tool is obviously the hammer

Javier (FO's dev) is a good guy, & appreciates PRs & issues. Give him a chance & some notice.

How do I encode multiple images with the same file extension that are in a folder in the main directory? What I mean by that is that I create a new folder (let's say "Images"), then I put all my image files (let's say they are all PNG files) into that folder, and I want to use a cycle (for/while/etc.) that enters Images folder, encode the images one by one and each encoded image must have it's original name (but now instead of .png they are .jxl) and saves said encoded files in Images folder. I know it's very specific but if anyone can help me, I would greatly appreciate it. for now I have been using this code in FFMPEG: for %f in (.png) do ffmpeg -i "%f" -c:v libjxl -distance 0.0 "%~nf.jxl" but not only it doesn't work for libjxl, it also reads images that are on the main folder

Thanks for the reply. Yes, I would love to losslessly save some bytes on my HDD in the case of lossy JPEG files.

Something like: `#!/bin/bash # Directory containing the PNG files input_dir="/path/to/your/png/folder" # Loop through all PNG files in the directory for file in "$input_dir"/*.png; do # Extract the file name without the extension filename=$(basename "$file" .png) # Convert the PNG file to JPEG XL cjxl -d 0 -e 9 "$file" "$input_dir/$filename.jxl" echo "Converted $file to $filename.jxl" done echo "All PNG files have been converted to JPEG XL format."`

Save the script to `convert_png_to_jxl.sh` Make it executable: `chmod +x convert_png_to_jxl.sh` Run : `./convert_png_to_jxl.sh`

And change the input folder to where ever your PNGs are

So impressive, thank you very much.

I genuinely appreciate it

`parallel cjxl --num_threads=0 -d 0 -e 9 {} {.}.jxl ::: /dir/*.png` my beloved <:FeelsAmazingMan:808826295768449054>

Personally a find * -iname “*.png” -print0 | xargs-0 -n1 -P$(nproc) cjxl person

I used to, but since I discovered `parallel` the syntax is much more simple