JPEG XL

so probable AVX2 is there, unless N_ means "No"

Yes, it means avx2 is used, so that's good

I can share screen with the profiler, if you like

What kind of speed does djxl report if you do --num_reps=100 --num_threads=1 on that same image?

one sec

If it takes 9s for about 200 mpx, that's only 22 megapixels per second. I would expect a vardct image without alpha to be faster to decode

26.72 MP/s

[23.40, 29.59]

Ok that's the same ballpark. Could you try the image encoded at e6?

one sec

Maybe it's an image where patches are used. That would also slow down decoding. At e6, patches are disabled

24.28 with e6

~25 MP/s decoding on single core when avx2 is available is kind of disappointing, I would expect it to be faster. <@179701849576833024> any idea what could cause this?

btw the source image is 32bit, so maybe it includes alpha? or cjxl accounts for the "unused" alpha?

when there is a meaningful alpha, the speed is around 11 MP/s

If you encode RGBA, it does not automatically strip trivial alpha

It will compress quite well and also decode quite quickly compared to nontrivial alpha, but it will still be there

Try converting the image to ppm first so the alpha gets dropped, and encoding that

(we should really start detecting trivial alpha and auto drop it when doing lossy, it's a waste of resources to keep it around)

43.69

but still 5 times slower than WebP 😦

btw, lossless without alpha = 15.87

Yeah, though with more threads the gap should be smaller

BTW, what makes alpha (even the trivial one) so slow to decode? It literally takes half of the decoding time. You think the lossless alpha + lossy RGB should be faster?

The squeeze transform in modular is a bit slow, and is used for lossy alpha. Also there might be other things that can/should be optimized, we haven't really been trying to optimize the decode speed of images with alpha yet.

unfortunately, cannot try with the lossless alpha yet - will wait until JxlEncoderSetExtraChannelDistance appears in the stable build

Assuming a `libjxl`-based decoder with the progressive detail option set to `kPasses` and a VarDCT image compressed with `--progressive_ac` and `--progressive_dc=1` how many `JXL_DEC_FRAME_PROGRESSION` events should I expect to get? My guess would be four (1:8 DC, 1:4 AC, rough 1:2 AC and non-rough 1:2 AC), but I am only getting three...

Hm, could be a bug... What do you get with kLastPasses?

With `kLastPasses` I also get three `JXL_DEC_FRAME_PROGRESSION` events. If I flush the image buffer when those events occur both `kPasses` and `KLastPasses` give me the same output buffer content for every event. With `kDC` I get one event (as expected).

Looks like we aren't actually distinguishing kLastPasses and kPasses then or something. Could you open an issue?

Yes, I can.

I am sorry it took me so long, but here it is: https://github.com/libjxl/libjxl/issues/2264.

No rush, it's weekend anyway 🙂

What is the quality loss if I set FasterDecoding to 4? Does it corellate somehow with the Distance? Like if I set Distance to 1.0, but then faster decoding to 4, will I get the same quality but less compression?

Yes, I think it just hurts compression, not quality. Not sure though. <@179701849576833024> ?

At high quality, it doesn't really affect quality much, but it does disable filters (gaborish and epf) at the higher values of faster decoding

Well, I see two times faster decoding for 4 in exchange for about 10-20% file size - this is with complex alpha. If it does not hurt quality, then it can be a good tradeoff.

No, unfortunately on the bigger set the statistics is much worse - Faster Decoding 4 gives approx 2.5 times increase of the file size on my data, while lower values don't offer a substantial decoding speed increase.

I started something... an encoder oriented FAQ: https://docs.google.com/document/d/18MBqu3o2yRpWz4GUxsxTLzcIR2F6WLfpDa-sVWRyG9w/edit?usp=sharing It is more drafty than expected though 😄

I'll try to correct and complete in my spare time, if some knowledgeable people mind having a look, especially at highlighted places

according to https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=174429822 jxl's always better maybe also include this sheet?

on slowest settings, i got 67283, 71992, 357758 respectively

veluca has put some work into patch detection that could improve jxl on these

could someone just confirm that anybody have the right to edit the doc ?

yes

thanks 🙂

wow, JPEG XL -e9 give result only 19 bytes less than WEBP

lossless webP is good, but what's better is that the guy who designed webp lossless now works for jpegxl

-e 1 253742 bytes -e 2 310505 bytes -e 3 387400 bytes -e 4 119893 bytes it's amazing how anomalous this image is

the 3rd image under squoosh @ e8 (I'm lazy) goes down to 473kb

only here I can't understand why in this case JPEG XL has performed so badly, it's the first time I've seen similar results in 3 years

webp lossless codec name is VP8L

It has nothing to do with VP8 though

wellknown

Yeah it's a dumb misnomer and probably just a way for the on2 codec team to put their name on it

to feel like they can take some form of credit for it or whatever

Who is truly responsible for it anyways? Didn't Jyrki here help with it or something?

afaik, he pretty much wrote webp lossless as shipped

Well... how did on2 get involved in all the branding and whatever anyways?

It kind of annoys me how they put their own fugly branding on matroska too

I mean come on, how can you possibly improve on this branding?

Does this look better to you?

Who are they to put their own name and branding on a pre existing thing that they didn't create anyways?

I dunno, dumb little things like that tend to drive me crazy, but I'm probably just crazy.

being put off by undeserved & potentially misleading arrogance isn't a crazy thing

can be considered something in lines of autism spectrum if these thoughts get too intrusive & get in your way of life though

but no you're definitely not crazy, i can feel these too (not that it's a good thing for my own stability; trying to get angry abt & fix everything wrong in the world just never works & isn't good way to spend time)

just not as much as i've used to nowadays

I designed WebP lossless -- Pascal helped me to add limitations that he considered important to make it hardware friendly -- such as the 16k x 16k pixels per side and he reduced the descriptivity of recursive control images -- also he asked for the RIFF header and 'alpha flag' which I considered somewhat unnecessary bloat

Zoltan helped with the velocity of development during the format design process by keeping the decoder in sync with my encoder changes

WebP was gated by Mozilla in 2011 -- they had a list of conditions that they'd want to see improved before they'd accept it -- one of the conditions was to have lossless coding and that is how I got involved (thank you Mozilla!!)

you worked for google at the time ?

he still does 😛

(so do I)

oki, nice ! remotely or at Mountain View ?

from the Zürich office

ok, it explains the fact both are awake at the same time as me 😄

https://research.google/people/105344/

impressive resume we have here

I was very selfish in choosing jobs when I was young. In my first job (a neurosurgery robots startup) I recruited Jarkko Oikarinen, the author of IRC, to work with me. He then taught me to code.

I was using IRC from day 1, also used its predecessor. Seeing such systems emerge (most often side projects initially) made it easier for me to understand how to build them myself.

A while ago I suddenly wondered how on earth I've ended up in a discord server full of Google devs, format creators and coders, just from me having fun learning codec options and trying to compress various files. Certainly an interesting world we live in now

IRC ! it makes me feel old 😄 it was our main way to communicate bewteen two games of CS 1.6

too real

It's nice both ways. Direct community interaction is also a lot of fun and very informative for codec devs.

<:BlobYay:806132268186861619>

Jyrki here really is the heroic warlock-priest of image coding

anyone who wants can be it, it just requires a lot of sitting still at the computer 😛

<@532010383041363969> When will you fix the codec for my latest pictures? because considering the number of problems, I would like to say you to literally** return the old version**, because the new https://github.com/libjxl/libjxl/pull/2252 fix has made more problems and artifacts

For autistic vision I'd say it does not look bad

Is good

Not good

????

I'm joking

But is not poor

are drunk 👆

Like usable for autistic people

More than acceptable

2 and 10 images have good colours

WHO ASKED?

<@179701849576833024> can you do something with this drunk man

Is normal because libjxl is not a codec you have a cold block and you drag it the corner of the color temperature and paste warm blocks

Is a feature

It won't do as libwebp2

Because Jon Sneyers do not like this effect

He said denoising make pictures less bright

And codec job is not denoising

I think there's some miscommunication going on...

No is like that they prefer to not do this effect in libjxl

JPEG XL is good for archicterual images

omg the fab is back

fab x dzgas wild interaction

<@532010383041363969> any news? I see that you are work. just don't talk about them. I did the tests on the last build and I see that on the very first 4 pictures that I provided - there are absolutely no artifacts, They work Perfectly! But there are still new artifacts in the new pictures. Here is an example, the first image is new, and it began to look very bad, the colors seemed to have disappeared. But next to 2 old images, which in the last build look just **perfect**

I observe the dim of color in all images. containing a red channel. I don't quite understand why this is so.

recommendation for using this argument is written here. but such argument doesn't exist anymore

now its --modular_palette_colors

I know that for VarDCT, when the quality is -q 0, the image is size resampled 2 times. but I can't understand why to make it for Modular

I absolutely do not understand what the --modular_lossy_palette parameter does (on any -q parameter), the file becomes 10% larger with completely identical contents

It's supposed to enable delta palette and shouldn't be combined with lower q settings. But it could very well be broken in current cjxl, it's rather experimental

-m 0 -q 100 do not work. it continues to be Modular

for VarDCT -d 0 i use -d 0.0000000000000000001 (or less)

(Although this is actually one of the most pointless things you can do with Jpeg XL)

There simply is no lossless VarDCT

<:AngryCry:805396146322145301>

the libjxl artefacts on https://artifacts.lucaversari.it/libjxl/libjxl/ have started taking almost a terabyte of storage, and I figure it might be a good idea to get rid of some of the older ones 🙂 what do y'all think is the best option? (a) remove everything older than six months (b) remove everything older than a year, and keep 10% (chosen at random) of commits between a month old and six months old (c) other ideas?

I think (a) and older release versions

instead of random it can be every nth

Maybe one set of commits per week/month? Then we still have old versions to test with without storing every single change

Or hybrid, keep everything for a month then trim down to once a week

or since there are release versions on github, then (a) + keep one old build every 3 months or something like that

(a) + release tags

(b) would be idk how useful

you'd probably want all of the versions in some sort of different archive thingy to reproduce per-commit stuff

is any of it useful?

(d) delete all of it

yeah honestly i'd keep 2 weeks of versions and do something extra for rest

older releases does make some sense (but at that point maybe I should also name them :P)

or. well. idk. maybe 6 months are fine too if it fits your storage quota

it's my hard drive, so "storage quota" is an ... adaptive ... concept

yeah storage quota is what you decide it to be tbh

tbh if you need to reproduce per commit stuff you should probably build it yourself 😛

can you download releases from GH without being logged in?

Yeah, github has the releases itself anyway, it's just the commits inbetween that matter

fair. this leaves one with just few months plus releases, & anything else is build yourself

sounds.. okay

the main point of this existing at all is that you can't for CI artifacts, so I guess keeping around the last couple of months is sufficient

if you need old releases go to github

(it's also much simpler to implement with "only newer than 6 monts" :P)

maybe do 3 months rather than 6

Something else to consider, Github also has 10GB of storage for the artefacts itself, so anything recent will always be on there

and yeah idk if keeping releases as special is any worth

as they should prob be just attached to tags on github

meh, I'll go with deleting things older than 4 months and calling it a day

im sure it'll be fine for people who just want to try stuff out without compiling on their own

yep

this would delete anything older than `2022-11-11`

which I think is fair

ok, done 😄 thanks for the discussion 😉

how much space did that save?

about 700 gb

now it's ~300 gb or so

Thank you for your kindness and help. I'm working towards more improvements on challenging images. I have a nice improvement in my client, about 5 % on max error of such images, and butteraugli and simulacra2 are also happy -- but when I look at other images, I see more ringing, more artefacts -- I need to choose some other way than what I have now. I will likely need to make Gaborish encoding side non-linear, and possibly turn off Gaborish for certain kinds of images (highly dithered/high contrast/high frequency/patterned pixel graphics), but I haven't yet fully convinced myself about that, so some more experiments are needed.

this is currently the worst image I know about

this image confuses VarDCT and the results are rather disappointing -- some blue pixels seem to turn yellow etc., a lot of unthinkable things are happening

we should fix that...

Yesterday I noticed that when Gaborish is disabled, the errors that I show become very clearly visible even on very high quality like q95 (and even on q96-98) The first image in my collection is the most revealing of all. on the libjxl version from February 27, with the quality of q95 and with an absolutely identical file size, the difference is simply amazing, after fixing in latest builds of libjxl, this image is simply corrupted

I have an idea just compress everything into a normal archive

I think you can delete everything and leave only the One build of every 5 days, that is, the build from the 15th from the 20th from the 25th of each month

A bit late for that

Since this problem has not been solved in 2 weeks since the pull/2252 fix, I personally had to rollback to the version Before this fix in order to use Jpeg XL for my needs

Using album art covers as test images is a great idea.

I literally encode one image and see bugs. I use Jpeg XL only a couple of times a month. but it's enough to see them all. - The image with my albums is literally the only image that I have encoded in the last month use Jpeg XL. if I used Jpeg XL every day then rest assured I would report errors every hour

although in the coming **days **I have plans to compress about 500 thousand manga pages to JPEG XL. But since these are black and white images, I am sure that there will be no problems with them. just for this reason, *I recently asked about the analog of YUV400 in Jpeg XL, and received the answer that 3 color channel are hardcoded* which will undoubtedly greatly slow down the compression, because as I have already shown - compressing an empty image with 1 color takes the same time as compressing a regular image. although I'm more concerned about memory consumption

Monochrome/greyscale images?

I'm pretty sure JXL supports images with only 1 color channel.

idk if libjxl does

libjxl supports grayscale images.

3 color channels are hardcoded only in vardct

iirc

> VarDCT is hardcoded for 3 channels. If 2 are all-zeroes, in principle it could be optimized but in libjxl that isn't done. https://discord.com/channels/794206087879852103/848189884614705192/1082049729782878279 <@1028567873007927297><@886264098298413078><@260412131034267649>

yeh thats what i said

We were doing tests on manga yesterday, you can look at the parameters we ended up with https://discord.com/channels/794206087879852103/794206170445119489/1084857747079700500

no. you said 👉 iirc

I'm going to use VarDCT i.e. lossy compression

Lossless was smaller, and naturally, lossless too. Although lowering the bit depth would help both cases even more no doubt

I had a thought, you can use `r` with benchmark_xl to set a target bpp, why not have it in cjxl too? As far as I'm aware -r does nothing currently while -R is squeeze transform, unless we don't think it would be used enough

Is this true? does libjxl really not support greyscale images?

no, it's not true

VarDCT is hardcoded for 3 colors internally, but in grayscale you have X=B=0 which compresses to some constant very minimal overhead regardless of the image size

there's no special case handling for it cause it isn't really needed

upon decoding if the image is marked as grayscale it just discards two channels

The main reason for adding a special case is to save some memory — currently we do allocate buffers for 3 components which is a bit much. E.g. a 1-bit grayscale image we currently represent using 3 float32 buffers, so 96 bits per pixel, which is kind of wasteful 🙂

But yes, compression/bitstream-wise there is no problem, this is just a potential implementation optimization

that's pretty extreme to have 3 planes for a bitonal image

from an implementation pov

Yes. It's pretty convenient to have only one code path to worry about though.

Yep. But An empty channel with a size of 8000x8000 is file size about ~200 bytes

that's 200 more bytes than needed...

arguably that's just probability-weighted coding though, like it could be 199 bytes less, but then you waste 1 byte on all colored images. so the question is the rate of grayscale images, and I think the answer is probably not very much

I really hope that this can be issue can addressed at some point because once it is then paint.net will be able to include jxl support by default https://github.com/libjxl/libjxl/issues/1450

quite a few people use paint.net including myself

and I have seen in a lot of cases that people will use paint.net as a viewer for formats that they can't open normally with the default programs included with windows

tried to compile via the docker, got the following error ```CMake Error at third_party/brotli/CMakeLists.txt:4 (cmake_minimum_required): CMake 3.10.4 or higher is required. You are running version 3.10.2 ```

crap, the image seems to be 18.04, cmake can't be upgraded via apt...

18.04 is like 5y old by now

https://tenor.com/view/old-man-gif-4679601

is it linuxy thing?

Ubuntu? Yes, April 2018 no

its complicated

At work we use RHEL7, and its always fun seeing what doesn't run due to an outdated glibc.

We still use Centos 7 on production, can't be easily upgraded without causing any downtime. We will probably migrate into Rocky Linux at some point.

many such stories...

still can't beat windows XP ATMs

oh

i know what can

flightmares

https://en.m.wikipedia.org/wiki/Airline_teletype_system

this teletype

I did wonder, we're able to set bpp in the benchmark, but isn't there a place for specifying filesize in cjxl too? The only times I've known of being able to set a filesize target is usually with constant bitrate after calculating from a video file, but it feels like something that should be more commonplace

Huh, interesting

yea the old frontend used to do the binary search for you

the new frontend is based on the library, which doesn't do that

you can still try with benchmark_xl though iirc

`./lib/jxl/color_encoding_internal.cc:234: JXL_FAILURE: Invalid gamma 2.200000`

am I supposed to pass `0.45455`?

Yes, it's always confusing what gamma means, 1/x and x are kind of both used

when I put this image into cjxl with "-e 9 -q 99" the text at the top that says "printer" gains a inner outline https://github.com/bigratmonster/bigrat.monster/raw/main/media/printer1bigger.png

I tried a older version of cjxl from October of 2022 and still had the same problem

I also tested lossy webp (libwebp 1.3.0) and it ended up having no problem with the text

first image is cjxl second image is cwebp

and here is the source image:

looks like gaborish=0 fixes that

i thought it was crazy until i tried it myself

oh wow it does solve the problem

desktop now covered in rat

for %i in (C:\Users\Use\Pictures\telefono\niko\*.jpg) do cjxl -q 96.373 -e 9 -I 15 --epf=0 --faster_decoding=3 --dots=0 --lossless_jpeg=0 "%i" "%i.jxl"

is there a way to crop an ImageBundle?

what do you mean by an ImageBundle?

I assume the libjxl class

ah

I guess the answer is by cropping each of the channels manually

is there an efficient way to do that

an Image(3)F has a Crop method iirc

it is O(1)

Also doesn't dealloc any memory, right?

so it's also good if I have many crops of the same imagebundle?

I don't think that will work with the code as it is now, but it probably is possible to make it work that way

work as in work at all? or as in work well?

It's not made for that, I don't think it will work at all.

after what % of the JPEG XL file download does its first preview image begin? what does it depend on? as far as I know, it shouldn't depend on weight at all and should be tied relatively in percentages. But I can observe that some images are progressively decoded and shown after **5%** loading, some only after 45% loading

here are 2 images, the first image 1 layer of display comes after 4.8% loading, and the second image only after 46.1% of loading, I can not understand why so

this is linked to the question asked in <#794206170445119489>, is seems the info of "where does the progressive pass start and stop" is available in the header, I plan to make a tool to read this from a file

I don't think these are related topics.

don't you want to know after which percent of the file a jxl can be progressively decoded ?

I want to understand why this figure is so different from file to file. What I see in my examples shows that it is almost a static number of bytes, which depends on the number of blocks

If calculate the percentage and size of the file, can see that the first layer is displayed after 1. 18 kb 2. 17 kb

indeed I didn't cared about why 😄 and I don't know

☝️ my concern is that in files that are compressed with quality d 5 - the first layer is displayed only after loading Half of the entire file

generally the first preview is available when the 1:8 (DC) image is available. When that happens % wise depends on some things: - how complicated is the DC? some images have 'easier' DC than others - how much info is in the AC? generally at higher quality, the proportion of AC vs DC increases

then there are also some things that are encoded together with or before the DC, like Patches and the AC metadata (chroma from luma etc)

in images with lots of patches, that can cause the first preview to happen later

Yes. perhaps. it's all very big for images that are created with d 5+ quality

Another question is - is the macroblocks of size 256x256 is hardcoded or frozen by specification?

I mean the blocks that I see during image loading. Is it possible to make them of any size or remove them altogether? will this be supported by the decoder?

256x256 is hardcoded in VarDCT — but of course a decoder can choose to only show complete passes instead of showing every 256x256 chunk as it becomes available

in modular mode the spec allows choosing between 128x128, 256x256, 512x512 and 1024x1024

-g 0, 1, 2, 3 respectively

Are there plans of some kind regarding the release date of 0.9? Or some kind of idea about whether it’s somewhat close or not, or the kinds of features that are wanted for it? Also, what kinds of changes are there in `main` compared to 0.8.1? (If it’s easy enough to summarize, I mean.)

I don't think you need to rush, after a few global changes it works slightly worse in quality than the old version. But immediately after the release of build 0.9.0 on the githab I will compare them

version number is just a number

Generally, assigning a version number is indication of some kind of stability or reached milestone.

Not sure if there will be a 0.9, the next one might be 1.0

Are you sure? because I don't see the full Preparedness

1.0 was wanting to remove all abort() and something else?

Actually I guess it is all listed here: https://github.com/libjxl/libjxl/milestone/1

Abort VMYK Lossy 16 bit Other language Unicode characters Are the big ones, other than optimisation and documentation

jxl recompression when?

it was a serious question i want to be able to input jxl and losslessly re-encode with different settings

well you wouldn't be able to change varblock sizes or HF coeffs after quant

so not sure what else ngl

For lossless to lossless or lossless to lossy it makes the most sense and it would be relatively easy to add

but for that you can just decode to pixels

For lossless recompression of vardct jxl, it would be a lot of work for very limited potential gains imo, since it's only the entropy coding that can possibly be improved, but that will not make more than a few percent difference

unless wwww meant accepting jxl in cjxl

which I agree would be nice

also allow ssimu2.1 and other tools to work with jxl directly rather than png

It would certainly fulfil the "Re-encode with different settings" they wanted, maybe for going to higher distances too. Adds quite a few test cases

The reason they asked is likely that they have files converted with cjxl v 0.8 and `-q 100 -e 9 -E 3 -I 1` (according to their site) but since -I 100 is what -I 1 was before (in <=0.6.1), the files could be multiple percent points smaller

(in fact, I DMed wwwwwwww on the same day mentioning that they likely forgot to switch to -I 100)

Oddly I've had `I 0` or 1 sometimes give better results than 100

at least for windows there is a solution by supporting a wic api, like cwebp or avifenc (with patch), so that all system wic decoders work and for example cwebp accepts webp, jxl, avif, ppm and any other wic installed decoders

which reminds me that the last WIC implementation date from 3 years (https://github.com/mirillis/jpegxl-wic) and would need a refresh

Third possible tool called rjxl just for that task jxl → (smaller) jxl and that doesn't have options one should not use? 🤔

A bit newer, but still nearly a year old https://github.com/saschanaz/jxl-winthumb

nice ! thank you for the link

I want lossless to lossless, including lossless jpeg reconstruction to lossless with optional from pixels. I know it's relatively simple to do manually, just would be super nice if cjxl can handle it

having cjxl and other related tools accept JXL input is on the todo list iirc

This would be great. How probable is this?

Very probable - if nobody else implements that, I will do it myself when I eventually have time to do it

https://ci.appveyor.com/project/louquillio/libavif/builds/46841469

compile this for sse4 gb80

codec avm on

this is a pull request

So now that more and more tools integrate JXL writing I wonder if there couldn't be a environment variable to tell applications which options (that might not even exist yet) to use to save jxl. It looks like ksnip only uses quality 100 and xfce4-screenshooter will likely use quality 100 as well. However, when taking screenshots in PNG and converting them using cjxl, I can easily get smaller JXL files...

I know I wanted something like that for FFMPEG a while ago, but never found anything except building it yourself with the options baked in. Would be nice to have a way to change the defaults

Just opened an issue regarding that. Will likely not be that active for a month starting tomorrow thus I wanted to just get this out by now.

adding more options to the FFmpeg libjxl encoder wrapper is on my to-do list btw

atm only supports distance, effort, and force modular

I meant having a way of setting options to be used by default, so for example -hide_banner would be applied to all commands

There's no real neat way to do it as far as I'm aware, only building it with the defaults changed in the code

yea, there's no config or environment for that like there is with mpv

Hm?

he said compile this for sse4 gb80

https://tenor.com/view/cant-tell-me-what-to-do-its-always-s-unny-in-philadephia-gif-4955839

<:Poggers:805392625934663710> <:PepeHands:808829977608323112>

when I try to make a progressive lossless JXL the file size is noticeably larger then a adam7 PNG

If I understood correctly, modular mode and progressiveness don't work well together.... Which is a shame because the progressiveness of FLIF and FUIF was mind-blowing

FUIF is the same as JXL in that regards

FLIF does progressive like adam7 does it, i.e. with NN subsampling rather than actual downscales

that has advantages especially on nonphotographic images, since it e.g. never increases the number of colors

but it also means you don't really get usable downscaled versions of the image out of it (NN downscaling is a quite horrible way to downscale an image)

the way FLIF gets good compression while doing adam7-like interlacing, is by effectively predicting pixels from 8 of its 9 neighbors — everything except the neighbor at the right or the one at the bottom is already known at decode time, so it can be used for prediction and context modeling

Squeeze does 'real' progressive (in the sense that the previews are actual downscales where every pixel contributes to the averaged value, not just the top left one of each N x N block)

for photographic images, it does it in a way that is reasonably effective in terms of compression

for non-photo, it will almost certainly be worse than what can be done when not 'introducing new colors'

for progressive lossless non-photo, I think we should just use a lossy preview frame at reduced resolution and call it a day, I don't think there's really much more that can be done

for non-photo images, FLIF's interlacing or PNG's Adam7 will be less bad for compression than FUIF/JXL's Squeeze, but the previews you get out of it will also be less useful. If it's just a preview shown while transferring the full image, it's OK, but if you would want to use a truncated bitstream as the final image for a 1:2 or 1:4 version of the image, it's not good enough.

<@532010383041363969> Is there a reason d1.8 is listed twice in the benchmarks associated with your commits? <https://github.com/libjxl/libjxl/commit/492725c4d12d5ed7903c361a25071990600ac2eb>

Didn't want to bother needlessly, but I kept seeing that and imagined it could represent unintended bias in conclusions.

probability is now approaching 1: https://github.com/libjxl/libjxl/pull/2444

»Allowing jxl input for cjxl can be useful if the input is lossless or higher quality than what is desired for the output.« Doesn't this also allow going from lossless to lossless with some additional bytes saved (better options)?

Awesome! Looking good!

Yes, that too.

Could it be a 16-bit png that uses sBIT to make the channels 8-bit?

does this file produce this? https://buzo.us/W.png

it doesn't for my version of imagemagick

but it's a 16-bit PNG with 8-bit sBIT chunk

my application that uses libjxl doesn't seem to free its memory correctly. i destroy `JxlDecoder` and `JxlResizableParallelDecoder` at the end, but depending on the image, the memory usage can be as much as 20 MB higher than it should be, and that doesn't count the actual buffer

any insight from valgrind or msan?

how's this? i'm not good with debuggers

yes, i'm using libjxl in a zig application

hi, is there a guide to all the information that needs to be provided to the encoder/decoder to support arbitrary number of extra channels (lossless, if that makes a difference)

compile with -g

the api docs should cover it

```powershell $ cjxl -e 10 -d 0 --brotli_effort=11 --allow_expert_options --num_threads=0 -I 100 -g 3 -E 11 cover.jpg cover.jxl JPEG XL encoder v0.8.1 c27d4992 [AVX2,SSE4,SSSE3,Unknown] Note: Implicit-default for JPEG is lossless-transcoding. To silence this message, set --lossless_jpeg=(1|0). Read JPEG image with 269801 bytes. Encoding [Container | JPEG, lossless transcode, effort: 10 | JPEG reconstruction data], JxlEncoderProcessOutput failed. EncodeImageJXL() failed. ```Should I report it, even if the command is quite exotic/uncommon ?

I think that's already reported

one sec ill try to find the issue page

it seems like this has already been solved but the fix is not in any stable marked builds/versions currently. https://github.com/libjxl/libjxl/issues/2034

compile libjxl with -g? what should that do?

no, compile your own code with -g

what that does is it changes those `????` to actual function names

with lines of code and source.c

also you say the memory is ~20 MB higher than it should be, but how are you measuring this, and how are you measuring how much it "should be"?

gnome system monitor, i have other decoder libraries in my program that have at most 1? MB memory usage after decoding libwebp, spng, libavif, libjpeg-turbo

i'll try

the memory buffer is reported separately as Shared Memory, so it's clear how much of it is ideal

the application is designed to use very little system memory, so there is clearly something up with my usage of libjxl or libjxl itself

zig build doesn't recognize `-g`

`-g` is a C compiler flag. You need to figure out how to keep debugging information in Zig if you’d like.

d1.8 doubled -- just a mistake I had on the command line

the optimization uses a set of randomly chosen somewhat geometric series of distances in a selected range, usually something like d0.4 to d6.0 -- the last verification is more for manual viewing and getting some idea how the optimization scales to d0.1 and d7+ etc.

of course I shouldn't have d1.8 doubled, but it is not going to make a significant difference here I believe

it is very possible that libjxl isn't handling memory properly. you can run djxl through valgrind and find numerous issues

a number of them are mismatched free() and delete()s, idk if we care about those or not tho

I'm not really a C++ programmer so I don't know how big of a deal that is

well in any case we need to ensure that 1) libjxl has no leaks and 2) it returns error when alloc fails, instead of just aborting

I'd say (2) is higher priority

file an issue about the valgrind -- how you did it and what are examples on what you find

then you can relax and do more interesting things and the bugs will fix themselves 😛

pretty much any input has mismatched free/deletes, so lemme try with a conformance sample

well, maybe not quite, but it can be more fun for a competent C++ programmer to fix

if you find bugs and are able to debug and fix them -- while enjoying the time -- then it is of course appreciated

I would if it were C but wrangling C++-specific issues (mismatched free/delete) is outside my scope of what I'd call "fun"

since I just don't know the language-specific stuff that well

interesting, it's gone after I updated libjxl

bug must have been fixed already

good job to whoever did that :D

I *suspect* this is actually a valgrind issue

i.e. it's not replacing *all* calls to free/malloc with its own wrappers properly, or stuff like that

well apparently after recent commits that say "fix msan issues" it works now

so not sure

ah, maybe

``` ./lib/jxl/dec_ans.cc:367: Histogram can represent numbers that are too large: 124 ``` what does this mean?

https://github.com/libjxl/libjxl/blob/5fef3ec57452f2865a62c0e01cdcf3635f09b920/lib/jxl/dec_ans.cc#L360 long story short, some code tries to upper bound the # of bits needed for coefficients during decoding (to use smaller buffers if possible) and gives a warning if it can't know 32 will be enough

Quickly made an example, both 8bit and 16bit RGB/RGBA PNG export works fine on it.

With 16bit output, i got that 16/8-bit thing too. ``` Depth: 16/8-bit Channel depth: red: 8-bit green: 8-bit blue: 8-bit ```

Yep

Oh wait, it encoded 16bit rgb example without errors, but the result is this

And that's encoded 16bit RGBA example

I used JetBrainsMono Nerd Font, so it's probably the font not installed in Inkscape.

<@456226577798135808> Shall I open an issue?

https://github.com/libjxl/libjxl/issues/2457

``` lib\jxl\render_pipeline\stage_blending.cc:40: JXL_FAILURE: Trying to blend XYB reference frame 0 and non-XYB frame lib\jxl\render_pipeline\render_pipeline.h:81: JXL_RETURN_IF_ERROR code=1: stage->IsInitialized() lib\jxl\dec_frame.cc:653: JXL_RETURN_IF_ERROR code=1: dec_state_->PreparePipeline(decoded_, pipeline_options) lib\jxl\decode.cc:1142: frame processing failed ``` hmm, I bumped into something while decoding a layered jxl (with coalesce) that encoded losslessly with effort 5 and up...

it weirdly only affects a certain part of an image, and went away when that part got cropped

Can you open an issue?

sure, I'll also attach the files there

hello

why are some libraries are cloned with libjxl git ? some of them are so used that i doubt that a system does not have them. And even in that case, it will be easy to install, these days.

libpng, zlib, libjpeg (instead of sjpeg), lodepng can be removed too

also brotli is pulled as third party and recommended to install from the system on the BUILDING.md

it's just a matter of throwing an error if they are not installed (if they are needed)

it's a thing that google libraries do for some reason

Isn't this mostly just for convenience for Windows builds?

I don't know, but most google libraries do this shaderc, highway, etc.

and the default is to not use the system libraries and instead statically link them in

even if they exist

<@853026420792360980> it's not because google libraries are doing so that you should also doing so

shared libs are good, and save space on disk

on windows you also have vcpkg, or other similar systems, that could install libpng or other libs

but, imho, relying on system libs makes the build system a lot simpler

also, when i want to test the latest code, pulling from git is a lot faster

and for windows build, it's easy : install msys2, pacman -S libpng etc... and some comands for cmake

I don't disagree

I was just pointing out the pattern

From my current experience, using vcpkg is a good way to ensure a proper build, but this also can be a real pain in the neck (my project has more than 300 C++ dependencies at the end of the day (including libjxl) ) but when it works it really a good way to ensure quality end product, and the best is that if you do not want vpckg embedded you can still do it by hand (which in my case i do not support at my job)

hey, is butteraugli used for XYB - RGB colorspace conversion ,

?

<@931493879801348106> i have written a small program than compile the dependencies of the framework i'm using. It compiles them from source

<@931493879801348106> it can also cross-compile them on linux for windows

i prefer that rather than vcpkg

because i know how they are built, i can tweak the build as i want, etc...

it is not

XYB to RGB a biased cubic transfer followed by a linear transformation

butteraugli is a metric

there's also a (simple) linear transform before that 😛

ye tru ig

kek

thanks

<@918053334554918933> : Sounds great, in my case i have 375 packages at the end of the process, that's why i picked the vcpkg way, and i like the fact that we can still do it by hand since i use CMake to control all the configuration logic of the specific software. Lots of different options exists, but there is probably no silver bullet for all cases

<@931493879801348106> i have far less packages (around 85)

????

cjpegli from here https://artifacts.lucaversari.it/libjxl/libjxl/latest/jxl-linux-x86_64-static.zip

I'm trying to statically link a windows exe with `jxl_dec-static.lib`. I built libjxl by generating visual studio build files with cmake and compiling in visual studio. The linker cannot find the implementations, e.g. ``` error LNK2019: unresolved external symbol __imp_JxlDecoderCreate referenced in function ... ``` It works fine when linking with `jxl_dec.lib`. Am I misunderstanding the intent of the static versions?

Not sure but I would try to define JXL_STATIC_DEFINE

Awesome, that was the missing ingredient. tyty

Are there any plans for automatic chroma subsampling with jpegli, like mozjpeg does? I think jpegli has a lot of potential but this holds it back right now

jpegli is first and foremost a library -- the user of a library decides about coding particularities -- (we are having an increased focus on yuv420 now, optimizing its quantization matrices etc.)

would you create an issue in libjxl for this? -- is it this image https://wallhaven.cc/w/x67o8v ?

seems reasonable -- do we have libjxl issues for all of these?

it seems like it: https://github.com/libjxl/libjxl/issues/2126 https://github.com/libjxl/libjxl/issues/1763 https://github.com/libjxl/libjxl/issues/1719 https://github.com/libjxl/libjxl/issues/1450 https://github.com/libjxl/libjxl/issues/683

I took those directly from the issues haha, so yeah

Yes. That image

But with the cjpegli binary, it still loses to mozjpeg in either mode but isn’t prone to losing overall; switching between 420 & 444 at a particular quality level would probably do some good, no?

Does libjxl decoder support all format features now?

In other words: is decoder part completely finished?

yes, it has been for a while

up to undiscovered bugs, of course

<@532010383041363969> I've found another image (vector + text) that jpegli behaves badly on - lots of nasty color bleeding! (Tested both older versions and the libjxl git main branch commit 05966ba4).

Original image.

At -d1

At -d3

Blue background bleeds purple into the "Jules Ducatel" text. The circle/diamond shapes in the text bleed cyan/magenta onto the background.

could you give a zoom of the worst bleeding

this image had alpha, removing the alpha channel fixes it

at lower quality or specific content it is possible that 420 is better

The abort() thing is an extremely serious issue and is a stopper for libjxl being used by Paint.NET

It makes all other bugs seem like not a big deal by comparison

Of course the most important thing for any new compression is how good the compression is, but having a usable implementation is a close second...

Yeah, I mentioned the top issues a while back and Jyrki did take notice, but not sure if anyone's actually assigned to it yet

It's one of the things we need to address before 1.0

https://github.com/libjxl/libjxl/issues/1450 It is not forgotten

almost the anniversary

This https://discord.com/channels/794206087879852103/794206170445119489/1107298314237530132

I see that there are quite a lot of pretty big changes planned for 1.0 so maybe it would be worth it to do a 0.9 release at some point in the meantime with the abort() issue solved?

Happy to report that ImageMagick seems to work fine when `LD_PRELOAD`ing jpegli and converting to JPEG! (with libjpeg so version set to 8 because Arch)

makes things much easier sometimes than using `cjpegli`

I don't think it's solved in the master branch yet

I know

I was just trying to say that once it is solved it could be pushed to a 0.9 release

that's awesome. It would be nice to install jpegli as the system libjpeg.

I notice that there is no separate folder in the source tree for public API header files, like an "include" folder.

That kind of makes the project look messier.

The time is nigh for libjxl to conquer the planet

Better JPEG-compatible compression for free is likely going to be a big factor in that.

When you compare jpegli to other encoders it's just a no brainer. it's like night and day

It's truly excellent, except for some odd cases where it trips and falls down the stairs -- like weird color bleed and ghost spot issues <@226977230121598977> and I have encountered.

<:CatBlobPolice:805388337862279198> I am sure that what is jxl now cannot release as 1.0

lol. Well does it truly produce worse-looking results than mozjpeg in those cases?

Is this when encoding to jpegxl or jpeg1?

or both

I want to use the libjxl decoder as a library in a C-to-web-assembly project. How can I produce wasm object files to link with? I do have it compiling and running the CI tests with emscripten.

you can try to follow this : https://github.com/libjxl/libjxl/blob/main/doc/building_wasm.md easier with an up to date ubuntu (or a docker)

this page is a bit buggy though, be ready to navigate

lossless libjxl 0.8.1 with e3 gets a larger file size than the uncompressed version of the image above, am i missing something?

not surprised with that image

try e1 or e2

idk maybe the dithering is causing problems?

ohhh that's an interesting testcase

it's certainly true that jxl lossless and dithering don't play very nicely with each other

<@179701849576833024> ok i think i found the issue, the original image is using pal8 instead of rgb24. That could explain why when loaded in photoshop it showed as "scala di colori" instead of rgb.

mhhh yeah, e1 at least doesn't paletteize non-rgba images

hold my beer, https://github.com/libjxl/libjxl/pull/2518

now e2 and e3 will be embarrassingly worse than e1 on that image since they don't do any palette detection at all but at least e1 doesn't care if you give it pixels as RGBA or RGB anymore

now it works for cases like this? https://canary.discord.com/channels/794206087879852103/794206087879852106/1060234070409347173

yes, and I also changed it so e2 and e3 also do some palette detection so we don't get the embarrassing behavior that e1 does better on png8 input than e2 and e3 (though e2 might still be better than e3 depending on the image)

Are you trying to go for the Metal Gear Solid look with your profile image, <@794205442175402004>?