JPEG XL

for non-photo, I think the best option is often to just do lossless

Not all non-photographic(synthetic) have this issue, only image that features high contrast sharp edges and smooth area have this issue.

yes, I mean more specifically non-photographic with text or drawing styles with strong edges

Pff . I use -d 15 😄

there’s really little reason to store most non-photographic content in lossy form.. humorously, in one of my archives of non-photographic content, mid quality JPGs have roughly the same file size as PNGs of those compressed JPGs (even when storing all the artifacts). If only I had the originals…

if you can actually check them, I suspect running some restoration software on them and then compressing them to PNG might help 😄

I don't want praise avif, just a example, I don't want praise avif, just a example, In cavif Q90 and Q95, that features high contrast sharp edges and smooth area is compress very well, can keep smooth, but avif don't have maximum error metric(butteraugli), so probably have some error, I like jpeg xl, because jpeg xl use butteraugli to keep compressed image have high fidelity and reduce error. https://github.com/kornelski/cavif-rs

For me, even if it is not the job of the image encoder, adding in the future an denoiser algorithm in encoder option would be super useful and efficient to please those who want to fall into the very high compression and to please well in tests against avif. Maybe it could be better than reducing resolution, some noise reduction algo are amazing 😄 (big artefacts vs noise reduction, i prefer noise reduction :D) For me you are creating the best ultimate file format, so you need to beat visually, denoiser format like avif and webp2 at low bpp 😄

this is not so good for Generation Loss and IMHO is the main factor of avif bad adoption from big companies

it subtle remove details

Yes, but it's the magic of option, optional 😄

I think denoising is something to do interactively, in photoshop or gimp

Or maybe with some good AI

avif are no agree at all <:HaDog:805390049033191445>

I haven't found a way yet to algorithmically distinguish unwanted noise from wanted subtle texture.

And I suspect there is no way. At the pixel level, there is no real difference. It is all semantics.

What's your favorite cjxl speed? Answer with an image or GIF

https://c.tenor.com/AIFF7A29t5UAAAAM/vombat-wombats.gif

https://tenor.com/view/chipmunk-nuts-gif-11219351

https://tenor.com/view/kittens-meowing-kitty-kittie-adorable-gif-3471852

https://tenor.com/view/ninja-turtle-angry-turtle-angry-turtle-michelangelo-gif-15243800

https://media.discordapp.net/attachments/412215006718066689/788436248506793994/50fps.gif

https://c.tenor.com/eXMRBgNIvQYAAAAM/falcon-flying.gif

<:This:805404376658739230> no fans?

I mean, peregrine falcons are quite fast.

In dives

So far, the fastest speeds do not offer such a benefit over more supported formats and JXL has not yet been added to applications for saving screenshots and such, where fast speeds are so necessary

The raptors like gathering at the highways here and eating any rodents unlucky enough to get stuck/run over

https://c.tenor.com/9J-PcRNutaUAAAAM/rabbit-animal.gif

https://media.discordapp.net/attachments/233804746400595970/722898971298627694/urgant.gif

`--speed human` ?

he doesnt know

We need `--effort sisyphus`

Constantly convert jpg to jxl and back again?

i used -s 3 -d 8.08

maybe for some images with 0.3.1

or s7 -q 15

i don't remember the settings

so weird, cjxl -d 0.5 -s 7 --epf=3 and -d 0.5 -s 3 --epf=3, -s 3 still have some error(noise) but is better than -s 7, maybe i crazy?...this result is unexpected.

I try to make things better on a wide front without too many completely silly compromises. If silly compromises, I try to fix them soon.

soon in jpeg xl (0.3.3 or so): d5 BPP * pnorm improved by 2.6 %, d23 BPP * pnorm improved by 9 %

so first 0.3.2

--speed 🥔

looks much better, but default encoding got 20 % slower (from 10 to 8 MP/s) ... needs some more work...

also some more ringing

but for every instance of ringing 7 nasty other problems were fixed

of cause ringing is just a no-no... difficult decisions 😛

I was hoping to change this with VarDCT being nearly ok at graphics. It would be so awesome if people didn't have to think a lot what to do -- just compress.

just use d1.0, or d0.6 to d1.0 always and no thinking

Any advice for compiling JPEG XL on Windows? I've been trying for hours with VS 2017 and after installing LLVM to get the Clang compiler, I get this error.

Cygwin doesn't seem to work, either.

<https://github.com/m-ab-s/media-autobuild_suite> (with selected Clang compiler) or <https://encode.su/threads/3564-JXL-version-0-3-released?p=68594&viewfull=1#post68594>

Thanks, <@111445179587624960>

The binary in the second link works perfectly. Thanks again.

Does anyone else have success with very large images? I'm trying to compress a 20104x15527 PNG and it keeps crashing with the message, "..\lib/jxl/image.cc:102: JXL_CHECK: bytes_.get()".

probably running out of memory? 🤷

it's only what... 900 MB in RGB data?

https://gitlab.com/wg1/jpeg-xl/-/issues/99

Also, I've just thought about something else. Is it normal I'm getting: ```[bluezakm@bluezakm-pc ~]$ cjxl /home/bluezakm/Documents/220324.jpg /home/bluezakm/Documents/220324.jxl -s 7 -d 1.0 -j --num_threads=1 J P E G \/ | /\ |_ e n c o d e r [v0.3.1 | SIMD supported: AVX2]``` While others are also getting AVX2/SSE4/Scalar for SIMD stuff?

Is it a good thing or a bad thing?

Nothing too important. I'm just curious.

This happens when compiling with -march=native on AVX2 CPU (or -march=haswell and newer)

Ah ok then, that's why.

It makes sense.

That's nice. The image takes up 20GB of RAM at peak.

https://tenor.com/view/kittens-cute-cat-pet-cheeks-gif-16382546

<a:catdance:770738257846599710>

https://twitter.com/jyzg/status/1359661198350893060

The one I notice first is usually pixelization

I picked banding because it can remain an issue even when viewing at large distance or downscaled.

in my last quality change I reduce banding and 1x8 pixelisation, increase ringing and blurring (and 8x8 pixelisation)

once the next round of butteraugli masking improvements is properly verified with tensorflow synthesis, I can improve the heuristics further and likely get another 5 % improvement in quality (removing the new ringing that the latest changes brings)

after that, I suspect that further improvements in VarDCT quality will only be possible for very large images or for using the layered approach for very low bpp

also the last change reduces weird visual activity that comes from the smallest transforms being quantized heavily -- like when you quantize a dct2x2 or 4x4 too much, some pixels start popping out and the area becomes visually more active than the original

the air force chapel image is a prime example of that

I'd like to learn what people think about delta palette

I have many ideas around it (likely 20+ % improvement potential), but I'm keeping them on hold and working on VarDCT instead

👍

I think delta palette is mostly useful to bridge the gap between highest-quality VarDCT and lossless

I think I know what "pixelization" is supposed to refer to but ... I don't recall ever seeing it as image compression artefacts. 🤔

I'd call it blockiness, no? though blockiness typically means bigger blocks (8x8 and larger), while pixelization is more like 2x2 or 4x4 or staircasing at diagonals

I think we need a lexicon of compression artifact terminology, with examples

especially with all the new ones that modern codecs bring, like smudging and smearing, plastification, chroma spread, over-eager copy, etc

I remember doing the codec comparison when it was posted in encode.su forum. What I noticed for the airforce chapel picture for avif vs jxl at lower bpp was that avif retained more the details and lost more of the color. where as jxl kept more of the colore but the image was more pixelated

I made an ontology of magnetic resonance imaging artefacts as part of my Ph.D. thesis 😄

I consider the airforce chapel image embarrassing for me at low bpp, and I'm improving on it furiously. 😄

the way avif encoders do chroma from luma takes chroma trickery to a next level compared to just subsampling it. It allows to reduce chroma quality by a huge amount while avoiding the typical ugliness you would get if you would just subsample CbCr 8x, because the chroma signal still 'follows' the luma signal at full resolution, just the actual information is gone so it might be the wrong color, but it will look OK (when looking superficially)

in video compression it may be even more important to create comfort in every frame

the next seven frames will fix the problems in this frame -- if there is bandwidth for it and the scene hasn't completely changed already

I have observed a video compression artefact were you first get the room and the objects and the texture on tapestry is painted only 200 ms later or so

things like that can happen if prioritization is done only by psnr without consideration to masking

what is delta palette?

it is a fixed 5x5x5 RGB palette, inteleaved with a smaller 4x4x4 palette, and a small number of 'deltas' that modulate a predicted color mixed "on the canvas"

plus you can add your own colors and deltas in between of the fixed palette and the deltas

it's a bit over the top for my understanding

I'll let you know what was my impression when doing the comparison as I was side tracked with other things then and didn't write it. maybe it will help as an oppinion

I came to the conclusion that I would prefer for the image to contain more fine details than color accuracy. Like I would prefer to know that the object in the distance is a car, than to have the better color of the car

and in some pictures avif had more detail

this need of balance can change with resolution, i.e., if you are zooming 300 %, then details are ++++, if you look at it at 8k, then color accuracy can be more important ?

Yeah, you're right.

for lossy you need to find a balance

OH!? A box-centered cubic lattice! \:-) I tested those for turning true color into 256 color palette imagfes \:-) (<@532010383041363969>)

Oops. I meant to quote "it is a fixed 5x5x5 RGB palette, inteleaved with a smaller 4x4x4 palette,"

It might be worth going for *two* 4x4x4 palettes ... so, 5x5x5 + 4x4x4 + 4x4x4 (each shifted slightly)

(for lower noise w.r.t. luminosity)

the fixed palette are what you get when you use "out of bounds" palette indices (index values larger than the palette size)

you can add whatever other colors to that, e.g. if some significant region of the image is in a single color, it's probably worthwhile to add that color as an explicit palette color

the implicit palette could be nice for tiny images, to avoid the cost of encoding the palette itself

yes, sometimes the palette is as much as 7 % of the size (from WebP lossless experiences)

with delta palette you can have a million colors for gradients and it doesn't take any space for the palette

but still get the benefits of palettization for 'pixel-perfect' no ringing/no blurring areas in the image

the delta palettization runs at constant quality (of around 2.5 bpp for photographs)

compared to normal palettization it is a big improvement -- normal 256 palette requires about 4 bpp for the same image and looks much worse

with different encoder choices, we could probably make it more or less lossy, maybe even adaptive

and those results are just with the default empty palette (using only implicit palette colors and implicit deltas)

could likely do better with some well-chosen image-specific palette colors and deltas

there is a sweet spot

after certain amount of loss it is clear that everyone should use vardct

oh yes

or modular

I think we are close to it

but the dithering it does is detrimental for large constant colors

adding large constant color ares into the palette would help a lot

also we can do a better job in context modeling the pixels based on the surrounding palette indices

also we can 'invent' new palette indices that map to a short sequence of pixels through context modeling, the repetition is possible there too, not just through lz77

for short sequences it can be more effective

2-4 pixels

because then the context modeling can relate to which sequence to ignite

for lz77 it would be just the distance and that is just random numbers for context modeling

could probably also easily make a mode where the encoder guarantees a max per-pixel error (just adding more colors or deltas into the palette as needed to satisfy that constraint)

yes, I started with 8x8x8 palette augmented with a 7x7x7 palette in the gaps

and went smaller down to 5x5x5 augmented with 4x4x4 because that still worked well for both butteraugli and my own eyes

About high contrast sharp edges and smooth area, in cavif Q90 and Q95 smooth area can keep smooth, but in jpeg xl vardct -d 1.0 s7 and -d 0.5 s7, smooth area have some error, I research some avif discuss, in some non-photographic encoding discuss, disable av1 CDEF can get some benefit in non-photographic content(anime), So i test vardct -d 0.5 -s 7 --epf=3 and vardct -d 0.5 -s 3 --epf=3, result is unexpected, -s 3 smooth area sill have some tiny error, but quality is better than -s 7, -s 7 get more noise, file size -s 7 is smaller -s 3, if i increase --intensity_target value, file size will increase too much, I think probably some encoding method(loop filter?) will get some bad effect in non-photographic image?

consider posting examples from what doesn't work

if you can create a relatively small images, like 64x64 to 256x256 pixels, I could just include them into a 'difficult images' optimization set

please post such crops into a bug on jpeg xl

feel free to create as many bugs as you like, we can manage them easily

currently I don't fully understand the problem -- other than superficially -- and I'm not tracing it's performance over quality improvements

to have a better dialog I should be able to understand the problem you have, then make detailed improvements, communicate them to you and you then verify if that actually agrees with your experience

until now I was too busy with low bpp improvements

I think I'm going to take a short break of that now and look at quality more holistically and that could incorporate your findings

currently I'm running a combo of about 50 cropped high quality images in my verification/optimization set

ok, thank you 🙂 , i should upload sample to encode.su or other site?

some of them are collages of difficult images

but best if each difficult image is separate and as small as possible (say down to 128x128)

best if you upload them into a bug in jpeg-xl site

https://gitlab.com/wg1/jpeg-xl/-/issues

actually i still not sure this an issue or encoder limit...

i see rigging in all semi costant color

Crixis -- file bugs on gitlab about it

if we get a huge diversity of opinions on how to balance the difference errors (like banding vs. ringing), we could add controls where people want their errors

no one will know if it is an encoder limit until it is fixed 😄 .... I think we can improve the encoder a lot

also if it is something that happens rarely, we can just up the adaptive quantization there locally without consuming a lot of bits

even if we would otherwise fail in producing a decent encoder, we could just teach a neural net where are the areas that people don't like -- and give another go with a neural net trained to give 50 % more precision in those areas

that approach only needs that the failures are rare (spatially somewhat isolated)

A future auto selection of lossy modular mode when it is better is possible?

Seems like I've found a bug with one of GIF files

This is the schema of post-WWII tram network progression in Warsaw, where I live

Discord fortunately didn't mess with the file, hashes match

I get the error no matter if I try this command: ```PS C:\Users\zieme\Downloads\jpeg-xl-mingw64-ef3f7a62> .\cjxl -m --palette=1024 -s 9 -g 3 -E 3 -I 1 ..\1435c.gif ..\1435c.jxl J P E G \/ | /\ |_ e n c o d e r [v0.3.1 | SIMD supported: SSE4,Scalar] Read 2600x2711 image, 29.7 MP/s Encoding [Modular, lossless, tortoise], 2 threads. terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc```

Or this, the simplest possible one: ```PS C:\Users\zieme\Downloads\jpeg-xl-mingw64-ef3f7a62> .\cjxl ..\1435c.gif ..\1435c.jxl J P E G \/ | /\ |_ e n c o d e r [v0.3.1 | SIMD supported: SSE4,Scalar] Read 2600x2711 image, 29.3 MP/s Encoding [Modular, lossless, squirrel], 2 threads. terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc```

My current RAM usage is 6.5/7.9 GB (82%)

Surprisingly though the other GIF from the same site, with pre-WWII tram network, encodes without any issues... ```PS C:\Users\zieme\Downloads\jpeg-xl-mingw64-ef3f7a62> .\cjxl ..\1525c.gif ..\1525c.jxl J P E G \/ | /\ |_ e n c o d e r [v0.3.1 | SIMD supported: SSE4,Scalar] Read 2600x2684 image, 30.5 MP/s Encoding [Modular, lossless, squirrel], 2 threads. Compressed to 325551 bytes (0.373 bpp). 2600 x 2684, 1.12 MP/s [1.12, 1.12], 1 reps, 2 threads.```

...with both commands. ```PS C:\Users\zieme\Downloads\jpeg-xl-mingw64-ef3f7a62> .\cjxl -m --palette=1024 -s 9 -g 3 -E 3 -I 1 ..\1525c.gif ..\1525c.jxl J P E G \/ | /\ |_ e n c o d e r [v0.3.1 | SIMD supported: SSE4,Scalar] Read 2600x2684 image, 29.3 MP/s Encoding [Modular, lossless, tortoise], 2 threads. Compressed to 241801 bytes (0.277 bpp). 2600 x 2684, 0.40 MP/s [0.40, 0.40], 1 reps, 2 threads.```

Discord left this GIF intact, too

i can encode it

The first one? `1435c.gif`?

one moment

yes

Im on Linux Mint Jxl 0.3.1 Clang 11

I'm using Scope's latest Windows build

https://encode.su/threads/3564-JXL-version-0-3-released?p=68594&viewfull=1#post68594

any ideas what should we do next with libjxl?

put it in firefox XD

(not that we actually have any free bandwidth from security work, memory reductions, etc.) 😛

Better support for animations and layers ;)

😄

what were the most surprising good features of JPEG XL for you personally?

noise=1

Thank you! It may be difficult to understand how much effort went into such a 'simple' feature as noise=1.

how to make it look natural while not spoiling image features

I suspect that about one man year was spend on that alone, which is quite ridiculous, since a simple noise model can be done in 7 minutes

Funny enought it encode well in wine

I haven't looked how other compression formats do it, but I'd be surprised if they have as much rigor in it like we do

(rigor leading to image quality)

<@!532010383041363969> yes, yes, I will include you in my prayers 😄

Effective lossless JPEG compression

Also usually being better than lossless WebP

the progressive mode, from flif era

release a 1.0 version.

Patch, spline, points, modular blocks

a smart encoder will be very surprising

Patches, it'll actually make lots of PDF-to-JXL converters easier – just pre-render patches for every char from font(s) used and then put patches in right places.

Idk, i can do a multi-page jxl?

can do multi-frame

so it must be at application level

kind of the same as multi-page tiff, or animation: applications that don't support it will show first page / frame

a fully featured viewer could show animations based on animation timing, but also allow manual seek to next/preview frame

then you can do multi-page as just an animation where every frame takes a year

I think a metadata can indicate if it is an animation o a multipage jxl but i don't think this will become a thing if not proposed as a "standard tag" from the reference encoder

we could define something at the file format level for that

or maybe not encode it as animation in a single codestream, but as a container with multiple codestreams – makes it easier to have e.g. pages with different canvas sizes

seem better just include in pdf

yes, that also makes sense

in general, I would not recommend increasing the intensity target solely as a means of increasing quality; intensity target is meant to indicate the maximum luminance in the image

so by using a very high intensity target for an SDR image, one would effectively be indicating that the image should be displayed super bright

which image viewers may actually start to honor once HDR equipment and awareness is more widespread

Good point. Maybe we should allow lower distances then though

in the form of a meme:

(original image from: https://mixinglight.com/color-grading-tutorials/10-things-about-hdr-grading/)

I would appreciate smaller distance targets.

Is the 0.1 limit in cjxl or in libjxl?

cjxl

you can use `benchmark_xl` to set arbitrary distance targets, cjxl is clamping the --distance to a "sane" range, but I think it's not really up to us to define what "sane" means

for most use cases, d0.5 should be more than good enough, but if you're planning to zoom in 30x on the pixels, then you will need more

``` ./ClassA_8bit_WOMAN_2048x2560_8b_RGB.ppm.png Compr Input Compr Compr Compr Decomp Butteraugli Method Pixels Size BPP # MP/s MP/s Distance Error p norm BPP*pnorm Errors --------------------------------------------------------------------------------------------------------------------------------- jxl:d0.1 5242880 4004785 6.11081695557 1 1.484 20.809 0.34396189 0.14815344419 0.9053385788035352 0 jxl:d0.075 5242880 4622245 7.05298614502 1 1.481 18.871 0.28397459 0.12216761202 0.8616464749793261 0 jxl:d0.05 5242880 5625598 8.58398132324 1 1.458 17.847 0.26443976 0.09208368485 0.7904446309536913 0 jxl:d0.025 5242880 7586001 11.57531890869 1 1.429 15.487 0.23897156 0.06607332590 0.7648198186797819 0 jxl:d0.01 5242880 10301559 15.71893157959 1 1.156 10.406 0.22908260 0.05861984391 0.9214413156050707 0 jxl:m 5242880 7038526 10.73993835449 1 0.227 2.827 0.00000000 0.00000000000 0.0000000000000000 0 jxl:m:q99 5242880 4526472 6.90684814453 1 0.466 2.606 0.38793665 0.16204786118 1.1192399693316055 0 jxl:m:q99.5 5242880 6260970 9.55348205566 1 0.366 2.473 0.25324365 0.10762652258 1.0282080521629313 0 jxl:m:q99.8 5242880 8479261 12.93832550049 1 0.338 2.218 0.12373304 0.04831840360 0.6251592334949853 0 Aggregate: 5242880 6211483 9.47797133873 --- 0.744 7.097 0.25374938 0.09267222019 0.8647272779811280 0 ```

on this image, -d 0.05 still makes sense

at some point, lossless is smaller: -d 0.01 probably never really makes sense

also very high quality lossy modular might make sense in the range between d0.05 and lossless

(on some images -m -Q 99 is already larger than the lossless image though)

Hmm, it would be handy to have other extra options and higher limits in benchmark_xl, like the ewp2 encoder in WebP2, compared to the regular cwp2

we need to clean up cjxl and make it use libjxl, benchmark_xl too, and avoid the code duplication for encode options

I wrote a batch file to convert JPEG images to JPEG XL and with the 60 random files I chose, it was able to reduce the size by 25%.

I didn't think a lossless JPEG compressor could make them that much smaller, especially since lossless WebP makes JPEG's bigger.

I'm not a great believer of FLIF progression, FUIF was a lot better at that.

What are the small distance targets for? (just curious, not judgemental)

I liked the idea that you could dd a file and reduce it's size. or that part of a file is a reduced version of the file

I've read that now in jxl the progression is based on size steps (1:4,1:8,1:16....). So now if I have somewhere in between 1:8 and 1:16 of a file it would still have additional detail or it would be the lowest step 1:8?

difference between FLIF and FUIF is nearest sampling vs. averages

you cannot show a 8x8 nearest subsampled image to a user in many cases -- for some photographs it may almost work, but it is pretty far from from good quality

FUIF upgraded it to averages

initially pik (and brunsli) had 8x8 preview only

in pik it was expanded to 16x16, 8x8, and then "scan script" over the dct, i.e., you can get high frequency inform earlier, just some more ringing (just like in normal progressive JPEGs)

when lossless and jpeg recompression were added, some guarantees of always progression disappeared

If we have a normal mobile connection with 17 mbps speed, we can deliver a full 4k image (8 mpix) in 500 ms

how many progression levels do we want in 500 ms

I like to delay the first update to happen at 50 ms to have 8x8 subsampled image first (instead of showing some other lower quality versions first)

perhaps show another rendering at 200-250 ms, and then the final at 500 ms

so now if I have a file encoded with --progressive, and I have transferred, say 27% of the file will I see the 27% quality of the full file, or the 25% (1:4) ?

it is really complicated -- many options

if passes are used, you'd have a pretty good image at 27 %

(actually, I don't know the --progressive flag, someone who knows it should answer)

likely you'd roughly have 1:2 resolution at that time (perhaps 70 % of the frame and 30 % of the frame in 1:4)

in VarDct you'd have the dc + ~12 % of the 256x256 groups

in VarDCT with passes you might have a decent full resolution image, but with some blurring/ringing somewhere

thanks for the explanation. I'll have to do some digging...or ask Jon then 🙂

Quality paranoia and zooming a lot. Though of course as noted at some point lossless just makes more sense, and indeed I will probably use lossless pretty often even for photographs.

https://tenor.com/view/the-mandalorian-this-is-the-way-the-way-mandalorian-star-wars-gif-18999449

#lossless

did you find a case where -d 0.1 was not enough for you? (or even -d 0.5)

You can play with dd yourself, with the two djxl options to accept partial files

The image you promote lossless with has noticeable visual artefacts

I don't have enough data, but since changing it lower than 0.1 is apparently a simple change I don't see why it shouldn't be there. And note that the word choice there is quite deliberate with *paranoia*. There is always the possibility that I missed some imperfection so I would like to be able to add a bit extra quality just to be sure.

when doing butteraugli testing with 2500 images I never saw a difference at or below a score of 0.6

0.1 is quite a lot of headroom already

consider it quality 155 in libjpeg terms

or maybe quality 120, not quite sure how the extrapolation would go there 😛

haha, ok

most of my testing so far as been with lossless modular :)

though for not seeing a difference, did you do a lot of zooming?

If you start zooming a lot and/or adjusting brightness a lot, then lossless is your best bet

Lossy is meant for a reasonable range of viewing distances, which does not include microscopes

I noticed some difference at -d 0.6 when zooming in a little or looking closely, but at -d 0.3 this did not happen. With -d 0.6 it was easier to see the difference in some kinds of artificial images, but sometimes I could see it in a complex, almost photographic image, for example:

Source

-d 0.6

(chainmail and some other areas, but when zoomed in a bit and flip)

I was using JPEG XL to re-compress some images I saved from a Discord server a while ago and I noticed that sometimes lossless WebP, which I originally used, was smaller so I wrote a batch file that can be used after the conversion to delete whichever image is bigger.

secret: there is a bit of loss caused by the gaborish option which does not depend on the distance at all 😄 (our encoder approximates an inverse convolution on the input, but it cannot be done exactly unless quite a lot of computation is done)

Don't we disable gaborish by default at very low distances?

(or if we don't, why not? It shouldn't be useful anymore at d0.1, right?)

we don't - and I think the main reason is that the quantization matrices are tuned for gaborished input

I see. Does gaborish have big impact on how you want to shape your quanttables?

I think so yes

yes, something like 30 % more quantization on HF

I'm still happy if WebP lossless is useful :---------------)

<@532010383041363969>: Do you want a copy of the conversion scripts I wrote?

-d 0.6 will currently not give (and like never will) a butteraugli score of 0.6, it is somehow related, but not quite 🙂

they used to be hard quarantees in early pik, but diverged about three years ago

No, I'm just the author of WebP lossless now working on JPEG XL. I'm always happy if two of my (co-)creations compete with each other.

That's amazing. I didn't know that.

I'm quite proud of WebP lossless -- I think it got much less attention than it deserves.

I've been using it since (I think) 2016 or 2017 since I store so many PNG's.

of course now after 10 years we know how to do things better already, but still it is performing pretty well

I made it's design in April 2011 to December 2011, Zoltan Szabadka helped me already at that time by keeping the decoder in sync (as a 20 %eer in my project).

Yep, lossless WebP is very well tuned for different types of images and still beats JXL on pixelart, tiny images and some grainy photos, as I mentioned in <#803645746661425173>

It has `-g inf`, that helps

And it treats pixels as RGB(A), not planar, that helps too

Lossless webp is what I used before jxl's bitstream freeze

In some earlier efforts of JPEG XL I tried to guide them towards photography so that it would be clearly a different strength from WebP lossless

but we removed those lossless efforts altogether and built new ones based on the FUIF ideas improved further by Luca

I wasn't excited by FUIF because the initial coding speed was not acceptable to me and I didn't see how it could be sped up

Luca just dived into the code and figured out the ways

"For the Dark Lord to do what we cannot imagine requires only that he has a better imagination." (cit.)

I think we have even more future opportunity there by constraining the context modeling in very special ways that will allow lookups

WebP is old enough to be supported by Picasa but Google probably won't bring it back so it would be good if someone reverse-engineered it and added JXL support.

❤️ Picasa ❤️

I don't know why I don't like photography when the images are in the web, perhaps just not enough patience to wait for them

perhaps one day I get used to it, or the internet gets fast enough (or compression improves sufficiently :-D)

Have you tried digikam?

no, I just archived my camera and no longer take photos 😄

https://tenor.com/view/sad-crying-spiderman-cry-face-ugly-face-gif-5701170

except some with the phone, but it is not real photography

Almost all of my photos are from my phone. The best camera is the one you have on you.

Also some discussion about Jpeg XL https://github.com/OSGeo/gdal/pull/3449

> **boxerab** > Thanks, I'm aware of jpeg-xl, and the lossless compression performance sounds good. But, XL is designed more for lossy web content, performing well at low bit rates. It lacks many of the killer features of JPEG 2000, such as progression by resolution or quality, and fast random access into very large images via TLM and PLT markers. So, the new standard is complimentary to good old JPEG 2000. > As far as I can tell, XL tiles are limited to 64x64. Also, tiling introduces artifacts at the tile boundaries - with PLT markers we get fast random access into large single-tiled images, without those artifacts.

Well, in my opinion it's the only usefull part of WebP. JPEG is way superior compared to WebP lossy for medium to high fidelity content. But WebP lossless and also near lossless is really nice.

What nonsense is that? Ugh...

I added the near lossless already in my december 2011 version. It wasn't understood and embraced by the WebP eng team so they just removed it during the productionizing work.

I was able to convince them later about it and we added it back 2-3 years later

the initial near lossless didn't cope with predictions specially well

Marcin Kowalczyk (riegeli author) made it work with spatial predictions, too

that increased its application area by about by about 50 %

I consider the near lossless the best option in webp, too 😄

we still have that btw, or at least the main parts of it. It's what `-q 100 -s 3` does at the moment. Great speed/density trade-off, just doesn't work well on some kinds of non-photographic images.

re "JXL lossless tiles are 128x128, 256x256, 512x512 or 1024x1024. There is no way to do 64x64 tiles"... How is the actual tile size determined, and would there be a way/use to force a given tile size ?

`-g 0` sets it to 128, default is 256 or `-g 1`, `-g 2` set it to 512, `-g 3` sets it to 1024

(actually default can also be 512 now, if the image is e.g. 300x300 it will avoid the silly tiling)

any way to do this through libjxl ?

probably not, most encode options still need to get exposed in the api

in the gdal use case, given we'll always be encoding 512x512 images, would there be an advantage in setting the jxl tile size to 512?

depends

for compression, 512 is better because there are fewer poorly predicted pixels (at the tile boundaries, predictors cannot 'look over the seam')

for multicore speed, 256 is better because you can do some encode steps and most decode steps in 4 threads

but if you already have parallelism externally, you might be better off with single-tile images

if I was building a system on images now, I'd go with 256x256 tiles

it allows 100+ way parallelism at 4k resolution and 25+ way in FullHD

my thinking is that the core count will go up pretty soon -- transistors are getting cheaper and denser but not faster and not much more energy efficient

so it can be tempting to reduce clock rates and pack more cores -- that leads to energy savings

I'd consider that having that option for parallelism is likely worth 3 % in density -- of course it depends on many things

<:24Rip:567491904179142678> RIP picasa. Anyone remember the Instant Messanger "Hello" that Google bought and killed. It was designed for photo sharing. You could dump a ton of photos in, and it would create tiny thumbnails and send it to the other person, when the other person clicked on one it would resize it to a small preview version, like 800px in size and send that, and if they clicked "download" it would download the full size original. Or you could download all and it would give you a zip of the originals as a direct transfer from PC to PC. It was a great idea at the time, very useful for photographers. with JXL that could be something added into any chat platform now for almost free because of the progressive nature.

Wow, I can't even find any reference to that app ever existing anywhere online. It would have been some time between 2005 and 2012. Probably around 2007 or 2008ish

...Welcome to the Twilight zone 👻

It would be cool if they released a motion JPEG XL VFW/DirectShow codec that could losslessly compress MJPEG videos.

Shouldn't be too hard, if you only want pixel exact lossless, not bitstream exact

(bitstream exact is also doable but requires a new bitstream reconstruction box in the jxl file format for mjpeg)

I don't know Polish, but the content is understandable by using Google Translate (it's about Asymmetric numeral systems invented by Jaroslaw (Jarek) Duda and also used in Jpeg XL) https://debica.naszemiasto.pl/pochodzacy-z-debicy-naukowiec-autorem-jezyka-swiatowej/ar/c1-8133741

(since I don't think we need another channel for news or discussions about JXL, so it will be posted here)

rANS itself is not that revolutionary, it is basically just the LIFO version of range encoding (which is basically the same thing as arithmetic coding), which makes decode faster at the expense of requiring the encoder to buffer.

I think <@799692065771749416> at some point independently discovered rANS when we were playing with the predecessor of FLIF. I think back then we just thought 'meh, somewhat faster decode, much more complicated encode, not gonna bother with this'.

<@794205442175402004> https://github.com/sipa/FLIF/commits/stackcoder 🙂

It's in any case more the context modeling that matters than the actual entropy coding itself, imo.

I think I came up with that and experimented with it in 2004 or so, but at least what I tried back then was way too slow to be useful for anything.

stackcoder, hah I like that name

Anyway, it's a bit weird to see that Duda guy give interviews where he makes it seem like he is the genius behind all the new compression codecs of the past decade.

<@794205442175402004> Does JPEG-XL usr rANS or tANS?

rANS

Are the probabilities dynamic? I remember hearing they're fixed per context.

Yes, fixed probabilities

Whole tokens can be decoded at a time because of that, not bit per bit

Up to 8-bit tokens

Ah and tANS is only bit per bit?

(there is also a prefix coding option, then the tokens can be up to 12-bit iirc)

How does one captialize tANS at the start of a sentence?

I think you can make tANS also for larger alphabets than single bits, not sure

I would expect that you can. It could be a lot faster.

Tabled is just avoiding a multiply, I think

Isn't there a division too?

That gets avoided?

Uh, yes, in the encoder I guess

I meant in the decoder

Right. rANS is a division in the encoder, and multiplication in the decoder.

In software, rANS is fast enough. In hardware I guess tANS has advantages

At the very least, with fixed probabilities you can convert those divisions into multiply/add/shift with precomputed values.

That can be an order of magnitude faster.

Yes, I think we don't do any actual divisions

Ah, nice.

Integer division is something you want to avoid, it's very slow

Yes.

iirc up yo 90 cycles on modern x86_64 cpus.

In the weighted predictor we needed it, with varying denumerator so tricky to avoid, so we ensured it is specced in a way that is actually only an approximate division and can be implemented with a small lookup table and a multiply/shift

The first thing I noticed when starting the collaboration with the google pik team was the very strong focus on decode speed

We had focused on very different things, in many ways pik and fuif were very much complimentary

like CELT/SILK for opus?

Fuif focused on a modular approach to transforms which is all thanks to <@799692065771749416> for doing it like that from the start in flif

Which leads to elegance and universality

Also, have there been any preliminary tests of JXL's adaptive resolution as it may be in browsers and how much better it is than separately encoded images (better for caching, but worse in size, limited in resolutions, etc.)?

https://youtu.be/uzYHXqjGgL4

Pik focused on decode speed and perceptual modeling, which I had mostly ignored in fuif

Threading, perceptual color space and encoder tuning, color management and icc compression and all that were pik things

Combining the two in a way that kept the best of both worlds was nontrivial

I mean applicable to some static resolutions, is there a benefit of using it instead of a separately encoded image if consider the large scale, such as three static resolutions of the same image viewed millions of times (and most likely not everyone will download full size) and probably will be more effective to encode a separate image (because part of the progressive image with this resolution will be less effective in size)

Yes, I think probably something like having a 2000 pixel wide jxl and a 3000 pixel wide jxl will be best, which gives you truncated files that do 500, 1000 and 750, 1500 in a reasonably efficient way, together covering the actual needed widths well enough.

But for that, we need browsers and/or CDNs to be smarter so they can fetch/deliver the right thing, and fetch more at viewport change events etc.

Because this is also one of the very significant features that so far has not been used in other formats (perhaps Jpeg200 could, but it is not supported in all browsers) but how much will it work in practice

Eric Portis did some tests in that direction with the old jpeg, but that is limited by the zigzag order in the kind of progressions it can do effectively.

in the beginning I didn't believe that they could be combined in a useful way -- Luca proved me wrong

At lowest bitrates the progressive DC is much smaller and better looking than a traditional DC model

I don't think the progressive model we have in VarDCT is well suited for serving the same image for FullHD, 4k and 8k

with modular we might get closer to that, but I have doubts that this will be a practical thing to do

modular is perhaps a 60 % hit in density to VarDCT and stopping the progression at a non optimal 2**n resolution might be too limiting

it can be awesome for thumbnailing and early previews, but I don't believe that it solves the resolution problem universally and efficiently

(my personal viewpoint :-D)

The progression in VarDCT is probably effective for 1:2 and 1:1, which halves the number of variants needed for responsive images

You could do 1:4 too but then it starts sacrificing density a bit much.

The trade-offs are complicated with this, sending everyone a tailored 1:1 image is ideal for density, but a nightmare for CDN caches

yes, I love the passes-thing in VarDCT even though I wasn't driving it myself -- I just failed to understand that we need it, and I was too focused on the 8x8 + final idea based on my own fixations and beliefs of how best performance and user experience is formed

my back of the envelope calculation was that perhaps it takes one second to load a desktop full of images, and the 8x8 preview can be shown at 100-150 ms when done well

I didn't think we'd need intermediate updates at 500 ms or so

but now with the LCP discussions where the LCP folks are not willing to consider 8x8 as good enough to trigger LCP, we need the passes approach to propose for an earlier LCP trigger

For non-photo, squeeze tends to be bad for density compared to non-progressive lossless. But that's with full squeeze. I wonder what happens if we squeeze only to 1:2. It might be a lot better, and could be a good way to do two resolutions with the same file, or get faster LCP for large non-photo images too

So, with images, would it be possible to do heuristics to detect how many colors there are in an image, and if the image is monochrome, apply --palette=0 and --lossy-palette(if lossless for BW images)? I'd assume it'd be expensive to do an exhaustive search to look at each tiles' color info pixel per pixel, unless I am missing something.

There are cases where palette=0 generates larger images for grayscale than the default. But I would also be in support of a better heuristic resulting in more optimal settings for monochrome images 👍

jpeg xl 0.3.2 will merge some encoder improvement? or only bug fix?

Just a quick bug fix I think

I understand, Thank you 🙂

bug fixes are encoder improvements... though I think 0.3.2 is just a decode fix?

There was a typo in icc profile storage that caused both encoder and decoder to do the wrong thing

Which caused the automatic testing to miss it

So the 0.3.1 encoder was in some cases producing invalid output that only the 0.3.1 decoder would decode, and the 0.3.1 decoder would not decode some valid inputs

That's why a quick fix had to be made

I think it would only happen with non-standard icc profiles that actually require storing a compressed icc profile. In most cases, that is not needed - we have an enum that can represent the most common cases like sRGB, Display P3, Adobe RGB 1998, etc in just a few bits instead of needing to store the exact icc profile.

At least when doing lossy encoding. I think in lossless encoding it happens more often that we store the exact icc profile, because there are several variants and they are not exactly identical, which does not matter much for lossy, but lossless is lossless.

Hi All. Is there a way with CJXL 0.3.1 to do YUV420 subsampling like AVIF does ?

I don't think so. What do you need it for?

I tried to test JXL for tiny thumbnail preview encoding (40x40 under 1kB) and AVIF give better color result with YUV420. I wondered how to do the same with CJXL

you should probably update to 0.3.2

but if you really need subsampling you can probably just use mozjpeg and then losslessly recompress with cjxl, though it's not the best idea...

if your source material is RGB, YCC 4:2:0 should *not* look better color-wise

This is unaspected

a typical 40x40 file should fit in 1kB very very easily even at super high quality

``` veluca@veluca ~/jpeg-xlm/build master $ ./tools/cjxl /tmp/x.png J P E G \/ | /\ |_ e n c o d e r [v0.3.0 | SIMD supported: AVX2,SSE4,Scalar] Read 40x40 image, 4.2 MP/s Encoding [VarDCT, d1.000, squirrel], 4 threads. Compressed to 835 bytes (4.175 bpp). 40 x 40, 0.32 MP/s [0.32, 0.32], 1 reps, 4 threads. ```

4 bpp lol

smaller stuff is harder to compress 🙂

Less entropy to exploit, I guess

Yep but 4 bpp is under 1KB, and avif have a overhead of 200 B, I aspect jxl be very good

that, and constant overhead that doesn't matter for 20 MP images matters for 1KP images 😛

In tiny image how good is modular?

for that image, lossless modular is ~4k

-Q 90 is 993 bytes, and it should be good quality - but likely not as good as the default option

Vardct much better

In my test Q90 is similiar d1 but 15% bigger

40x40 ... likely better to go with 32x32 or 64x64 (or even 48x48)

40x40 forces the system to use some 8x8 transforms

if you can afford 1 kB, go with 64*64

VarDCT should be rather efficient without YUV-trickery, especially so when the input is not YUV420 spoiled.

if someone else already YUV420 spoiled the input, then you take a ~10 % hit in quality by not repeating the YUV420 mistake again 😛

the quantization matrices are another more efficient way of doing the same thing

just they allow more elegant control of the compromises, where as YUV420 is a big hammer that reduces data without looking at it

Jarek Duda advanced our field and formulated those advances with clarity. Let's be thankful for his contributions.

for me it was (back in the day) interesting to find that on most cpus the floating point division and sqrt run practically the same algorithm and tend to be equally fast

I had expected sqrt to be a bit slower

integer division is slow? compared to what, floating point? <:WTF:805391680538148936>

Compared to any other instructions, like multiply

Division by a small integer constant usually gets compiled away to a multiply/shift, but division by a variable int is something you want to avoid in an inner loop, e.g. for every pixel.

is jpeg xl good for web images?

like to me it seems only that the colour change

i don't see evident change in color space, noise, colour, image quality

like it looks like jpeg

webp was a revolution

this seems like a work in progress

like for compress screenshot even at q15 sometimes it looks same as original but value will change

but does really beat webp2?

like for mp3 they don't use jpeg xl, pdf not, web not, android not, softwares icon not

like i see changing of the colors in the contours but nothing impressive to me

over 1 bpp

if you encode a jpg it trascode lossless without change

also we don't have a change that dev will make new version and will contest MPEG patents not because they work at img compression but because they are part of mpeg

so they wouldn't contest themselves and risk to quit job

at least they don't hide about open source

hope they will be new versions of jpeg XL

webp end avif are good at low fidelity but over 1 bpp the can't mantain texture and details

on lossles jxl win on webp and webp2 (avif is bad on lossless)

is there a change webp2 will keep invisible pixels?

will work in this way?

or even webp discards invisible pixels?

and is meant for doing that? the webp algorithm works like that? sorry for off topic

jxl is open source

webp is only 4:2:0, it discard a lot

do you have image compression experience?

amateur

you can compare on medium and high

https://eclipseo.github.io/image-comparison-web/#abandonned-factory&AOM=s&AOM=s&subset1

on texture details and dark zones

and sky texture

on some image terrain's texure are better on jxl low then avif or webp2 high

is it possible with 0.3.0 or 0.3.1 to use jxl as input

and jxl as output

webp, webp2 and avif are very good to make the img look "good" bat with low fedelity on original

webp2 is better then avif

yes 416b489 is great

but low fidelity obviously

no, you must use djxl to remake trancoded jpg or pixel png

for all images, modular, vardct, jpg to jxl (-j), jxl losless recopressor (jpg)

<@!532010383041363969> is working to make better on very low bpp

how many colours -s 4 -q 99.2 vardct discard on 0.3.0?

how to know?

is not fixed, i take the distance from butteragli metric

is better use d flag then q flag

d is the distance from butteragli, q is remapped on d

d 1 must be similiar q 95

d 0.6 is similiar lossles to defoult zoom

with 0.3.1 is worth -d 1 -s 7 -s 8 -q 68.8 -s 8 -q 96

d 1 is actually q 90

d 6 is similiar q 40-50

it change for some img

ok,

i use -s 8 or -s 8 -d 0.6