JPEG XL

Could probably also do strips of 2048x256 within those tiles

But I guess 4 megapixel units should be good enough

We designed the bitstream to be ok for huge images, but we focused the implementation on the web use case for now, since we think it is most critical for adoption

sure, it's a marginal thing, I agree anyway, thank you for your help

Well it's something I want to have too at some point, especially decoding in chunks

something like libjpeg shrink-on-load would be very handy too I see you can read out the DC components, is anything more on those lines planned?

For now we focus on what is needed for progressive decode (again, the web use case), which is not entirely the same as shrink-on-load but similar

yes, we'd like shrink-on-load to help with eg. fast thumbnail generation

1:8 is always first in the bitstream (at least in the vardct case), so for thumbnail generation that is often good enough

yes, that sounds great we'll probably merge load and save for the next release and tag it as experimental thanks for this nice thing btw

Are you a libvips dev?

yes, it's my baby

Cool

Libvips looks like it could be useful for Cloudinary's use case too

that’s very nice, I had come across it before and it seemed very appealing

it made me wish I had ideas of stuff to implement so that I could use it 😄

We are currently doing a lot with ImageMagick that could probably also be done with libvips

is cloudinary still using graphicsmagick? yes, libvips ought to be quicker and use less memory, though of course you can make benchmarks fall either way

We use imagemagick for lots of things, and my own code for some of the hot paths

I use opencv too for some stuff

oh, nice

opencv used to have rather slow read and write, though it was very quick once you had the image in memory I've not timed it for a while now

I don't use the opencv image format stuff, just things like convolutions are quite fast in it

I think libvips convolutions are a bit quicker, but again you'd need benchmarks etc. libvips does run-time AVX2 code generation for conv

oh, and it's threaded too, of course anyway ... off topic

For image loading/writing I just use libjpeg-turbo/mozjpeg directly. For PSD and TIFF I wrote my own decoder to do downscale on decode

We do everything single-threaded, we have enough image-level parallelism anyway

We decode/process/encode about 5k images per second on average iirc

wow that's a lot!

would image-level threading lower latency, perhaps?

It would, but latency is only for first request anyway

So most customers don't care that much about it, as long as we can keep it below one or two seconds

Subsequent requests are cached with several layers of caching so they get fast responses anyway

For video we of course need to parallelize, but for images so far we don't

if latency were much lower you'd need less caching, which might translate into infrastructure savings I guess

Caching is cheaper than cpu costs

Also to get good latency (on most requests) you need caching close to the end user, but CDN edges don't have enough compute to do what we want

I think that depends -- storage can be expensive one of those tricky things you have to look at in detail to make a calculation

true, you certainly don't want edge nodes doing anything serious

Yes, there are many trade-offs and it's tricky to make good decisions

We have considered going multithreaded for avif encoding 😅

haha I bet!

avif is a beast

Huh, that's an interesting bug report: https://gitlab.com/wg1/jpeg-xl/-/issues/211

I don't have that issue with Clang/LLVM12 on OpenMandriva latest.

Are we still in an active development phase? The last release was almost one month ago. <:ugly:805106754668068868>

they are working on chromium

Stuff is still happening yes, just nothing urgent/important enough to do a new release for

Also this week it is JPEG meeting week, which keeps some of us like me quite busy

Can't wait til next sync with the jxl_from_tree improvements <:BlobYay:806132268186861619>

How long until standardization?

Standardization of what?

Final published IS

ISO/IEC 18181 part 1 (the jxl codestream) is basically done: the FDIS text was ready in January, we are now waiting for the ballot results which we will probably have in July if all goes well. Then before it actually becomes a published IS depends mostly on the speed of the ISO editors and the process I guess, but in practice it doesn't matter that much: after FDIS no more technical changes can be made so the spec is final (modulo later corrigenda, revisions and amendments, but that is always the case).

So sounds like autumn to me <:Poggers:805392625934663710>

Part 2, the file format, i.e. how to add exif/xmp metadata to a codestream and how to reconstruct a jpeg bitstream (not pixels but a bit-exact jpeg file), we hope to advance to FDIS in the current JPEG meeting, so to be approved on Friday if all goes well.

So part 2 will probably be 2022 already before it's a published IS, but technically we are in agreement on what it will be and the current implementation already does things that are conforming to what will be the final part 2 spec.

Part 3, conformance testing, is currently only a working draft and we hope to advance it to CD (committee draft) status at this meeting. If the CD ballot approves, we can then move to DIS and FDIS and IS.

Part 4, reference software, was approved as a CD and we hope to advance it to DIS stage at this meeting.

In practice, the adoption can start now, it does not need to wait for the slow ISO process to complete. Before FDIS of part 1 it was premature, since the bitstream could still change, but since January it can be used in practice.

I was just asking because some people think if it's still a draft standard, people will think it's not ready'

well, technically it is a (Final) Draft International Standard

but it is perhaps not obvious that no technical changes can be made at that point

Adoption isn't going to take off if people can't get the spec. Can they add it to https://standards.iso.org/ittf/PubliclyAvailableStandards/index.html ? Is that a topic for this meeting?

we have brought it up but so far the answer seems to be no

So the spec will always be behind the ISO paywall, or are there going to be different versions of the spec from different organizations?

Just interested

not understood, it's the ISO or the jpeg that wants the spec to not be public

It's ISO

but jpeg why has iso standards

even vp9 has it?

So it can have a standard

It needs it for adoption and just to be standardized

vp9 I'm almost sure has it

what it iso, what works do Jon for that?

It's just how they chose to standardize, Jon can probably say why, it probably has something to do with it being a jpeg format

or a vp9 developer

what they did

I don't quite understand what makes it possible i.e. for hevc to have a public standard and for JPEG XL not to

anyway, the code of the reference implementation is a sufficient approximation of a spec for now IMHO - and in the longer term, we'll likely write something outside of ISO itself that can be used as a spec...

dou you all attend ISO

or conferences

13 hours only Jon

or also others

Jon said he was doing a lot of jpeg conferences but idk about ISO

like veluca

or every nationality and every member

who has created the standard

as far i read

JPEG is Joint Photograpic Expert Group, where Joint means it is part of ISO and a bunch of other things

ok

also you attended 13 hours of conferences

ISO, IEC, ITU

Well that explains my question about why you standardized via ISO ( :

or did you have permission to leave earlier ??

how does it? 😛

there was a call for proposals from ISO a while back, PIK and FLIF decided to participate and join forces and make JPEG XL

(it's a tl;dr)

I was not involved yet when this happened (I started working at Google and on JPEG XL in September 2018, the call for proposal was before that)

I mean jpeg is a part of ISO so it makes sense that a standard from jpeg would make their standard with ISO

yes, of course

but the interesting part of the decision was deciding to make it a JPEG standard

Ohh

I guess maybe for the jpeg name

Can we make a FLIFv2 that adopts all of the changes made by JPEG XL from its FLIF origin, and then publicly release that spec?

probably at least in part 😄

Since it's supposed to succeed jpeg

And PNG

we can make a public not-officially-a-spec of JPEG XL with no issues

except for the time it would take to write it (tl;dr: a while xD)

no jpeg xl is enough for at least 10 years

and will be for the public

I wrote a fairly large chunk of the JXL spec (of course with lots of help from others, but still), and let me tell you - not looking forward to doing it again

having said that - the next ISO meeting is at 6 am tomorrow morning, and my brain requires sleep 😆

time to sleep indeed, again a 6 am session tomorrow

`jpeg-xl-ab7c5e9b-mingw64-avx2` <https://encode.su/threads/3564-JXL-version-0-3-released?p=69454&viewfull=1#post69454>

how do you actually install jxl after compiling from ci.sh?

like the equivalent of `make install`

figured it out after reading more documentation myself

after running ci.sh you go to the build dir and use `sudo cmake --install .` this command

<:Hypers:808826266060193874>

`make install` should work too

Nah, ci.sh doesn't make a makefile

For some reason

But it does make the cmake files needed to install it with the above command ( :

you can also do `ninja install` 😛

(ci.sh uses ninja instead of make, it is slightly faster to figure out what to rebuild IIRC - also less verbose)

interesting

Sigh

vcpkg is complaining about there being no visual studio instances.

When I have visual studio installed.

This is why I hate compiling things.

Oh, thanks for that link, Scope.

Well, my processor didn't seem to like that build.

It's late, I'll figure it out in the morning.

<#822120855449894942>

(there are also non-AVX2 builds)

Yeah, that's probably why it didn't work. Didn't notice that was AVX2.

unless it's avx2-only it ought to work though

Oh, that's the Firefox guy who's working on JXL: https://gitlab.com/wg1/jpeg-xl/-/issues/226

Yes, related. Maybe the issue was found when adding a test example to firefox...

but like I answered, this isn't actually an issue, you just cannot assume a jpeg decodes to the same pixels with every jpeg decoder

(this is one of the reasons why I don't consider "JPEG -> PNG -> lossless compression in whatever format" to be an actually fully lossless thing to do)

Well, we do have a very different API than libjpeg-turbo, so we could certainly not be just a drop-in replacement for libjpeg-turbo where applications that support jpeg via libjpeg-turbo would suddenly support jxl too. That wouldn't work for various reasons, even if we would spend the effort to mimick libjpeg's API, since: - some applications use the low-level api and expect to be able to get access to the raw DCT8x8 coefficients etc (something that doesn't exist in the case of generic VarDCT or modular) - applications would only get 8-bit RGB, no alpha, no high bit depth, no animation

What we _could_ do however is to make the libjxl decoder also take JPEG bitstreams as input, and decode it to pixels in the same way as if you'd recompress it as jxl and decode that to pixels (but of course without the intermediate step of doing the recompression). That way, applications that support libjxl would no longer need to link to libjpeg(-turbo) to be able to load JPEG files.

Assuming a single libjxl for both encode and decode, that would come at nearly no cost in binary size, since the jxl encoder needs to be able to parse JPEG input anyway (to do lossless recompression). Only if you want a decode-only libjxl_dec, it would add a bit to the binary size of the decoder if it also has to handle jpeg inputs directly. But still a lot less than having to ship/statically link to both libjxl_dec **and** libjpeg_turbo.

https://jpeg.org/jpeg/software.html

they are currently actually making ISO/IEC 10918-7, 2nd edition that contains some version of both of these, so they are actual reference implementations

but ISO of course doesn't want to be sued over a bug, nor play helpdesk for people who have some compilation issue or something

those are as official as it gets

see https://jpeg.org/jpeg2000/software.html for an example

(I suppose they will soon add "Reference software (active)" in the "Notes" column for both libjpeg-turbo and libjpeg)

for JPEG 2000 there is one active reference software (OpenJPEG) and two old ones (JasPer and JJ2000) that are no longer maintained afaik but they are still considered reference software (described in part 5 of the j2k standard, I assume)

and then they mention Kakadu and ffmpeg on the software page but they are not official reference software

only time will tell. At the moment I see no reason for forks or maintenance stopping, I think the current libjxl devs have pretty much aligned goals and targets for it

libjpeg-turbo was a fork specifically aimed at speed, I don't know why they couldn't just contribute the simd patches directly to the original IJG implementation but maybe they didn't want them (sometimes slow and simple code has its advantages too)

mozjpeg was a fork specifically aimed at better but slower compression, and it could not be done without forking because libjpeg-turbo is aimed specifically at maximum speed and doesn't care (much) about better compression

then there's Thomas Richter's libjpeg (https://github.com/thorfdbg/libjpeg), also confusingly named libjpeg but not related to the original IJG libjpeg, which aims at completeness: it actually implements everything in the JPEG spec, including exotic things the other implementations never bothered to implement.

completeness/correctness, speed, and good compression are to some extent contradictory targets, and that naturally leads to differences of opinion and possibly forks

(there's also the story of the IJG libjpeg where they at some point started adding new stuff that isn't in the spec, pissing off everyone else)

Anyway, for libjxl we aim at providing several trade-offs between speed and compression (so that is unlikely to lead to a fork, we just have encode stuff disabled at faster speed settings if it's too slow), and we also aim at being complete, because if there is one lesson to learn from the old JPEG, it is that partial implementations lead to *de facto* standards that are just that subset of the spec — if there had been complete implementations of JPEG (and a not-hastily-designed file format) from the start, then things like lossless JPEG, 12-bit JPEG, JPEGs with alpha, somewhat better compressed JPEGs with arithmetic coding, etc would all have been there all the time (those are all in the 1992 spec!) - probably even making PNG redundant.

So I do not anticipate any major disagreements on what direction to go with libjxl.

But you can never predict the future of course, who knows what will happen in 20 years.

it fail, cjxl fail also with cmyk jpeg

We fail on anything that is inconvenient/impossible to do with VarDCT

Please make this happen! 👍 🤩 👍 libjxl produces noticeable less banding than libjpeg on many JPEGs.

(which means: CMYK we don't do because varDCT expects 3 color channels, weird subsampling factors like 3x subsampled we don't do because it doesn't happen in the wild and we haven't implemented it, and things that are in the spec but most decoders don't decode we also don't do, like arithmetic coding, hierarchical, 12-bit, lossless, etc. Nearly all JPEGs that exist in the wild (with exception of CMYK ones) can be losslessly recompressed though.)

maybe open a gitlab issue with this as a feature request for it, to remind us. I think it's a good idea, but it needs to be discussed because there are also potential downsides (decoder binary size grows, attack surface w.r.t. security grows, etc).

but now that you set 1 gb limit for RAM in web jxl

what happen when you load a 7000x6000 reddit image on reddit

for decoding i used 1,2 gb ram on modular

with 6000 weight

it is alllowed to refuse decoding it

with vardct it will use at least 2 gb ram

that's on sse4.1

and people who have new computers?

all basically?

how they will watch gs on reddit

is already decided

can't we make a petition

the layers/profiles don't care about actual decoder memory, but on general properties like resolution

ah thanks

it is kind of always the case that if you don't have the memory, it won't work.

with jpg there is an imposition to resize the size

we don't want resize the size

we want full size

wallpapers

images

whatever is

7000x5460 at least

what is limit on ram and resolution?

?

Level 5 is limited to 2^18 pixels in width or height (262144 pixels), and 2^28 total pixels (268 megapixels).

Level 10 is limited to 2^30 pixels in width or height (a gigapixel in just one row or column) and 2^40 total pixels (one terapixel)

Since the current encoder/decoder don't do chunked encoding / cropped decoding yet, with the current implementation you need at least ~4 (and usually more like ~7) bytes of memory per pixel to be able to decode an image.

so for a terapixel image you need ~7 terabytes of memory, which obviously nobody actually has

Do you anticipate encoder trade-offs for memory vs cpu time in the future?

ah ok so no problem

we'll see 42 mpx images

<@794205442175402004> So with the current codebase, you can't write a level 5 compliant decoder with less than 2 GIB of RAM?

Compliance tests will not include such huge images, and for a (software) decoder it is fine if it depends on available memory, it is just not allowed to flat out refuse to decode an image because it is "too large" even if it does have the memory to decode it

fair

It is allowed to refuse to decode a 300000x1 image if it wants to, even if that's not a big number of pixels

(and still conform to level 5)

and mobile

what they will do

In practice, likely in browsers they won't actually check any size limit, and maybe just fail to allocate the memory and show a broken image or a crashed tab

likely what resolution will load reddit mobile

if there is no webp copy

what they will do

load a 42 mpx jxl

and use 2,5 gb of ram

not sure if now is less with new 0.4.0 branch

Most websites that take user uploaded images enforce dimension limits of their own

reddit i have seen 42 mpx images

you can upload italy

42 megapixels are 126 megabytes uncompressed whatever format you use, and likely 168 megabytes since usually rgba buffers are used

(for HDR you can double that number)

so jxl get coded to png

before reading it

It is not very different between jpeg, png and jxl

or it needs only for editing

UNTIL gimp plugin comes

how it works

I do not understand what your question is.

when someone opens a jxl in chrome

what happen in %appdata temp

does it creates a png file

what happens with gimp

what happens with other software

what happened with old jpeg

what is the standardized behaviour of jpeg, jpeg xl

When you open a jxl in chrome it decodes it to memory

Creating a png file is more effort than decoding a typical jxl

ok

the gimp plugin is already existent, though i don't think it can write unless you compile with a specific flag

i think file format signatures as used in that plugin are outdated

it was able to open my jxl files okay

FYI ... libvips has merged jxl support to master and it's now being fuzzed! no bugs found yet, I guess you've already fuzzed a lot jxl support should be a feature of libvips 8.11 (due in a month or so), and then appear in downstream projects like sharp etc. no animation support yet, but most other features are exposed (I think)

yes, we're doing quite a lot of fuzzing ourselves too 🙂 but more fuzzing is always good!

I don't know if it offers more than fuzzing with Google's internal fuzzer 😛

(it's probably the same thing)

(to the tune of "Smelly Cat") ♫ Fuzzer-bug, fuzzer-bug What are they feeding you Fuzzer-bug, fuzzer-bug It's not your fault ♫

Is it possible to do a lossless transcode from JPEG to XL plus also adding noise=1 to the JXL?

Or more generally asked: Could the noise parameter be changed by adding/removing a few JXL bytes?

If I understood correctly (I think <@!179701849576833024> talked about it), then it is simply a signal for the decoder to display noise.

it tells the encoder to, in lossy mode, try to figure out a good noise model for the image

Is it possible to simply add the noise model of a specific JXL to another JXL?

not unless you change the source code yourself atm

So yes, bitstream-wise, and no, current tooling wise.

(I probably should have started this conversation in another channel.) I would just like to try out how noise would look on a transcoded JPEG. Is there a way to detect the appropriate offset where generic noise can be added to such file (when only having the bitstream)?

Editing jxl bitstreams in a hex editor is near impossible, because headers are not byte aligned and data is ANS encoded (so changing one value often causes many bytes to change)

It will be easier to just hack the encoder code to override noise with some hardcoded values.

(not sure if noise in non-XYB is allowed/implemented, <@179701849576833024> do you know?)

allowed, yes. implemented, also (at least in the decoder). will do something sensible... eh...

probably yes in YCbCr, but otherwise, I wouldn't bet on it

RGB is not _that_ different from XYB (it just has a poor man's luma that is just G, and a weird X that is R)

A Hamming distance of 2 is practically nothing! 😄

yes, libvips is fuzzing libjxl on google's oss-fuzz: https://github.com/google/oss-fuzz/blob/master/projects/libvips/build.sh#L147-L185 no more features than the usual clang fuzzers afaik, but they throw a lot of cpu at it and have nice tooling etc.

it's just an experiment, we only added libjxl fuzzing two days ago (I think)

I got a question about security multiple times: is libjxl being fuzzed?

Oh yes

Still finding fuzzerbugs too, though I do have the feeling that the cases are getting more exotic and we will soon be at a point where it becomes hard to find another fuzzerbug

branch 0.4.x should be the "more stable" branch going forward, where we include new commits etc only if they've been fuzzed for a while or are bug fixes

(well, until a 0.5.x branch exists)

git master is failing to build for me on ubuntu 20.10, perhaps it's getting confused between the system brotli and the built-in one? I see: $ cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=OFF .. ... $ cmake --build . ... [ 73%] Building C object third_party/brotli/CMakeFiles/brotlidec.dir/c/dec/state.c.o [ 74%] Linking C shared library libbrotlidec.so /usr/bin/cmake: symbol lookup error: /lib/x86_64-linux-gnu/libbrotlidec.so.1: undefined symbol: _kBrotliPrefixCodeRanges gmake[2]: *** [third_party/brotli/CMakeFiles/brotlidec.dir/build.make:130: third_party/brotli/libbrotlidec.so.1.0.7] Error 127 ie. it's trying to link against the system brotli for some reason I don't understand

Hey, have Google ever hit this build problem?: https://github.com/google/highway/issues/183 I think Chromium is also in minor platforms

never got that, but AFAIU it's because hwy is only implemented for x86, ppc and arm (and wasm, but whatever) and we don't really have a "fallback to non-simd" that works by default

Hmm, the patch is already there, so it should be good 🚀

So the request is: could you update highway when you sync next time? Thanks 😉

I had a ":" at the end of my LD_LIBRARY_PATH and cmake got confused ... sorry for the noise

How do I assign a dev to the issue? 😁 https://gitlab.com/wg1/jpeg-xl/-/issues/232

wow it really looks perceptibly better

I noticed something funny about libjxl lcms usage libjxl isn't setting the cmsFLAGS_NOCACHE flag, which (I think?) means you will see races there's a comment about lcms races in https://gitlab.com/wg1/jpeg-xl/-/blob/master/lib/jxl/docs/color_management.md#cms-library would adding the flag possibly fix this? note that lcms itself sets this flag when it calls cmsCreateTransformTHR(), so it would seem to be recommended

well, we're using skcms anyway in most code paths, so I think that's not too much of a problem 🙂

Here's lcms setting the flag: https://github.com/mm2/Little-CMS/blob/master/src/cmscnvrt.c#L1029 the cache being referred to is a one-pixel cache, which can help single-threaded applications, but which will race if you try to use the same transform from several threads

ah, ok ... as long as you never call cmsDoTransform() from several threads I'm sure it's OK

ah, yes, I believe we workaround that in some way

jolly good -- just a thought

bonus: djxl --bits_per_sample=10 (and converted back to 8 bpc)

zero banding :O

<@&803357352664891472> How is this possible?

8bpc PNG to lossy 8bpc JXL to 8bpc PNG looks worse than 8bpc PNG to lossy 8bpc JXL to 10bpc PNG

maybe it's just me, but I see quite a bit of banding 😛

So JXL knows that the original PNG contained dithering or noise but won't put back this information when decoding back to the same bit depth? But is fully capable to undo SDR dithering/noise and produce true high bit depths upon request? 😲 ???

We don't do dithering when producing 8bit output

Download the original.

yeah, in the original

Lossy jxl is not 8bpc internally

It's XYB and 32-bit float internally

I can see very much of banding

The original has less

But it still exists

I tried -c 1 and saw the same result. So JXL stores 8-bit int images as 32-bit float and still manages to save so much space? <:WhatThe:806133036059197491>

-c 1 doesn't do anything when recompressing jpeg or doing vardct

Unrecognized token IDENT (`doesn`) in expression

Lossy compression does not store pixels

It stores quantized dct coefficients

What it actually encodes are mostly signed integers close to zero

But after dequantization things are done with float precision, and gradients can be without banding

Would there be a noticeable difference in size when dequantization would use 8-bit internally? Or would it just look worse?

That would just look worse and have no impact on file size, since the bitstream only contains the quantized values

Dequant is something the decoder does as part of the decoding process

cjxl sky.png sky.jxl --> 16 kiB

cjxl sky.png -c 1 sky1.jxl --> 38 kiB 🤨

Ah if it does something then it is probably something broken

Looks fine to me, the bands are slightly different.

I wonder what it does. VarDCT in RGB instead of XYB? I'm surprised that it even works

it does work, it just doesn't compress well

It just uses the XYB quantization constants?

Does it end up doing things in linear RGB or in whatever the input space was?

<@!179701849576833024> <@!427945413669158912> Here is the direct link: <https://cdn.discordapp.com/attachments/804324493420920833/840283238223577119/10.webp>

I looked at the original

There's slight banding

yup

Difficult to see on my phone

input space

Sorry, then it's your display's fault ;)

<@!794205442175402004> Will the result be the same as dithering if the decoder would add a small bit of noise for every JXL that didn't use --noise=1?

Not really the same thing

Dithering is just to render stuff nicer at 8-bit, currently we basically just round every pixel value to the nearest 8-bit integer when converting the internal floats to ints, which introduces slight banding that can be avoided with dithering (just alternating between rounding up and down in a checkerboard way would already help a bit)

Floyd-Steinberg?

But if you add noise to the 32-bit info, trivial rounding to 8-bit wouldn't be an issue.

Yes, adding noise is one way to do dithering

Doesn't need to be fancy dithering though, it's not like we're converting stuff to 1-bit

https://c.tenor.com/QPapotlLW18AAAAM/dance-cute.gif

We may have the first fully feature-complete decoder on the private dev repo now, with the commits <@179701849576833024> added today

wait like with progressive, hdr and animation support?

libjxl already supported those things, there the issue is in making the chrome integration also support it

Feature-complete in the sense that it can decode everything that is in the spec, not just the things the current encoder happens to produce

This is an important stage, it is needed to be sure a decoder will be able to decode anything a future encoder might produce

is there test file repo for all these features?

We have backwards-compatibility since end of December in the sense that what the encoder produces is spec-compliant so remains decodable.

But now we (hopefully, if we don't discover any bugs/missed cases) have forward compatibility too

i mean you guys probably already test on some kind of actual files even if theyre made by hand

also <@!604964375924834314>

Building such a test file repo and ensuring that it is covering all corners of the spec is one of the mandates for the jxl adhoc group to be done before the next jpeg meeting.

most of those more exotic features are hand-tested 🙂 there are a few things that have actual bitstreams in, but not everything

jxl art might actually be a way to add more bitstreams

Win win: more art options, and more conformance bitstreams :)

Splines would be nice to see

well, a splines encoder would be hard, but a way to manually specify weird patch types and splines would be very very useful for testing...

ooh, jxl art as test files

I took a quick look at jpeg-xl's use of lcms. It seems that a transform is created for each thread to allow thread-safe usage. <https://gitlab.com/wg1/jpeg-xl/-/blob/040eae8105b61b312a67791213091103f4c0d034/lib/jxl/enc_color_management.cc#L869-881> <https://gitlab.com/wg1/jpeg-xl/-/blob/040eae8105b61b312a67791213091103f4c0d034/lib/jxl/enc_color_management.h#L59-60> Probably that could be replaced by creating a single transform with the `cmsFLAGS_NOCACHE` flag, similar to libvips. <https://github.com/libvips/libvips/blob/41cff4e9d0838498487a00623462204eb10ee5b8/libvips/colour/icc_transform.c#L361-L373>

<@!604964375924834314> <@!768090355546587137> are the ones that take care of that I believe 😛

yes, what I recall is a `thread_local cmsContext`

ah, or that vector, yeah

would replacing it with a single transform with cmsFLAGS_NOCACHE make things easier in any way?

now that it’s already written anyway, not sure

It could remove that `transforms_` vector and some unnecessary `cmsCreateTransformTHR` calls, which might speed things up, otherwise there is no real benefit (I think).

Can JPEG XL become an alternative to DNG in case of original, non-demosaiced data?

It should be theoretically possible (it's been discussed before), but it was just an idea back then.

Simple case of RGGB Bayer could be handled by saving R(G1+G2)B at half resolution (either lossless Modular or lossy VarDCT for data savings) and G1-G2 in a separate Modular channel (which could also be optionally lossily compressed). There's a problem of metadata since we have to know the original CFA order (RGGB, GRBG etc.), but it still shouldn't break forward compatibility.

And there are other CFAs, like Huawei's RYYB, Quad Bayer, Samsung's Nonacell, Fujifilm's X-Trans, ...

How should we decide what we put into visible RGB data (which pixels do we average and how much are they downsampled)? X-Trans can be naively downsampled into RGB in multiple ways and we have to arbitrarily decide which one to use.

What about partially non-RGB sensors (RYYB, RGBW, RGBE) or completely non-RGB sensors (CYYM)?

<@&803357352664891472> sorry to ping you all, but I think this problem should be solved before claiming JPEG XL fully frozen and forward compatible.

Just to clarify, "fully frozen" is for web usage scenarios, where I don't quite think debayering patterns are exactly common (we don't ever plan to support that in Chrome, say)

We can always make something like a second edition that includes ways to express interesting debayering patterns or stuff like that, as long as it is intended to be used in non-web scenarios (where I believe fwd compatibility is a lot less important)

CMYK and Bayered data are not for the web, we can add support for those but it will not be level 5 and simple decoders will not be required to decode those exactly right

Let's just hope that they'll be at least *somehow* decodable in Chrome. With lowered resolution, maybe a bit worse color and brightness rendering, but it should show *something reasonable*.

And thanks for the answer.

why do you want them? it's not like chrome can render DNGs, can it?

JPEG XL could compress raw files more and with better quality while having to provide fallback RGB image (because of it being directly involved during decompressing back to Bayer, see: my "Simple case of RGGB Bayer [...] shouldn't break forward compatibility" wall of text above).

I see. Yeah I can see how it can be possible. Just not sure whether there's a lot of use for viewing them. For previewing stuff in browser before downloading to do further processing?

Would need some additional metadata to indicate that there's extra stuff to complement image. But I think it could be done. Just not sure whether that hackyness is worth it. But it's not for me to decide.

We need a DNJXL file 😄 i'm agree, only using metadata without file format normalisation cannot be a good solution. The easy way is to force adobe to add jxl inside dng 😄

So, I think I might have found a bug related to butteraugli...

You see, aomenc-av1 now has `--tune=butteraugli`

It seems to segfault after a certain number of frames when activated no matter what.

I get this interesting atomic issue that is extremely similar to the one found here if I enable debug mode in aomenc: https://gitlab.com/wg1/jpeg-xl/-/issues/85

`/usr/include/c++/10/bits/atomic_base.h:217: void std::atomic_flag::clear(std::memory_order): Assertion '__b != memory_order_acq_rel' failed.` <:Thonk:805904896879493180>

Yes, that would happen. The deltaG channel would be ignored by applications like Chrome that do not know how to deal with it, and you'd basically get a 1:2 image.

Is that with the same butteraugli code as in libjxl, or an older version of it?

Same actually, with the current master build.

One thing I'm noticing is that it's not clearing memory with each encoded frame, and the encoder just starts running out of memory and then segfaults randomly<:Thonk:805904896879493180>

Memleak? Hm probably best to open a gitlab issue to further investigate it

There might not be a need afterall actually.

Look what I found that does not seem to have been fixed. ```void ButteraugliComparator::ReleaseTemp() const { temp_in_use_.clear(std::memory_order_acq_rel); }```

Was the fix actually ever merged into master?

```void ButteraugliComparator::ReleaseTemp() const { temp_in_use_.clear(std::memory_order_release); }```

🤔

🤦

Thanks for the find, <@321486891079696385> !

No problem.

What's interesting though is that the patched build of JXL and new aomenc still results in tune_butteraugli making the encoder crash interestingly enough.

I'll try it out with the next JXL patch comes out since it's already quite late here.

I think oss-fuzz might have found a nasty JXL memory bomb -- I have a 2800 byte file that needs c. 5GB of ram just to get the dimensions this is with git master libjxl file is here: http://www.rollthepotato.net/~john/mem.jxl I can open an issue if that would be helpful

the sequence of returns from JxlDecoderProcessInput() is BASIC_INFO, COLOR_ENCODING, memuse is about 30mb at this point, all normal, then the next call to JxlDecoderProcessInput() takes a long time (several seconds), memuse jumps to 5GB and it returns DEC_ERROR

🤔

jxlinfo mem.jxl xsize: 130780 ysize: 108984 have_container: 0 uses_original_profile: 1 bits_per_sample: 8 exponent_bits_per_sample: 0 intensity_target: 255.000000 min_nits: 0.000000 relative_to_max_display: 0 linear_below: 0.000000 have_preview: 0 have_animation: 0 orientation: 1 num_extra_channels: 0 alpha_bits: 0 alpha_exponent_bits: 0 alpha_premultiplied: 0 color profile: format: JPEG XL encoded color profile color_space: 0 white_point: 1 white_point XY: 0.312700 0.329000 primaries: 1 red primaries XY: 0.639999 0.330010 green primaries XY: 0.300004 0.600003 blue primaries XY: 0.150002 0.059997 transfer_function: 13 rendering_intent: 1

djxl mem.jxl Read 2998 compressed bytes [v0.3.7 | SIMD supported: Neon] No output file specified. Decoding will be performed, but the result will be discarded. ../lib/jxl/codec_in_out.h:65: JXL_FAILURE: Image too big. ../lib/jxl/dec_file.cc:117: JXL_RETURN_IF_ERROR code=1: VerifyDimensions(&io->constraints, xsize, ysize) Failed to decompress to pixels.

jxlinfo/djxl return quickly for me

``` luca@desktop ~/jpeg-xlm/build all_fuzz $ /usr/bin/time ./tools/djxl mem.jxl Read 2998 compressed bytes [v0.3.7 | SIMD supported: AVX2,SSE4,Scalar] No output file specified. Decoding will be performed, but the result will be discarded. ../lib/jxl/codec_in_out.h:65: JXL_FAILURE: Image too big. ../lib/jxl/dec_file.cc:118: JXL_RETURN_IF_ERROR code=1: VerifyDimensions(&io->constraints, xsize, ysize) Failed to decompress to pixels. Command exited with non-zero status 1 0.00user 0.00system 0:00.18elapsed 7%CPU (0avgtext+0avgdata 10220maxresident)k 0inputs+0outputs (0major+1319minor)pagefaults 0swaps ```

also for me (but it's on the private repository, fwiw)

huh I don't have jxlinfo, I guess I must have disabled it during build

I think it's in examples/jxlinfo

I think djxl is giving up after BASIC_INFO (it sees the crazy size), but it should get to JXL_DEC_NEED_IMAGE_OUT_BUFFER without needing a lot of memory, shouldn't it? I'd expect any very large amount of ram to be allocated there

no, there's need for intermediate buffers to hold stuff that can get as large as the image itself

(I'm working on getting rid of them, but it takes a while...)

ah I see! at the moment when I scan a JXL image to get the metadata I call JxlDecoderProcessInput() until I see NEED_IMAGE_OUT_BUFFER but it sounds like I should stop after JXL_DEC_COLOR_ENCODING, is that correct?

I suggest you do something similar to examples/jxlinfo.c

🙂

if you subscribe to JXL_DEC_COLOR_ENCODING, is it /always/ returned, even if an image has no color encoding info? the docs don't say

there is no such thing as not having color encoding info

ah, good to know, I'll stop after COLOR_ENCODING then, thank you!

yw 🙂

Would such checkerboard dither affect solid colors or would djxl have to limit dithering to DCT gradients?

Hm, good question

I suppose it depends on what the solid color is

I would do something like instead of doing `int pixel = floatval + 0.5` (which is basically what we do now), alternate between doing + 0.25 and + 0.75

So if the actual value of the solid color is 123.9, then it will become solid 124

But if the actual value is 123.4, then it will checkerboard-alternate between 123 and 124

Which kinda makes sense

How would I add https://jxl-art.surma.technology/?zcode=c8osSUktKMlQMDTg4spMU%2FBTsFMw4FJQ0FUITi1BYhlxAQA%3D as a layer to a typical JXL image?

Matt, I believe we will eventually get the dithering right in the mainstream version

for the first launches we take minimal risk in having additional new features and additional computation

in earlier versions (pik) we used to have 2x2 ordered dithering already

my understanding is that quality is currently comparable (or better) than previous tech (jpeg, webp, avif)

for the first launch is the safest to be marginally better in all attributes

when dithering is added there are two main consequences: the smallest amount of additional computation, and compatibility with some existing systems is slightly reduced (like PNG files will be somewhat larger)

I had asked Zoltan to add dithering in pik for the same reasons you have now and those reasons have not changed for jpeg xl

just we consider right now decoding speed and simplicity to be even higher priorities for the first launch

---

if there is a degradation, i.e., jpeg xl is worse than jpeg (and other codecs) because of lack of the dithering, then the lack of dithering adds to the launch risk and we should now reconsider the removal

Oh, by the way: https://jxl-art.surma.technology/?zcode=c8osSUktKMlQMDTg4spMU%2FBTsFMw4FJQ0FUITi1BYhkaGBgAAA%3D%3D Zoom in and you'll see a barrel distortion, zoom out and you'll get a pincushion distortion 🙃

Just wondering though... is a "jxlmux" planned? (Btw, could you hire more people?^^)

But -d 1 isn't visually lossless for gradients by default. 🙃

Would it be better to only apply dithering when the user encoded an image with a new flag --dither=1? So you wouldn't need to worry about PNG file sizes for the default case.

Well, sadly you should also use dithering when resampling images. So: extend 8-bit to e.g. 16-bit --- convert sRGB to RGB --- scale down image (e.g. Lanczos) --- convert back to sRGB --- dither to 256 colors per channel --- drop LSBs to get final 8-bit output.

That cannot really be done without changing the spec

Applications can ask for output in 32-bit float, or in 16-bit int, and then it will be up to them to dither

But I agree that in case application ask for 8-bit int output, we can/should do some dithering to alleviate the limits of 8-bit color displays

<@179701849576833024> what would be the speed impact of doing rounding with checkerboard-alternating 0.25f/0.75f instead of 0.5f when doing float to int?

probably none

I doubt it would look good though

There are two codepaths that produce 8-bit output: a fast one that does inverse XYB in int16, and a slow one that ends up with floats which then get converted to ints at the very end

the int16 one is only for arm though

there's two places where the conversion to int happens, and IIRC they use NearestInt, not +0.5 followed by a truncate

NearestInt does something different than that?

so one would need to do something slightly different for getting some dithering out of it

I think it translates to some asm instruction

but we could replace it with +0.25 / +0.75 + truncate, probably not too different in speed

however, I don't know if a checkerboard pattern is a good dithering pattern

Or do an explicit +- 0.25 before nearestint

It's not very good, but better than nothing at all

Could do some 4x4 pattern too I guess

I dunno, I think it might produce visible lines

but yes, we can use any fixed pattern easily

I remember a gitlab issue about this where someone tried some things

https://gitlab.com/wg1/jpeg-xl/-/issues/136

checkerboard works okayish:

doesn't look very good to me

compared to nearest int:

(how did you generate those?)

well, OK, it's a clear improvement 😄

still see some banding though

From the gitlab issue: ``` float grid16[4][4] = {.15,.45,.35,.85, .75,.55,.25,.65, .35,.65,.45,.75, .15,.85,.55,.25}; ```

I rendered JXL to a 16-bit PNG and screened the 16-bit JXL art on top in GIMP. Conversion to 8-bit was then without additional dithering.

Yes, some ordered 4x4 or 8x8 would be way better.

4x4 should be good for simd

8x8 is probably overkill

A 2x2 pattern that is not symmetric might also be slightly better, maybe

could do 4x4 ordered dithering with no real problems

0.2 0.65 0.8 0.35 or so

kinda want to try it now actually... can you share the jxl file?

Symmetric 2x2 only makes it effectively 9-bit, while asymmetric 2x2 makes it effectively 10-bit, right?

<@456226577798135808>

(unless you already did :D)

There's a good test image in https://gitlab.com/wg1/jpeg-xl/-/issues/136

Are you sure you responded to a right thread? I was talking about an optical illusion while in motion (it doesn't exist if you stop zooming in/out), something completely unrelated to dithering.

this is 4x4 ordered dithering

if it has a performance impact, I couldn't measure it

That's quite a visible difference - I cannot see any banding in the dithered one, while in the nondithered one it is quite obvious

I can see some banding in both for the gray one

but yes, it does look quite a bit better

I see banding in all of them but more noticeable in the gray images.

Jim make sure not to look at discord's downscales, they suck

(be careful about the zoom level, any kind of zooming will completely destroy the effects of dithering)

I was looking at the original at 100%.

Might be good to try to get dithering in Chrome's pipeline too (because browser downscaling is quite common and will make whatever dither the decoding produces irrelevant or less relevant)

ok then 😄 in my case I had to look at 80% as reported by chrome to actually get 100% because of some interface scaling

pretty hard if you store data in uint8 in the first place though - or do you mean after browser downscaling?

I mean after browser downscaling

I tried in Chrome & Firefox on 2 different monitors. Noticeable in all, though even more noticeable when the image was fitted to the viewport (downscaled).

the question though is whether the dithered versions look better or not 🙂

Very slightly better but mostly when downscaled. At 100% it seemed just as noticeable.

Maybe your 100% is not 1:1 then

yeah

can you look at them in an image viewer?

It is annoyingly hard to do 1:1 viewing in browsers nowadays

On my macbook, "100%" means "quite a bit upscaled"

Odd, in the image viewer the downscaled version it was almost unnoticeable but at 100% it was even more noticeable than in the browser.

The non dithered version looks better at 100% than the dithered in the image viewer.