JPEG XL

So I dunno if that's what's wrong

I will investigate and post an example image later today

I don't think linking with system libjpeg is the issue

I don't install the jpegli libjpeg anyway and everything works as usual

I am also using gcc but yeah I doubt that's the issue either. Maybe it's just me or my image viewer

I tested both v0.9.x branch and the main branch

There's an issue with images that have an alpha channel, even if completely transparent. Not sure if it's been fixed yet

completely transparent or completely opaque?

Maybe that's the issue. I thought the alpha channel would be discarded.

Right, Opaque, for some reason I was thinking "See through the Alpha to the image" haha

But either way, the mere presence of the channel was corrupting encoding

https://github.com/libjxl/libjxl/issues/2671

That's what it looks like. I tried with two different png files.

I know at least one has an alpha channel

iirc they're linked with system libjpeg for the non-jpegli side of it

like you can cjpegli anything that cjxl accepts, and if it happens to be a jpeg, it'll use system libjpeg to decode to pixels, before using jpegli to encode to jpeg

ideal behavior? no

but iirc it's the same behavior as cjxl/djxl

It still confuses me why there are 3 different jpeg encoders in the lib/enc tree. sjpeg, libjpeg and jpegli

they haven't spliced jpegli into the tree as an encoder the other parts of the toolchain can use

sjpeg iirc is like skcms, it's a lightweight alternative to `/usr/lib/libwhatever.so`

once they do I'd imagine they'd use it by default if possible

The libjxl tree isn't very lightweight...

no, it's not, and embedding everything as a submodule despite being able to link to the system libs is actually pretty annoying

but that's how all the google projects are set up

IMO there's no reason for it to pull in zlib in a submodule, but I didn't make that decision

Maybe I'll try to prune this heavy tree sometime. I believe it materially hurts adoption that the reference library is so much chonkier than all other codec libraries typically are.

that actual library's not that heavyweight if you link to system libraries instead of chunking it all together

All similar libraries have little to no dependencies, MIT license with no CLA and can usually be built with any C89 compliant compiler

C99 is more portable than c89 but yea

usually it requires a C99-compliant compiler because C89 lacks stdint.h and the like

C89 also lacks // comments, the inline keyword, etc.

Yeah, the newer versions are better, but basically everyone goes out of their way as much as possible to make it as accessible as possible for anyone to adopt, contribute, port to new platforms, etc

Legal/corporate reasons might prevent that but in that case it might be a good idea to foster development and adoption of third party implementations that no one is "responsible" for in a reputation or legal sense, and have the reference implementation merely try to be a good reference.

The more implementations there are, the better IMO, as long as they are all bitstream compatible.

can't build libjxl with less than 1gb ram per core

who actually uses C89 these days? Even Linux is C11.

didn’t Microsoft ignore C99 for the longest time?

pretty much went straight to C11 from what I recall

FFmpeg is C90 with a few features from C99

IMO C99 is the gold standard because it's the most portable version of C

`uint64_t` is pretty important for portability because `sizeof(long)` is not specified

sorta, they decided to start supporting C99 about 10 years ago

with MSVC

however, many C99 features are also in C++, so those were also supported

like `//` comments

msvc doesn't even support C++ correctly on fairly basic stuff

<@853026420792360980> wouldn't disabling the container format in libjxl\_anim mean ffmpeg would have to implement a jxl muxer as well?

No, you can use image2pipe

Muxer

oh

that's neat then

That said implementing a dummy muxer is a good idea

It makes it easier to tweak later

by dummy muxer, do you mean a bare bones jpeg xl muxer?

eh, it’s not the VC6 era anymore

support up to C++20 is quite good

https://en.cppreference.com/w/cpp/compiler_support#cpp20

(remember when variables declared in the init statement of a `for` loop would outlive the loop?)

there's a bug in how it handles utf-8 string literals which makes them completely unusable. this violates both C++ and C standards. I find this annoying! (specifically, `u8"foo"` should always result in a UTF-8 string. If the file encoding is something other than UTF-8, then the compiler should do the appropriate conversion. However, MSVC instead reads each byte in the string and pretends it's a code point, which it then encodes into UTF-8. UTF-16 and UTF-32 string literals OTOH work fine, it's only the UTF-8 ones that are broken)

ah, that does sound annoying

out of curiosity, how is the encoding of the file supposed to be known by the compiler?

I think I’d find it more reliable to use regular strings while ensuring that the file is UTF-8 encoded itself

I see little reason to use any other encoding

Of course your file is probably UTF-8, but that's not assumed in the standards.

But compilers do document what they expect, or have options for using different encodings. E.g. MSVC has a switch for telling it that source files are UTF-8.

The important part is that however the compiler interprets the file, every *character* in the string literal should be converted to Unicode and encoded with UTF-8. Note that if you're not sure what encoding your file will be in, you can use Unicode sequences like `\u0123`. But even when you do that, MSVC fails. Unicode escapes work fine in regular, UTF-16, and UTF-32 string literals.

note that regular strings use whatever encoding is specified by your locale, even if the file is UTF-8. The point of having Unicode string literals and Unicode character types (i.e. `char8_t` and friends) is to allow guaranteed Unicode behavior even when the system is using Codepage 437 or whatever. C2y/C3x is also going to have some actually good character conversion functions to make it even easier. But it's only useful for portability if it works correctly everywhere 🙁

yes

y would you use anything else tho

ah, yeah it's probably a good idea long term, but one would need to have access to a copy of ISO 18181-2

there's a draft in <#1021189485960114198>

oh, nice

i only searched arxiv and the usual places

Is there a way to build libjxl using the system-installed libjpeg headers instead of in the third_party folder?

I have all of the header files installed so I don't understand why I have to download it again in the third_party folder

`-- Found Brotli: /usr/include` But why doesn't it do this for libjpeg as well?

The problem is in `lib/jpegli.cmake` in the `configure_file` lines

You know, it won't take that long to replace all `jpeg_` with `jpegli_`...

Many google projects bundle all dependencies in a single repo, it's a common annoyance with them. I don't see any way around it atm

Well in this case it's literally just 3 configure_file lines in jpegli.cmake

It wouldn't take that long to fix it... I might even get around to it myself if it isn't done by the time I get my computer set up...

on one hand, I do see the annoyance, on the other vendoring really does help work around a lot of package management issues

It's able to use system libs for everything else except skcms

Jpeg is the one odd and obnoxious exception and only due to copying 3 header files it could and should easily get from the system

And only for jpegli

If you don't enable jpegli then there's no issue either as far as I can tell

hey

will there be a new release soon ?

No. Just check out the v0.9.x branch. It's more lulzy to not actually release it.

i've just received mail for a new release 🙂

according to the changelog :

JxlDecoderSetCms to enable decoding to desired colorspace

so it's possible to have as output BGRA (B lower byte, A higher byte) ?

if i'm not mistaken, by default libjxl uses RGBA colorspace

no prebuilt libraries?

probably, if your cms can do that

those should appear after some time

i would prefer something inside libjxl itself, it's just another way to fill the output

swapping 2 channels

<@179701849576833024> can you tell me where in the code i can do that (swapping the2 channels) ?

Not sure, you'd have to request an output ICC profile that swaps B and R

why not having an api which tells that output is BGRA or RGBA ?

we never felt it was useful, I guess

yeah, it’s not clear that this added convenience warrants the repetitive code that this would require (which may still not cover what everyone would want) and this potential new source of bugs

i doubt it will be error prone

just tell me where it is in the code

I'm rather late, but it seems the binaries failed to upload <https://github.com/libjxl/libjxl/actions/runs/7300997957/job/19897617556#step:14:11>

ugh

well, looks like we'll need a 0.9.1 anyway...

Only real way to test is by letting the public get their hands on it, at least now any serious issues are obvious and can be fixed quickly

on windows with msys2, i downloaded the 0.9 release, but running deps.sh, i have this error:

Downloading third_party/libjpeg-turbo version TURBO...

...

gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now

I have all the necessary libraries, why must i run this script ?

this is completely useless

add a requirements section and ask people to install them via their package manager if necessary

and honestly zlib libpng and libjpeg are certainly already installed in all the linux/bsd/solaris distro

zlib is even installed by default on msys2...

https://github.com/libjxl/libjxl/issues/3069#issuecomment-1867857652

May be related

well, some deps are tied to submodules, which you would get if you ran git clone --recursive, deps.sh is there for if you download the zip source code but idk if it's still maintained very much

<@557099078337560596> it seems it is not

i'm still convinced that it is a bad idea

let people install the required deps

and again, zlib, libpng and libjpeg are so used that they are certainly installed in all unix distro

0.9.0 windows build where

I certainly know that I can take them here: https://artifacts.lucaversari.it/libjxl/libjxl/latest/ but why is this not on github, what is the reason - since it's already been the day since the release

fun

wait, but if there are problems with the build of the latest version, then how did it happen...

a problem with deps.sh downloading a .tar.gz wouldn’t imply a problem with git submodules

``` $ cat TURBO.tar.gz.tmp 404: Not Found ```

presumably, we need to update an URL

still fun

found the issue, I’ll submit a fix

https://github.com/libjxl/libjxl/pull/3071

ooooh ! now can compress the image 60000x120000 ...... or have not solved the problem with eating all the memory yet? :)

in fact, it is a very incomprehensible parameter, so it also has so many options to choose from. It is not written which specific methods are disabled for encoding. There is not even an approximate percentage increase in decode speed... The developers clearly did tests on each of the parameters and know what the percentage increase in decoding speed is -- could write it in the parameters

that parameter has been there for some time 🙂

where is the article, large image decoding tests, speed comparisons with jpeg webp avif where AVG decoding speeds of 1000 images on each parameter <:AngryCry:805396146322145301> should I do myself

https://tenor.com/view/do-it-what-are-you-waiting-for-determined-angry-gif-5247874

😈

**libavif **does not have ready-made builds on github 😤 will have to use avif in ffmpeg, and jpeg. perhaps the same is possible with webp. I'm not sure about JXL. I think ffmpeg just doesn't have all the parameters like cjxl

There are for jpeg

<https://github.com/libjpeg-turbo/libjpeg-turbo/releases>

Libavif uses appveyor

🧐

I tested 500 images of random sizes and contents of e7 d1

first of all, well, there is a small difference. secondly, the real difference on One large image when decoding in 1 thread is about 10-30% of the decoding time (fast0 vs fast4)

it's not even 2 times faster. it doesn't even make sense to compare, webp will be ten times faster for decoding

judging by the uneven distribution, it is obvious that some functions were turned off and some were turned on for use on each parameter separately. Since I have no idea what these functions are, I can only say that, looking at what I see from the numbers, I absolutely do not understand why to do a Gradation into 4 levels of fast decoding. It doesn't have a rational.

one big image (4 threads)

well, that is, in my opinion, it is quite obvious that there is no point in making 4 different keys to disable different encoding parameters. need to do only one (fast-decode) where everything that is possible is disabled, As I understand it fast4

e3

I want to make an clarification that all images after decoding are recorded in PNG and this is included in time. but since I can just open the image to view it, I can say that this does not greatly affect the test results. Generally a disappointment

I found `faster_decoding=2` (or maybe it was 1?) to help my use case a decent amount, anything beyond that and I got no difference

FWIW that flag does little for lossless

it has a bit more effect on lossy

images with all fast decode parameters become larger by about the same amount, about 5%

for my testing I was using `cjxl.exe -e 6 -d 0.5` and then added `--faster_decoding=2` (didnt test 1 for thhis specific test) and gained about 100fps for my decode times

from 110 to 210 rougly

1080p video

yup makes sense, I don't remember exactly which level turns off EPF, but that's where you'll see most speed gains

(arguably at d0.5 you don't need epf at all, but still)

I did also test `cjxl -e 3 -d 1` and found a similar speed increase. but the video I used as a source is relatively simple, I did want to try netflix's chimera test clip, but my PC would OOM when encoding it in parallel to somewhat fast speeds and never had the time to keep the storage space in use to do it in serial

the only observation I can make is that the e7 fast4 has become like a regular e3 in terms of decoding speed - but this is still a very long time in single-threaded testing. The difference in the fast-decode parameter is completely invisible with multi threaded decoding, 10% of the plus performance is dissipated.... In general, it's even strange why do that the quality will not change at all from fast decoding. All the codecs that I know, and these are video codecs that have FastDecode, have very large differences in quality, and it's WORTH it that decoding becomes 50-100% faster

<:Thonk:805904896879493180> in general, the parameter is also insignificant, so bad that it cannot be compared even with other codecs, because there is no magical advantage

it can be by default for images larger than 10 megapixels or something like that

I use `faster_decoding=2` on all of my video sequences myself since it helps me a lot, for a 1920x1080p video https://cdn.discordapp.com/attachments/673202643916816384/1182221277776314398/image.png https://cdn.discordapp.com/attachments/673202643916816384/1182221278430634014/image.png https://cdn.discordapp.com/attachments/673202643916816384/1182221413357195276/image.png https://cdn.discordapp.com/attachments/673202643916816384/1182221413856313384/image.png

when I load these into a video editor, it's a night and day difference

I would like faster_decoding=3 and faster_decoding=4 to do more since for this video there was zero difference for me

my tests also show that there is almost no difference between fast4 and fast2.

T.T

it is difficult for me to say how the tests were performed by the one who made this parameter

some kind of roller coaster

I just need like 20-40 more fps for the clips I use T.T

I did tests on 4 cores. But judging by your tests, I can make an assumption that with fast 2, some kind of decoding function is disabled - which is a bottleneck for more multicore system . Which gives you much more performance

yeah my tests had pretty much unconstrained accsess to my 2600 via ffmpeg

but still, opening a giant jxl image waiting 7 seconds. And then opening webp for ~200 ms. There is nothing to compare

IIRC 3 and 4 mostly make a difference on arm

So, `JPEGXL_STATIC` is gone in v0.9.0, and now I can't figure out how to build standalone exes in `msys2/clang64`.

`-DBUILD_SHARED_LIBS=OFF`

Yeah, that's what I did. Now the exes are asking all these libpng, libhwy, libz, etc. I build it with system hwy. Trying to link with '-lhwy'. Getting `ld.lld: error: hwy::SupportedTargets() was replaced`.

ah I see

does msys2 even install static libs?

yes

``` mingw-w64-x86_64-highway /mingw64/lib/libhwy.a mingw-w64-x86_64-highway /mingw64/lib/libhwy.dll.a ```

maybe we could revive `JPEGXL_STATIC`

huh that's different from arch then

I guess

I don't like it, but apparently cmake doesn't really have a way to make static executables, so...

``` $ ldd tools/cjxl.exe ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffa1c1b0000) KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffa1a4d0000) KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffa19c60000) apphelp.dll => /c/WINDOWS/SYSTEM32/apphelp.dll (0x7ffa16df0000) msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffa1ac30000) libOpenEXR-3_2.dll => /mingw64/bin/libOpenEXR-3_2.dll (0x7ff9afa30000) libgcc_s_seh-1.dll => /mingw64/bin/libgcc_s_seh-1.dll (0x7ff9ef780000) libwinpthread-1.dll => /mingw64/bin/libwinpthread-1.dll (0x7ff9fc500000) libstdc++-6.dll => /mingw64/bin/libstdc++-6.dll (0x7ff9af800000) libImath-3_1.dll => /mingw64/bin/libImath-3_1.dll (0x7ff9ef720000) libIex-3_2.dll => /mingw64/bin/libIex-3_2.dll (0x7ff9e2180000) libIlmThread-3_2.dll => /mingw64/bin/libIlmThread-3_2.dll (0x7ff9fc3b0000) libOpenEXRCore-3_2.dll => /mingw64/bin/libOpenEXRCore-3_2.dll (0x7ff9af5b0000) libdeflate.dll => /mingw64/bin/libdeflate.dll (0x7ff9f76f0000) ```

still not super static

on Linux, it is

Arch disables both compiling static when possible, but also just out right disabled packaging them, butit's easy enough to enable it for repackagers https://gitlab.archlinux.org/archlinux/packaging/packages/pacman/-/blob/main/makepkg.conf?ref_type=heads#L95

simply remove the ! to enable it, and ofc make sure the package enables building it

will libjxl-0.8.x be receiving any security updates, now that 0.9 is out? I'm asking in the context of possible jxl support removal from cromite browser https://github.com/uazo/cromite/issues/627 if upgrading chromium code for compatibility with libjxl-0.9 will pose a problem.

I believe <@795684063032901642> is working on getting 0.9 running on Chromium 🙂 (API is the same, but there's a few snags with hwy)

Yes, this is wip

no 0.9.1 soon ? the build is broken

> The current reference software libjxl is quite optimized for the lossy case, but still has some room for improvement for the lossless case — in particular it does not have fast paths yet for the common case where the full precision of 32-bit per sample is not required. <https://news.ycombinator.com/item?id=33448192> has there been any progress towards this? libjxl has fantastic decoding performance for 16 bit lossless images compared to PNG (especially with fast_lossless), and recompressed JPEGs compared to libjpeg-turbo, and great performance for lossy when the image size isn't too big. however, 8 bit per channel tend to be significantly slower in my tests compared to PNG and WebP. decoding speed is however much better with fast_lossless, but for the standard `-e 7` setting it gives oftentimes (not always!) better filesizes with typically much slower decodes, compared to fast_lossless as it stands right now, i can't have my cake and eat it too with good filesizes and decoding speed at the same time for most files. i have an 8 core 16 threads CPU (ryzen 7 6800HS).

if you need some benchmarks, i can provide, but i'm mostly curious how optimized libjxl currently is with 8 bit per channel images

it does not support PNG decoding lol

There's an API/ABI break from 0.8 to 0.9

We had to #ifdef around it in FFmpeg

fwiw, Hydrium also depends on spng for png decoding but it's not in-tree

it uses a meson.wrap

uhhhhm that ought not to happen, what is it?

you removed an unused argument from the middle of a function, lemme pull up specifics

https://github.com/FFmpeg/FFmpeg/blob/f5f414d9c4bef90cb8986c0b71abd3c87a68fe4d/libavcodec/libjxldec.c#L216

that should not have happened... do you mind filing a bug?

we'll probably revert it in 0.9.1, I think

we probably should also have tests for this 👀 but they are kind of a pain

Yea, though im on vacation so it may or may not come for a few days

fair enough

so are we 😛

If not now, then when are we going to remove that deprecated unused argument? Or do you want to keep it also in 1.0?

we can break API/ABI for 1.0

and then 2.0, ...

if libjxl follows semantic versioning, it's ok to break public API compatibility in every 0.x release

that is technically true, but there's also no real good reason to do it

if it were C++, we could have both versions of the function

FWIW, libvips had a similar issue, but I was able to fix it before v0.9.0 was released - <https://github.com/libvips/libvips/pull/3774>.

I guess we could also leave it as is...

anyone know how to have `libjxl-git` and `ffmpeg-git` AUR packages installed at the same time?

when installing `libjxl-git`: ``` :: libjxl-git and libjxl are in conflict. Remove libjxl? [y/N] y error: failed to prepare transaction (could not satisfy dependencies) :: removing libjxl breaks dependency 'libjxl.so=0.8-64' required by ffmpeg-git```

We also prepared for the API change few months ago: https://gitlab.gnome.org/GNOME/gimp/-/commit/a4f02398188bc36d1808ef802082b3856cc73741

Also in kimageformats: https://invent.kde.org/frameworks/kimageformats/-/commit/3cb6519dcc24563fcf60f80ec7265ecb487d155a

I prefer breaking it now rather than later. Having a 0.x release that is already compatible with 1.0 is convenient, otherwise the 1.0 release will also cause inconvenience.

I mean, if people already fixed things up...

It would help a lot with stabilizing the API, if someone responsible for designing it would take a look at the way the libjxl API is being used in ffmpeg and libvips and graphicksmagick, and in this way, see how people are actually using the API in practice.

https://sourceforge.net/p/graphicsmagick/code/ci/default/tree/coders/jxl.c https://github.com/ImageMagick/ImageMagick/blob/main/coders/jxl.c https://github.com/libvips/libvips/blob/master/libvips/foreign/jxlload.c https://github.com/libvips/libvips/blob/master/libvips/foreign/jxlsave.c https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavcodec/libjxldec.c https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavcodec/libjxlenc.c

you’ll probably need to temporarily remove ffmpeg-git, install libjxl-git, then rebuild and reinstall ffmpeg-git

(alternatively, you could do something like `pacman -Rdd libjxl`, install libjxl-git with ffmpeg-git still installed but broken, and then rebuild/reinstall the latter, but afaict, you don’t gain much by doing that)

thanks, `pacman -Rdd libjxl` works

it was of course necessary to compile ffmpeg without libjxl support as it currently expects 0.8

it shouldn’t, according to ^

it gets linked with 0.8

install libjxl-git but when you do `pacman -U` pass the command `pacman -U --assume-installed=libjxl.so=0.8-64`

if you install libjxl-git first, it uses whatever your system libraries/headers are, which you can do

so you have to install libjxl-git first, and then build ffmpeg-git

I just `-Sdd` (install while ignoring dependancy breakages) then rebuild ffmpeg

ooh thank you!

[100%] Built target jxl_dec C:/Documents/msys2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles/jxl.dir/objects.a(encode.cc.obj):encode.cc:(.text+0x7124): undefined reference to `JxlGetDefaultCms'

git repo

hm, how did that happen / how did you build? I don’t encounter this

I build with clang, though – let me try gcc

seems to work

``` [490/490] Linking CXX executable lib\tests\decode_test.exe ```

might there be some stale stuff in the build directory (including maybe in CMakeCache.txt)?

<@604964375924834314> https://pastebin.com/0DqFFUBE

after a clone

ah, I get about the same error when trying to call `JxlGetDefaultCms`: ``` ld.lld: error: undefined symbol: __declspec(dllimport) JxlGetDefaultCms ```

Hi guys, what kinds of information will lose in a **distance = 0 with VarDCT** encoding progress?

I'll do a quick test, assuming you mean visual quality rather than metadata

no, for command line `cjxl`, 0 means modular. But by calling `libjxl`, one can use **distance = 0** in VarDCT mode and produce smaller file size than the **lossless mode**

Surprisingly, VarDCT did rather well. 2.5x larger than real lossless, with 1 value (8bit, so 1 out of 255 per channel) color differences along edges in the main area of the image. In the corner with an FPS reading, the max value difference is 15 in the Red channel and the highest difference in the image Original, VarDCT 'Lossless', Difference, Difference extrapolated (the 15 max value mapped to 255)

maybe it uses default distance value in that case

uh maybe no

I just did `-d 0.0000001` to get close enough with cjxl

0 does force on Modular even with `-m 0`

Looks like it gets clamped to `-d 0.01`

I believe the API does the same clamping

in fact I am fairly sure, everything would break with an actual 0 for distance in vardct

Is it normal to have a smaller result file with `effort: 3` than `effort: 7`?

My test on a 6100M pixels photo: ``` effort: // 1 -> 15.4mb // 2 -> 15.4mb // 3 -> 14.2mb // 4 -> 14.3mb // 5 -> 19.1mb // 6 -> 19mb // 7 -> 18.6mb ```

depending on the content, it is

that's quite size and quality difference, though

whatkind ofimage was encoded?

a linear data from 14bit raw file

oh

yeah, that makes sense then

5 6 7 should be much closer in quality to the target quality

I noticed that linear image contains a lot of low value data, so I have to set the transfer function to linear.

I have to use effort: 7 with distance: 0.2 to get visually lossless

if I want to increase the brightness of low value parts after the convertion

larger than d0.2 will produce some artifact in low light parts like this: https://discord.com/channels/794206087879852103/794206170445119489/1189992488195334277

yeah, that should not be

is color metadata set correctly?

is brightness boosted or not?

since the data is directly from uncompressed rgb dng, i set JxlPrimaries to 2100 and JxlTransferFunction to Linear

no, just raw uncompressed rgb values

not even white balanced

but correctly decoded

or just linear interpreted as a gamma transfer

In fact, I'm test the DNG spec 1.7

displaying linear on a normal display without transfer curve conversion is just incorrect

I replace the RGB part in the DNG with the encoded JXL data. Now it's a JXL DNG.

Then I put the dng to lightroom, the color and birghtness display correctly.

As I know, JXL will reduce the quality in darker area. Since a linear data are darker in value, so I guess JXL will reduce the quality averagely. Maybe that's the reason why I need to set distance to a smaller value to keep the darker area quality.

You could also try Adobe's DNG converter which I experimented with fairly recently

no, JXL uses its own brightness curve internally, if the encoder knows the transfer curve during encoding, transfer curve does not make a difference for quality

I've tried that, but adobe dng converter has a poor jxl encoding quality.

But why changing the transfer curve metadata can affect the final result file size?

only change the color metadata if it's incorrect of course, or missing

if source metadata is missing, you can do something like this instead: `-x color_space=RGB_D65_DCI_Rel_Lin`

or also add intended target luminance: `-x color_space=RGB_D65_DCI_Rel_Lin --intensity_target 250`

I already said that I set the transfer function

well, but I did not understand that sentence...

no idea what a DNG is...

Setting JxlPrimaries to 2100?

what does that mean?

why 2100

jxl uses 2100 instead of 2020 for gamut

ah

why though

that's dumb

literally the same

yeah it felt weird to me too,

just lemme use my cicp numbers...

iirc they align with the cicp numbers, it just doesnt have the full support for them

~~oh wait maybe not~~ no it does

but, where do you set that "JxlPrimaries" value

maybe an 3rd party tool linking to libjxl

You can use it in the command line and specify quality and effort

https://helpx.adobe.com/content/dam/help/en/camera-raw/digital-negative/jcr_content/root/content/flex/items/position/position-par/download_section/download-1/dng_converter_commandline.pdf

https://discord.com/channels/794206087879852103/803574970180829194/1187038265795149914

it doesn't particularly matter what it's called, BT.2100 though specifies HDR and BT.2020 specifies an SDR space

right

But the oldest specification, unless older than BT.709, is usually the one that is named.

I mean sRGB and BT.709 have the same primaries

and BT.709 is older

should it say BT.709 instead?

It seems like people don't like unstable API and poor error handling, judging by github issues and comments in open source projects, some even opt to use cjxl/djxl executables instead of libjxl API. Will those problems go away in 1.0?

thanks for mentioning this. I'll have a try

<@238552565619359744> quite weird. I'm trying Adobe DNG converter cmd with this ARW photo: https://www.dpreview.com/sample-galleries/0740273820/sony-a7cr-sample-gallery/8613531963 by using this parameters: `converter -jxl -jxl_distance 0.25 -jxl_effort 7 20230813-0187.ARW`. The original arw file size is: 79MB The generated dng file size is: 65.3MB <- it shouldn't be this large

I manually created a DNG with `distance 0.25` and `effort 7`from the same ARW, it's only 14MB. There must be something wrong with the Adobe DNG converter

Try `-lossyMosaicJXL`

Great. That makes sense.

I knew, `-lossyMosaicJXL` keeps the bayer pattern and lossless, it's about 23.7MB

What I mean is, `-jxl` doesn't work right in Adobe DNG converter

For example, if you try this on an Apple ProRAW(which is RGB linear dng), it works poorly.

I tested on one ProRAW dng, only 61.5MB to 59.2MB

Oh, wait

I think you need to specify `-lossy` or `-lossyJXL` along with those arguments, the command line is very cursed for the converter

`-lossy` works

In my brief testing it required specific ordering of arguments, complete file paths and redundant commands (Such as `-lossy` and `-jxl_distance` at the same time)

Sometimes it would throw a missing file error, sometimes it would do nothing at all

Thank you~

No problem, just remember to throw the results in here somewhere

Now the result size are normal. But since it's small tile based, it's still a littlbe big larger than a file from an whole image JXL encoding process.

I have a question about JXL. Will JXL encoder treat all dark image differently to a high dynamic range image? For example: I have two 1000x1000 linear RGB 16bit images. One is all dark colors like values below 50. The other contains colors across the whole 16bit range from 0 - 65535. Assume these is a same 50 * 50 square in the center of both images, the RGB values in it are below 50. Will JXL treat the two squares differently? (I mean with the same parameters, will the image quality for these two squares different from each other?)

since the high dynamic range square will be way brighter, it will of course take much more bits to store at the same perceptual quality

I wonder how much cjxl takes global contrast into account...

probably depends on pixel distance

<@238552565619359744> it seems i cannot set `-jxl_distance` when using `-lossyMosaicJXL`

I shared everything I had tried and found out, sorry

anyway, thank you~

up to 5% is noticeable, is it worth it? https://github.com/libjxl/libjxl/pull/3084

any plans to add support for streaming png at least for input? maybe not the most important for server use, but for users it would be beneficial as the most used lossless format to avoid additional conversion steps

I think a 5% difference between streaming and non-streaming is pretty reasonable

i thought it was a 5% difference for streaming before and after these changes

<@179701849576833024> <@794205442175402004> this ssimulacra2 implementation uses "true" gaussian blur is there any reason why libjxl uses recursive? for optimizations that can't be implemented for the "true" one and recursive can always be performed faster and for less memory consumption? or accuracy is enough for jxl, but still might be useful for ssimulacra? <https://github.com/dnjulek/vapoursynth-ssimulacra2>

ehhh old streaming mode was pretty much the same as non-streaming anyway 🙂

even this one? https://discord.com/channels/794206087879852103/803645746661425173/1184615338319548538

at least on my test set

In my experience the accuracy of the blur (or even the sigma and kernel size) don't matter that much for the accuracy of the ssim/ssimulacra2 metrics...

hmm for ssimulacra2rs it seemed to have a fairly important accuracy hit when I tried to use a simple 2 pass box blur

I made a patch so that chrome compiles with current v0.9.0 libjxl version here: https://chromium-review.googlesource.com/c/chromium/src/+/5163728

<@179701849576833024> what about blue noise? <https://github.com/libjxl/libjxl/pull/1362#issuecomment-1242313023>

I just wrote the code

<@532010383041363969> can deal with figuring out the best matrix 😛

photon noise is blue noise (or violet, dunno) -- noise that has a minimal low frequency part

since we are at the edge of what can be observed we probably don't want to spend a lot of computation for the dithering, and probably don't need blue noise kind of approaches for the usual dithering from 24 bits to 30 or more

Do you mean stable chrome releases? The patch shouldn't be much different, I suppose? Considering that there are a few downstream projects that patch back libjxl into flavours of chromium, I think it would make much sense to have this behind a build flag which is disabled by default, to avoid having to deal with these patches

Not sure how that happened https://github.com/libjxl/libjxl/issues/3111

sounds like a regression, so should be bisectable

Went for the bug, stayed for the dancing banana 🍌

`libjxl/lib/extras/dec/pnm.cc` contains the follwing function: ```cpp StatusOr<ChunkedPNMDecoder> ChunkedPNMDecoder::Init(const char* path) { ChunkedPNMDecoder dec; // stuff return std::move(dec); } ``` Something about this doesn't make sense to me, namely that (1) `dec` is stack-allocated, which means it is de-allocated when the stack frame disappears, regardless of `std::move` (2) `std::move`, as far as I understand, invalidates the previous reference, so strings, etc. don't need to be duplicated when, say, pushed back onto a vector. However, I don't understand what it accomplishes here, because as soon as the function call returns, the local variable `dec` falls out of scope, and since `std::move` explicitly prevents duplication of objects, it won't duplicate `dec` before it falls out of scope. That said, i'm not a C++ programmer, I'm a C programmer, so might be misunderstanding something here. Can someone enlighten me?

I think this is like returning a struct in C. The object will be returned in registers, or on the stack, according to the ABI rules. This may involve the data being copied to another location. The move would make the previous location of the data invalid, but I think that's moot because returning makes previous location of the data inaccessible anyway.

Returning a struct in C passes the entire value of the struct, well, by value.

which is what is being done in that code snippet

yea, but `std::move` prevents that copy, iirc.

no

`return std::move` is now useless (https://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1579) but not for the reasons stated

`std::move` doesn’t intrinsically move anything or invalidate any reference

well "invalidate" no but it tells the complier that it can move const things (e.g. strings) without copying, right?

all it does is convert its argument to an rvalue reference, causing the moving constructor/assignment operators to be called instead of the usual copy one

it doesn’t

hm?

I'm not really sure what an "rvalue" reference is cause I'm not a C++ programmer

``` std::string b = std::move(a); ``` will initialise `b` with an rvalue reference to `a` and cause it to “steal” `a`’s buffer instead of creating a new one

but it still copies the pointer in the struct, so to speak

(the pointer to said buffer)

I don't speak C++ so I'll write this in C. Say you have something like ```c struct { char *str; size_t len; } ``` A move constructor would create a new struct of the same type, copy over the pointer and length, and then "invalidate" the original to a valid but defunct value, like setting the length to 0 and the pointer to NULL.

it then leaves `a` in a “valid but unspecified” state

the reason I'm confused ig is cause I don't know how strings work in C++. If you did `const char *b = a;` in C, that's exactly what would happen unless you explicitly called `strdup`

C++'s string type is a struct that has a pointer and stuff inside of it.

oh, so a class has to *declare* a move construct, which explicitly specifies how it interacts with `std::move`?

I think there's a default?

sure, but you see what I mean, right?

yes, but when implementing it you do have to follow the rules as to what a move constructor is supposed to do

initialize a new object, invalidate the old object

as opposed to a copy constructor which leaves the old object valid

ye, I see. there's a specific type of constructor called a "move constructor" that is called when someone calls `std::move` and it must behave in specific ways or the class is noncompliant

yeah

I didn't realize that was a thing

but *that being said* what does `return std::move(foo);` at the end of a function do, if `foo` falls out of scope immediately? How is it different from `return foo;`

In Rust, they decided that moves should not have constructors, and that instead, all moveable objects must be memcpy-able.

I'm pretty sure at least in theory `return std::move(foo)` is always redundant.

in the absence of https://en.cppreference.com/w/cpp/language/copy_elision, I guess `return foo;` would return the value by copy

it used not to, if it called a conversion operator that could take the value by rvalue reference

prior to the defect report above, if a function had `StatusOr<T>` as its return type and did `return t;`, it would call `StatusOr<T>`’s constructor that takes `t` by value even if `t` was a local variable that was going to be immediately destroyed anyway

you had to use `return std::move(t);` for it to call `StatusOr<T>::StatusOr(T&&)`

aha

(also, it’s not redundant if `foo` is not a local variable, for example it could be a class member being “given away” by a method, especially a method declared `&&`)

oh, yeah, I meant when it's a local variable

you mean in general? or in this case with the conversion?

in general

I thought it was move (in general)

I don't recall the exact language mechanism, but I believe it will try to move before it makes it a copy

best decision ever

for example, try returning a std::unique_ptr, but disable copy elision

I believe that's not guaranteed to compile without a `std::move` before some version of C++

there's also some version that does this "guaranteed return value optimization" thing, where returning a local *doesn't even move it*, but just directly constructs it into the local of the caller (I believe... <@604964375924834314> can you confirm? xD)

that guaranteed copy elision is from C++17. I don't know the exact specification but I think it mainly applies to if you do somehthihng like `return MakeMyThing(args);`

right, I wasn’t fully sure so I kind of meant my “in the absence of” to encompass “if it currently applies, then in a hypothetical world where it wouldn’t”

in either case I ask because gcc is warning me about `-Wredundant-move`

It looks to me this is because GCC fixes the issue spider-mario made a screenshot of, despite it technically not being part of the original C++11 spec

see the second half of

I remember reading that in some cases, `return std::move(t);` where `t` is a local variable may interfere with optimizations. But maybe since C++17 that is no longer the case?

I think that's still right. I think there are still many normal uses which are not covered by mandatory copy elision

Actually looking closer, I don't think mandatory copy elision will affect expressions of the form `return std::move(t)` Not really sure if there's anything else in C++17 either

Does mandatory copy elision still apply to such expressions?

Hmm, apparently not based on some searching. Ok, so in case `t` is a local variable, using `return std::move(t)` is not merely redundant, it is actually worse as it interferes with elision (and even if elision wasn't applicable, a move would still be used without `std::move`).

Yeah I believe that's the general guidance. It looks like GCC even separates between the two cases with different warnings: `-Wno-pessimizing-move` for when copy elision was missed due to `std::move`, and `-Wno-redundant-move` for when copy elision could not be done but `std::move` is pointless anyways because C++ says to fallback to moving even without `std::move`

(sidequest : does someone knows how to pass explicit flags to the compiler ? I would like to compile for my cpu, like with `-march=native` on gcc/g++ but libjxl recommend clang, and I'm not sure how to do, nor if it would change anything)

(does someone have a CPU which supports AVX512 ?!)

I have

does libjxl works really better on it ? 😄

Don't think so: JPEG XL encoder v0.9.0 6768ea8 [AVX2,SSE4,SSSE3,SSE2], so no AVX512 🤷‍♂️

is it a version you recompiled ? or the one distributed on github ?

Distributed on github. I did not and will not compile it myself, sorry

what cpu do you have ? just curious ?

Ryzen 7950X

AVX512 was removed by default because of binary size, build times and low overall support for the extension IIRC

Correct. There are also several variants of it, so if you can compile for the specific variant you have if you want. For general-purpose builds, it's not really worth the binary bloat at this point (given overall support) so it's disabled by default, but if you want to make a specific build for your cpu, by all means check what the latest hwy can do and build for just the best target (that should result in a smaller binary size and be faster than default, at the cost of only working on your cpu).

I'll try it out once I update my computer

the initial question was, how to pass an equivalent of -march=native to clang ? with the cmake/make process ?

-DCMAKE_CXX_FLAGS=-march=native

but you don't need that

you just need to disable the code that turns off AVX3 in hwy

By default gcc, if you don't specify otherwise, will compile with x86_64 as a target (to be precise : `-mtune=generic -march=x86-64`), which may skip some vectorization/optimizations

won't it ?

most of the performance-intensive code in libjxl is manually vectorized, and it tells gcc/clang to enable the appropriate vectorization targets in these functions

ok, I get it

thank you for the answer 🙂 I'll test with and without, just to see if it change the perf 😄

you can also tweak cflags after running cmake by running `ccmake` in your build directory

it gives you a curses UI that lets you change settings

might have to push `t` to enable advanced mode

Oooh, i'm learning something!

Don't tweak. We wants bug to be fixed

Se this: [libjxl/libjxl] Issue opened: #3123 Visible artifacts on the blue channel

new release: https://github.com/libjxl/libjxl/releases/tag/v0.9.1 artifacts should appear shortly...

congrats on adding dithering to libjxl!

It's too bad that it happened AFTER the 0.9 release

wait for the next one 😛

Things like color banding, which noticeably stick out and can make or break someone's first impression of a new format... it would be nice to rush that out to a release build...

otherwise people will get the wrong impression that the format is bad at handling dark gradients or something

Oh, also, someone offered a really great suggestion of using a blue noise pattern for dithering instead of the "ordered" pattern

And there's a great website that has a python script useful for generating matrices of any size with the blue noise pattern

Other Release: Save to Jon computer

https://discord.com/channels/794206087879852103/804324493420920833/1192234113223102535

I'm annoying WhatsApp

Guys, do somebody have admin rights here?

Jon i'm sorry but my hearing sucks.

I finished

Encoding ready

Jon is the Admin.

Blue noise needs a larger matrix than 8x8 though, otherwise tiling in the noise pattern will be more noticeable.

<@1028567873007927297> pleasE Help

Weirdo.

Last libJpg encoder

After 17 releases

My voice encoded

<@416586441058025472> could you please keep off-topic content to <#806898911091753051> ?

I did a 26900hz SINGING

Sure

thanks

do better

For the original tag me

<@416586441058025472> I think that this would count as off-topic, since this channel is <#804324493420920833> and this content does not appear to be related to libjxl

<@604964375924834314> Done.

Fab is an enigma

<@318186133320237058> That was in response to your blue noise mention by the way, if Fab made you miss it

I'm not convinced that blue noise would give so significant benefits when dithering to 8 bits... but I am willing to be proven wrong 🙂

C:/Documents/msys2/home/vincent.torri/ewpi_64/share/ewpi/packages/libjxl/libjxl-0.9.1/lib/jxl/cms/jxl_cms.cc:1327:39: error: function 'const JxlCmsInterface* jxl::JxlGetDefaultCms()' definition is marked dllimport 1327 | JXL_CMS_EXPORT const JxlCmsInterface* JxlGetDefaultCms() { | ^~~~~~~~~~~~~~~~

Then again maybe a bayer matrix is better since the dither noise is kinda predictable and slightly easier to compress for most compression algorithms

And yeah when dithering to 8 bits it doesn't matter much if it's a bayer matrix or a void-and-cluster matrix

But still theoretically it's cooler and visually at lower bit depths it does preserve more detail