JPEG XL

If you do a downscale (say 50%) of a camera picture, what happens to the noise model?

unless my math is off, you would use twice the read noise and half the PRNU that you would use for the full-resolution version

Might be useful to have a scale-factor parameter too

Even better of course would be fully automatic noise estimation

🤔 `-s 8 -q 30 --resampling=2` ```JPEG XL encoder v0.3.7 [AVX2] Read 1473x820 image, 47.2 MP/s Encoding [VarDCT, d6.400, kitten], 8 threads. ../lib/jxl/image_bundle.cc:62: JXL_CHECK: ec.xsize() == xs```

Looks like `resampling` does not work correctly with `-s` higher than `7` But, it would be nice if someone else could reproduce it (maybe it's a bug only in my build)

That or it's a problem with the alpha channel?

No, it happens on different images

Also on ppms?

Yep, sometimes without error messages, but encoding does not work

Maybe open a GitHub issue? We have to test s8+ better...

And `-s 1` ... `-s 7` works fine

not *super* surprised...

same for my build at s8 and s9

I am trying to convert a ton of jpegs to jpegxl in parallel

I have a concurrent lock-free queue to handle the multithreaded reading of files

I imagine I am supposed to use a single ThreadParallelRunner, and one instance of the jxl encoder for each thread, right?

but for some reason (and this might be some kind of c++ life cycle thing that I do not understand) when I use a concurrent queue, I get segfaults

but serial queue it doesnt segfault

does this sound like i am running into some known pitfall about threadparallelrunner?

yeah, don't use a single parallel runner

it's not thread safe

you can just pass a nullptr instead

ah ok cool thx

is `--resampling` a recent addition to the bitstream? I mean, I've enabled experimental JXL support in Chrome and any resampling factor > 1 doesn't render there...

It was implemented ok, then became buggy when we were optimizing the decoder to use less memory, now it's ok again

ah, so just a buggy decoder prematurely shipped in Chrome.

Yes, well, 'shipped' in an experimental option that requires you to manually enable it and that shows some warnings first

ah of course, I'm very aware of that, just wondered if this is (or was) a libjxl bug or very late bitstream addition.

Is this behavior correct? ``` PS > cjxl -kuku Unknown argument: -kuku Use 'cjxl.exe -h' for more information PS > $LastExitCode 1 ``` but ``` PS > cjxl 0.jpg 1.jxl JPEG XL encoder v0.5.0 [SSE4,Scalar] Failed to read image 0.jpg. PS > $LastExitCode 0 ```

I'm writing a script for batch encoding a large number of files, and I need to check if `cjxl` failed on some files.

Please open an issue, it should only return 0 on success

Done

oss-fuzz is reporting an out of bounds write in libjxl master https://github.com/libjxl/libjxl/issues/360

looking...

is it just me or libjxl-git failing tests for anyone else too?

nope, not just you

looks like https://github.com/libjxl/libjxl/commit/b4c01782212931014c603d337d762174a742c5b6 broke something

Hi, guyz im new here and im trying to find a way to utiliza jpgXL via Python is there anyone hrere that can help me?

I don't think we have any bindings yet

https://github.com/olokelo/jxlpy Maybe this will help?

hello again. by the way thank you for the wonderful library. i was wondering what is the minimal set of api calls to convert a jpeg stream lossily? i am setting encoder distance, setting effort, setting lossless false, and calling add jpeg frame, but it to just do lossless xcode. i was hoping to avoid the whole setsrgb/basicinfo stuff but if i have to set that also then i will try to incorporate that

The api only supports pixel input if you want to do lossy

So if your input is a jpg, you have to decode it yourself

ah ok thanks for the clarification 🙂

is that actually true? 😛

I think so - but not 100% sure.

Afaik there's an api to encode pixels with whatever options, and then there's an api to encode jpeg files losslessly

I *think* it actually allows to do lossy compression if you feed a JPEG

isn’t our jpeg-to-pixels code in extra/?

which I thought we ensured the C library wouldn’t depend on

I may be saying completely wrong things though

ah, we don't really have a good way of doing JPEG to pixels, indeed, so nvm

I mean, you can always do JPEG lossless recompression, decompress that to pixels, and then feed that to lossy JXL... xD

Heh yes, that you can do with just the api

I suppose we could/should add a "decode jpeg bitstream to pixels" function to the api, it would be easy enough to add that without implementing new stuff

With the added advantage that you no longer need libjpeg-turbo in your application if you need it only for decode

delta palettes are cool

Deltas are differences, in the case of this commit they are colour differences that are useful enough (frequent and high-valued) to be added to the palette.

This change only applies to lossy compression.

You're welcome. 🙂

are the DCT coefficients float32?

In the implementation, after dequant, yes. The bitstream contains integer quantized coeffs, but the quant table is float and the dequanted coeffs are float too

(we are investigating fixed point implementations)

Also, are improvements in decoding features in browsers not currently in development anymore? Like progressive decoding, multithreading, etc.

<@795684063032901642> any idea what the status is regarding progressive jxl?

Multithreaded decode in browsers, I don't think that will happen anytime soon

Firefox supports multi-threaded decoding and the difference on large images (especially lossless) is significant compared to Chrome Otherwise, I think it's better to have a complete full-featured decoder as early as possible, without waiting for the status when it can be enabled by default (even considering that it has been postponed)

That looks like an unnecessary dependency.

gperftools? Why is that a dependency at all?

Ah, I guess it's for tcmalloc

It is! tcmalloc is one of those performance tools xD

now the actual release is out: https://github.com/libjxl/libjxl/releases/tag/v0.5

already updated wikipedia italian and English

Somewhat weird that we went from 0.3.7 to 0.5 in the releases

0.4 shall henceforth be known as _the forgotten release_

we do not talk about it

It's in chrome 92-93 though, no?

_shh_

the forgotten release, that sounds cool, like a top secret 😛

new rules 😛 : > Q: Why we don't have libjxl 0.4? > A: Don't ask, that's top secret.

nope!

What is using the 0.4.x branch then?

chrome

but not the 0.4 *release*, which just doesn't exist xD

Ah yes

It's the 0.4.x series that has instantiations in chrome but has zero releases

A: because of… the incident.

When encoder heuristics become too powerful, it's better to nip it in the bud than to risk a catastrophic singularity. Just saying.

(I, for one, welcome our libjxl 0.4 robot overlords)

strange that firefox capture didn't captured github page

From the "release process documentation", it seems that the main git branch will always get the v0.3.7 tag from `git describe --long --tags`?

how can I dump the probability distributions generated by the predictor?

There's no code for that yet, afaik. You'll have to hack it in.

ah ok

know any good starting points? I imagine somewhere in encode_frame would be good to start, right?

What would make most sense I guess is to modify the code that dumps a MA tree (it's behind a compile flag) to also dump the distribution in each node

(that is, the ctx id, and then dump the distribution for each ctx id)

The MA tree is effectively a DAG, not a tree, since there is still context clustering happening which unifies tree leaves

great that's really helpful ty 🙂

definitely sounds possessed

you deleted your github account?

tbh you're thinking abt this case too much lmao

some things are best left unresolved :^)

Qt jxl plugin doesn't work when I use 'make install' to install libjxl but it works fine when I use arch aur package. Idk why.

I run cjxl daily on arch linux. It never happened.

https://github.com/libjxl/libjxl/actions/runs/1101231986 Linux builds for every commit 🙂

(windows builds are a WIP...)

<@!532010383041363969> It seems that similar color shifts that I wrote about on encode.su a long time ago still happen From AV1 discord: > **just leave weebs alone**: i was encoding a image with a bunch of noise on it with cjxl at s7 d1 with build 0.5.0 and i noticed a large color shift > source: > https://cdn.discordapp.com/attachments/673202643916816384/872678376979197952/song.png

> s7 d 1 > https://cdn.discordapp.com/attachments/673202643916816384/872678396788879461/fucked.png

Nice! Could you find the smallest image where this happens and file it as an issue?

likely this relates to having different color modeling and different compression curves, but it could relate to many other things like CfL

having this clear of an example should make it easy for use to fix this

does it also go wrong with s9 ?

and congratulations -- this is the worst d1 artefact I have seen 😄

Yep, same for `-s 9` Source:

`-d 1 -s 9`

Full resolution: `-d 1 -s 9`

Hmm, the color shift is still there, but the result is different 🤔

Full resolution: `-d 1 -s 7` Yep, with `-s 7` the differences are more noticeable

Cropped 32x32 source:

I think he had installed old jxl build from archlinuxcn repo. So it complained about missing dependencies.

don't want to promise anything but it looks like Arch devs are looking into adding libjxl to official repos 😄

nice

But even though they are taking more time imo

Too many dependencies. This is too much

they're mostly build deps, and they (and we) are working on trimming down that list for the binary package 🙂

I had to edit pkgbuild , remove unnecessary stuffs and compile

Is it possible to make aur-git binary package using .deb or static? Afaik many aur bin packages were made using deb packages.

libjxl-bin-git or libjxl-git-bin <:Thonk:805904896879493180>

you shouldn't ask me 😛

Later

Hmm

anyway it likely won't work

Oh my bad

because debian has different library versions

having said that - for a git package specifically building it yourself is a better idea

Does static work? Last file in the list

Looks like it might work.

yeah it probably would

``` install: cannot stat 'build/tools/libjxl_jni.so': No such file or directory ==> ERROR: A failure occurred in package_libjxl(). Aborting... ``` anyone get this trying to build the `libjxl` aur package?

fwiw it asked me which jdk to install and i selected 11

IIRC it works for me

Try deleting src/build folder and try compiling again

same error

I removed the install line and java depends and edited the configure to not build jni ha ha.

what goes in the configure for that?

I edited the configure line in pkgbuild

yeah but what did you change? idk what variable it is. `-DJPEGXL_ENABLE_JNI` or something?

Sorry. You don't need to edit cmake line. Just remove the ' install -D -m755 build/tools/libjxl_jni.so -t "${pkgdir}/usr/lib" '

oh ok thank you!

Yep, I've been chatting with him :P

It can be used for both

here: https://github.com/libjxl/libjxl/actions/runs/1106694153

i still don't find the file

jxl-x64-windows-static?

yes

i posted in jxl a screen of what i'm seeing

do not know what i should run

there is cjxl.exe and djxl.exe in the zip

mh wait

ah, doesn't seem to work if you're not logged in...

can you download them?

Yep **jxl-x64-windows-static.zip** ``` benchmark_xl.exe cjxl.exe djxl.exe LICENSE.giflib LICENSE.jxl LICENSE.libjpeg-turbo LICENSE.libpng LICENSE.libwebp LICENSE.zlib```

so you just need to be logged in github

could be worse I guess...

msvc and clang which compiler is better?

In this case, yes, Appveyor seems to be better, because it allows downloading without a login Msvc also uses Clang (for JXL), it's just a working environment

I understand, thank you 🙂

Although, I think if putting important versions in Releases, it's ok to download nightly builds only when logged in

I'm making some script that puts on my webserver all the nightly builds 😛 if someone will want to build an UI for it at some point, I would consider including it, otherwise you just get a directory listing 😛

If you can make a symlink that always points to the most recent one, we can link it on jpegxl.info

should be doable...

Uh, just noticed that I followed up on this on <#794206170445119489> 😄 anyway, now it's done: https://artifacts.lucaversari.it/libjxl/libjxl/latest/

why did you delete windows builds

they are there

I was rerunning the thing so you might have seen something a bit old

I suggest noting fabian's IP and hiding them only from him

bad suggestion

building libjxl from the AUR fails with ``` -- Installing: /home/damian101/.cache/yay/libjxl/pkg/libjxl/usr/bin/djxl make: Leaving directory '/home/damian101/.cache/yay/libjxl/src/build' install: cannot stat 'build/tools/libjxl_jni.so': No such file or directory ```

i got that issue too

i fixed it by removing everything about jni from the PKGBUILD

but it now successfully builds with an unmodified pkgbuild

i think i had some outdated packages the first time i tried building it, are you up to date?

yes, just updated the system a few hours ago

try this pkgbuild

works

is it possible to decode a subregion of an image?

in theory yes

but we never wrote that code

Relevant GitLab issue: https://gitlab.com/wg1/jpeg-xl/-/issues/28

Run binary in OS X https://github.com/libjxl/libjxl/issues/395#issuecomment-898024551

well, it was an easy fix 😛

discord comments can get lost more easily

not really. that's a role for an issue tracker

I though github already had something like that for a while..? Is that different from default text for issues

I quite like golang's default issue text tbh

interesting. could be helpful. though i hope one could just do free form text too if needed

having dedicated place for putting content needed for issue reproduction may remind people to put stuff there

but not all kind of issues are the same, and not all issues are problems

I don't like `blank_issues_enabled: false`. One can do that only after analyzing existing issues properly, and having good sample size of them, and even then I think it'll result in needlessly inflexible workflows

Btw, about `--resampling=x`, it still doesn't work together with `--target_size`? Hmm, I get it, `--target_size` works with `--resampling`, but the speed above `-s 7` does not work https://github.com/libjxl/libjxl/commit/71937822464bbdef422e8876a91a14dab775db66

Is it better than lanczos2 / lanczos2sharp

Is it just me or are resampled <:JXL:805850130203934781> files borked for everyone? It shows just the top-left part of the image...

And it's not just Chrome... Firefox Nightly does it, too.

They both treat the rest of the image as... transparent?

What about chrome canary?

That was literally Chrome Canary 😜

uhm, that might be bad

Yep

And it's not just in browsers, it seems to be a problem in libjxl Nomacs:

As has been said before, it would be nice to have a site with different test JXL images with different encoding options, preferably all on one page, for quick visual testing of correct decoding

libjxl failed, but `djxl` decoded those images flawlessly, as you might've seen in <#803645746661425173>

Sounds like something that is broken in the api?

yeah probably

This is a great progress, thanks!

Now the questions: 😜

Why do the license files have those strange extensions? E.g. the "LICENSE.jxl" seems like a jxl image.

Could more binaries be include please? As always, I'm particularly interested in the GIMP plugin.

There is gimp plugin from another developer. It works fine

Yup. But people don't want to search for builds. If someone feels like trying jxl, why shouldn't we give them the binaries straightforwardly?

I'm just saying there is gimp plugin. Idk about binaries.

By more binaries, you mean windows and Mac OS?

I think it's a standard way of writing it, i.e. LICENSE.brotli is a thing

good question, I'll ask...

could you open a github issue for this?

As I'm on Windows, I mean mainly the GIMP plugin for Win. Jxlinfo.exe could be useful too. Of course, Mac users might like to see Mac builds. 🙂

if you want to open bugs for the missing features... 😛

If you mean it seriously (in spite of that tongue), then yeah, I can open a bug…

I meant it seriously

I do sometimes overly add emoticons 😛 ( 😉 )

Can we call the builds at artifacts.lucaversari.it/libjxl/libjxl/latest "official builds"?

I dunno, they are the github artifacts that I downloaded

depends how much you trust me 😛

Quite a lot… 😉

so I don't think they properly count as official builds, but nightly builds from a dev is fair imho

Besides opening that bug, I'd also like to update the list of links to Windows builds in Wikipedia articles. So I'm looking for long-term reliable candidates. And artifacts.lucaversari.it/libjxl/libjxl/latest seems promising.

What's the best way for users to access nightly builds on github/libjxl/libjxl? It's hard to find them on https://github.com/libjxl/libjxl/actions.

mhhh not sure if I want to make it that much official

they are there

I mean, artifacts.lucaversari.it exists exactly to get nightly builds from actions

THe new one commit has less quality

the one with butteraugli quality 12.5 13

it was the one i wrote it's worse

doing better than the two delta perceptually don't mean good quality

that's why we shouldn't nightly to encode

especially with normal heuristicsa

the one kiemez mentioned 722772

was good

but on r/av1 i wrote something

like muscle get preserved, fat girls get preserved, orange get preserved, color get preserved, bpp is 0.477 0.478 bpp, quality when you tilt the screen is decent

what could go wrong?

all that isn't this like fake beard, d1 fidelity

So you don't want to have your link in Wikipedia articles? 😢

I think that readme is better place for that than in wikipedia page.. I mean ive not seen many wikipedia pages having such links, and im not sure if it really makes sense

thats just a detail of where semi-official builds are located of reference impl, which feels too zoomed in for page about format, overview of format itself, etc

> fat girls get preserved Okay, now I need a parameter for `cjxl` to get slim girls preserved.

link to builds should be provided somewhere ofc, and i think that libjxl's readme would be good place for that

either that or semi-official jxl site i guess

I'm not 100% sure how much it will stay up (it's a server here in my flat :P) - and also I think jpegxl.info might be a better place

just kinda feels wrong and too "implementation specific" to put that on wikipedia

<@!532010383041363969> until https://github.com/libjxl/libjxl/commit/71937822464bbdef422e8876a91a14dab775db66 it looks pretty usable but not sure how to measure quality, i have few files in png and the one i have are in 7z 7std files

btw, you know you can send PRs to jpegxl.info, right?

I know and I already did it. 🙂

Guys, do you find it a good idea to start putting the builds to https://github.com/orgs/libjxl/packages?repo_name=libjxl ? I'm about to submit a feature request for it…

I believe we'll do that for releases

Nightly builds are not official in the sense that they might have severe bugs

Maybe change it to `LICENSE.libjxl` so it doesn't look like a <:JXL:805850130203934781> image?

Done 🙂 https://github.com/libjxl/libjxl/issues/459

By the way <@!794205442175402004>, what about this rebasing this (and merging into main branch)? https://github.com/libjxl/libjxl/pull/359#issuecomment-899723832

And this: https://github.com/libjxl/libjxl/pull/358

right - not now, in the middle of funky segmentation stuff

Ok... 😛

Yeah, this issue should probably be closed (since 176 is closed anyway): https://gitlab.com/wg1/jpeg-xl/-/issues/212

...or at least let's wait until PRs 358 and 359 are merged, let's see if they'll fix GitLab issue 212.

You mean that automatic photo/synthetic partitioning from <#794206170445119489>? Damn, that's awesome!

they can ask but it isn't going to happen 😛

<@!768090355546587137> <@!179701849576833024> Question about encode api: suppose I want to add logic that (at some slow speed setting) detects cases where lossless might actually be smaller than lossy. How would that fit in the API? Currently when a frame is added, the decision whether or not to do XYB has already been made (image header has been written already). So it's too late then in EncodeFrame to say "let's try lossless too and see if it is smaller"...

no idea 😛

it doesn't really do lossless, it quantizes the colors in XYB, encodes that as a modular image, and subtracts what it encoded from the main image so what is left is (more or less) black

and it just assumes that that will be better/smaller than using dct on that part of the image

the heuristic itself is quite fast, as you can see in the pull request from the example image where everything is photographic so it doesn't do anything (but it still analyzes the image to detect if there's something nonphotographic)

you could use EPF to reduce the problems of quantizing xyb, btw 😛

> at a significant cost in speed (since every pixel now has to be en/decoded twice, and modular is slower than vardct) Is there a way to just skip the block?

you can for example make the modular frame smaller and use more patches

Can just skip modular frame by changing jxl specification

`--resampling=2` for example from a 1920x1080 image is this some sort of upscaling from 960x540?

yes, it will first downsample in the encoder, then encode, and in the decoder upsample again

is it possible to do fractional resampling eg 1.5

you can add `--already_downsampled -q 100` if you just want to see what the upsampling looks like

(the default upsampling that is, in principle we can signal and use different upsampling kernels, but the encoder never does that atm)

fractional resampling, no

you can however set a header field for the "intrinsic dimensions" of the image

which is supposed to be a hint to viewers that they should rescale whatever is encoded to those dimensions

I don't think anything implements it yet though

<@!768090355546587137> <@!795684063032901642> chrome should respect those intrinsic dimensions and treat them in the same way as the recent Exif thing (see https://github.com/eeeps/exif-intrinsic-sizing-explainer)

that field was added to the jxl header exactly because it has render impact and all render impacting 'metadata' should be in the codestream, so you don't need Exif for that

There is no benefit from reducing the resolution on one side only, like anamorphing does?

could you make a test file for this and make chrome do the right thing with it?

there might be in some weird cases, though in normal cases I don't know of any perceptual reason why horizontal should be different from vertical

As far as I remember human perception is a little less likely to notice such losses, although I may be confusing this with some other research

there are 3 different upsampling mechanisms in the jxl bitstream: - for YCbCr, there is the usual chroma upsampling like in JPEG, where horizontal and vertical sampling factors can be different and each of the 3 channels can be set independently. So you can do 420 and 422, for example. When chroma upsampling is used, vardct is limited in what is allowed though (has to be dct8x8 etc), and this exists only for jpeg recompression (it does not exist in XYB). The upsampling is done by the decoder and it is a fixed simple upsampling that matches what libjpeg-turbo does. - all 3 color channels can be upsampled at the same time, with a factor 2x, 4x or 8x, and same with any extra channels like alpha (extra channels get their own upsampling, but it has to be at least as much as the color, e.g. 1x color, 2x alpha, 8x depth is fine, but e.g. 2x color, 1x alpha is not allowed directly in one frame - you can still do it by using patches / two frames). This is a fancy non-separable upsampling filter, done by the decoder, and the upsampling kernel can be signaled. It's also per-frame. - for the whole image, there's a header field for intrinsic dimensions. This allows arbitrary non-integer resizing, e.g. to get a 1000x800 image from 256x128 pixels or from 1024x768 pixels. It is not done by the decoder though and the resize method is not part of the spec (so it is the application responsibility to deal with this).

So, was a different type of quantization ever explored for jxl?

I've been doing a lot of research regarding what was implemented(or enchanced) into AV1 from the Daala project, and I'm always reminded by Perceptual Vector Quantization: https://jmvalin.ca/daala/pvq_demo/ Other stuff: https://jmvalin.ca/daala/revisiting/

Nah, we didn't... Vector quantization seems like it would have been very useful to avoid ringing, but we didn't try it

I am thinking about doing trellis quantization though (not the same thing at all, ofc)

In my opinion, I think trellis quantization not suitable for high quality, mozjpeg already have trellis quantization feature, but this feature will blurry some detail, so for high quality jpeg I prefer libjpeg.

trellis quantization is just trying to get a better tradeoff between size and quality - AFAIU it should always be possible to get as high quality as you want with trellis

at the moment, -v -v is enough I think

It probably depends on memory/cache properties

I *wish* it was that simple!

it also depends on things like the cooling solution that you have

I think on most "normal" computers you do want to user hyperthreading - you want to skip it if you have a super-powerful PC

but if you have fewer than, say, 8 physical CPUs...

yeah you probably want as many threads as you can

I think when # physical CPUs is <=4 it's always a good idea to use hyperthreading... at least on x86

maybe

user time is the total time your CPU spent working on it

it probably means it's not a good idea if you also want to do other things at the same time, and it will probably get your cpu hot faster (possibly causing thermal throttling), but otherwise I think real time is what you normally care about

whether you care more about CPU time or waiting time, that's something you need to figure out 😛

yeah that usually works 😛

> Pentium G4560, a single 16GB DDR4-2400 RAM > a low-spec desktop mini PC ...you're spoiled 😜

I believe I will have to quietly avoid commenting on this conversation... 😛

Haha

I don't even have a pc atm

my main machine from late 2012 to late 2015 was a 11.6" netbook with a Celeron 877 and 4 GB of RAM 😁

well urs has avx the pentium doesn't ;p

AVX as opposed to SSE4?

Once I built cjxl with debug flag. It took 2.4 minutes vs 0.8 sec to encode same image with same settings.

hah not surprised

I don't expect it to change more than 30% or so, but I didn't really measure

I tried to compile cjxl with -march=westmere. Even with -mno-avx2. Still it compiles with avx2. 🤔🤔🤔

seems like a lot of effort

~~march native on pre haswell cpu?~~

dynamic dispatching makes all of these tricks not work

you *can* just build libjxl with bundled highway with `-DHWY_DISABLED_TARGETS=HWY_AVX2`

How can I get statically built 32-bit and 64-bit libjxl.dll files? Using Docker with STATIC=ON and x86_64-w64-mingw32 build target resulted in no DLL. With non-static build it appears, but as I understand, it requires several other dlls? Or not?

Also, trying to build i686-w64-mingw32 on docker fails for me: https://pastebin.com/rY9sJwyJ

On both tag v0.5 and latest commit 84f0807

no idea ... <@!604964375924834314> maybe has an idea?

hm, I don’t think I have seen this before

but since the error is in gtest, maybe disabling the tests would be enough

I don’t remember if we have a cmake option for that

also, not sure it’s related at all, but clang 7 is a little old, isn’t it?

I did it according to doc/developing_in_docker.md , would you recommend to put another number instead of 7?

cmake has one by itself, ENABLE_TESTING or DISABLE_TESTING or what they call it

you can go with the newest version you have, like 11

Deleted `enable_testing()` line and added `option(BUILD_TESTING "" OFF)` line in CMakeLists.txt, still fails.

CMakeCache.txt might be a better place to set options

https://pastebin.com/724jqBTU

oh, yes, static build, somehow I missed that

yeah, static builds are tricky

```cpp size_t compressed_size = compressed.size(); uint8_t* result = reinterpret_cast<uint8_t*>(malloc(compressed_size + 4)); uint32_t* meta = reinterpret_cast<uint32_t*>(result); meta[0] = compressed_size; ``` `tools/jxl_emcc.cc` line 37

This reinterpret_cast depends on the endianness of the hardware.

ARM by spec is allowed to be big-endian, even though x86 is little-endian locked

ugh

yes

ARM is always also little-endian in practice afaik, but yes

one option is to just add an endianness check in a configure script with `#if` but it's ugly

how did you find this? is it a practical problem or just a theoretical one at this point?

I checked the issue tracker and someone provided a stub issue pointing to that file on line 40 for something unrelated. but since I looked at that file I was like "wait, what's going on"

I learned this when working with PNG and found that "network byte order" is a pain for hardware since it's usually the opposite

yeah, it's basically always the opposite nowadays

but yes I discovered this cause I looked at the source, not because I had it fail to work on big-endian hardware

arm spec actually allows the operating system to flip the virtual address space of a particular process to a different endianness

which is kind of odd

but I think that was for emulation/testing purposes

to the point that we no longer find implicit little-endianness assumption bugs because big endian is so rare

yea, if this were my own project I would just refuse to support big endian hardware

but this is a reference encoder for a standard so I feel like caring about it is annoyingly important

yep

please open an issue about it, we have to fix that code to be endianness-neutral

this is why we have this stuff: https://github.com/libjxl/libjxl/blob/main/lib/jxl/base/byte_order.h

maybe we should also somehow add some big-endian environment to the continuous integration

opened

armbe exists on qemu I believe 😄 that one we wouldn't have caught because IIRC it's only compiled under emscripten, but...

<:megapog:816773962884972565> <:megapog:816773962884972565> <:megapog:816773962884972565> https://aomedia-review.googlesource.com/c/aom/+/145301 Thanks a lot <@!604964375924834314> for your efforts <:FeelsAmazingMan:808826295768449054>

https://github.com/libjxl/libjxl/issues/540 lol

Let's ask the repo sync bot to implement it :)

3 Major Developers

this is almost as painful as reading fabians msgs but looks like she wants exif info in her jxl photos cuz they didn't carry over during the encode?

oh wait

exif support on djxl?

idfk

<:kekw:808717074305122316>

i think they are saying that they expected djxl to carry exif data from jxl to jpg, and that trying to add it manually failed

oh looks liek she was also complaining bout exiftool not supporting jxl?

my brain hurts <:kekw:808717074305122316>

im not really sure though because i thought it already did that?

fabian actually makes sense 30-90% of the time

my eyes hurt everytime i see a screenshot from fabian with his custom fonts <:kekw:808717074305122316>

confusing

In any case there is stuff to be done to implement compressed metadata and have tooling that makes it easy to handle it

I just thought it was funny that 1) the issue specifically names devs that should do it, and 2) one of those 'devs' is a bot

<@!768090355546587137> regarding the encode api and properly handling jpeg recompression of exif-oriented jpegs: I think we have a bit of a conceptual problem there, since orientation is an image-global thing but jpegs are added as frames (so when the imageheader is already written)

``` CMake Warning: Manually-specified variables were not used by the project: JPEGXL_ENABLE_GIMP_SAVING ```

just built from git master

is this a problem or just forgot to clean it up

nah, doesn't matter - I think Gimp saving was rewritten by <@!877735395020927046> and I guess it doesn't use the build flag anymore?

that matches my understanding

How can GIMP plugin be built now? <@!179701849576833024> would it be possible to include it in CI (and thus in your nightly artifacts)?

I think it just work if you build manually

as for CI - at some point we will

Unfortunately I'm really space-limited so I can't risk setting up `crossroad` on my WSL

I could try to see if I can get it to build

ah, but it's not included in the debian packages

nvm

I don't know enough to get it done

but if someone wants to figure the debian packaging and the CI, we won't mind 😛

maybe <@877735395020927046>? 😄

although I don't know if they should go in libjxl, jxl or a new jxl-plugins package...

I would be able to build it for Debian (I'm on WSL), but I use GIMP on Windows and I can't build EXE

oh I mean adding it to the CI

shouldn't be *hard* (no idea for windows though), but...

I took out the flag because it was a pain, and the codestream is supposed to be stable now. Adding the plugin to the current debian packaging should be straightforward. If you want to separate it into its own package, it would be good to get it to build on its own with cmake, like the examples do.

that shouldn't be hard

OTOH I've seen that there's no gimp for vcpkg, so for windows it's likely harder

and either way beyond my capabilities 😛

for windows, i got the gimp plugin building with msys2 and crossroad

ah but our CI doesn't use that

i don't know how it's setup behind the scenes.

I don't really know either - I know it's github actions + vcpkg which IIRC it's not related to/usable with msys

but I'm not really sure either

it looked like crossroad was grabbing mingw and msys packages... then putting them in the right place for cross compilation from linux. so if someone knows how it's supposed to be setup, i guess they'd just need to put the files in the right place. as long as they're compatible with the compiler/linker.

I had posted a windows binary of the gimp plugin on r/jpegxl, but it actually got some downvotes, so I removed it. People on r/gimp didn't seem that interested, but at least didn't get downvoted... It has a different save dialog and some bug fixes compared to the one currently in the libjxl repository.

I'll try to figure out the package building 🙂 If anybody can ping novomesk to make `.deb`s in github CI then I could also include that in the packages on https://artifacts.lucaversari.it/

Why TF would ppl downvote you...?

¯\_(ツ)_/¯

haha. I don't know. I thought Windows users would like a binary, since it wasn't so easy for me to figure out how to build it. (Even though I'm just using scripts that other people built.)

I'd like to have it either, but apparently some redditors are... well, redditors.

Also can faster_decoding use negative values or is this not implemented yet? https://user-images.githubusercontent.com/8353098/131713081-1ec09220-a8e8-4b56-9424-f6abec6f22ca.png

I dunno if we implemented it already, but for lossless it might make sense to have "slower than default" decoding, which atm would be `-E 3`

The GIMP plugin doesn't have access to JXL features that aren't exposed in the API. Faster decoding uses a value from 0 to 5. `cjxl` seems to use the absolute value, so -5 is treated like 5.

really? I thought it only went up to 4 😛 and I don't remember doing anything special with negative numbers

<@!179701849576833024> you're right... it's 4. don't know how 5 got in my head.

Clang vs GCC would only affect the speed of cjxl right?

It wouldn't have an effect on quality?

Only speed, and Clang is faster than GCC on libjxl.

Alright. I'll just use GCC then, as I already have it installed, and don't need to much speed.

theoretically, there could be (very minor) effects on quality between different compilers / compiler versions / target platforms, since float arithmetic can be slightly different depending on the actual instructions and order of operations things get compiled to, which could affect rounding and occasionally change the results of encoder heuristics

Fair.

in float arithmetic, it's not even necessarily the case that A + B - B == A

but any differences will be in the least significant bits of a 24-bit mantissa, so I wouldn't worry about it

catastrophic cancellation would like to have a chat with you 😛

(not that I really expect that to happen in jxl)

I'm sure you can construct pathological input images where results of encode+decode have a relatively surprisingly large variance depending on compiler/architecture, but I doubt you can make a visual difference that way

upsampled x8 spline creates a 8x8 checker pattern. intended?

upsampled how?

i used Upsample 8 in the tree

likely not

is it on the latest main or is it on an older version?

i am currently on <https://github.com/libjxl/libjxl/commit/0a6dc0acb868f78c4cebee4c23dd9a338bcd71fc>

does it still happen before https://github.com/libjxl/libjxl/commit/9d03bfe8111f49210e2d096688dd4db303654270 ?

I think it should but what do I know 😛

yeah still happens in the parent of that

interesting...

my guess is it's the upsampling, and would not be just splines

what happens if you render the spline normally, and then encode the file with cjxl --already_downsampled or whatever was the option?

decoding to pfm say

what

remove Upsample 8 from jxl_from_tree, decode to pfm, then encode that with `cjxl --resampling=8 --already_downsampled`

But the plines will get lost with pfm?

well, they will be converted to pixels

but this will tell us if this is a spline-specific problem or not 🙂

yup it's there

to this

just to clarify - those images are not zoomed in, right?

can you share the `.jxl` files?

using cjxl --resampling=8 --already_downsampled

tree

... maybe I'm blind, but I don't really see the issue

djxl testspline.jxl testspline.png

when zoomed in, without any upscaling, the pixels show a pattern

ah, I see what you mean

that's just the 8x upscaling I think

ok

i would have hoped maybe splines can get treated differently

I guess it could be done in theory

<@!604964375924834314> wdyt?

yes, in fact I think at some point we had an implementation of spline drawing for downsampled output

it’s feasible in principle to scale the parameters

(I think)

strange, I was almost sure that there was a point where this was implemented, but I can’t find a trace of it in the history

maybe I wrote it locally and never submitted it

let’s see

which of my 140 branches could it be 😒

ah, probably `spline-downsampling`

from June 2019

“Make spline drawing compatible with downsampling (modulo the crash)”

the what now

ah there is another commit “Don’t crash” after that