JPEG XL

it has only 2 nonzeros

that *has* to have been deliberate

yes, it's from the 3x3

yep

what I mean is: 0/1/4 are the only rows that touch the 3x3, and 2 make sense, so the 3rd one can't have been hand-picked

the order of the rows doesn't make sense to me! if you had that 0/1/4 together earlier then the normalisation step would need to do less

rather we probably chose various candidates for things that make sense for the other two and picked whatever was left for the 3rd

I think it's such that in a 4x4 order the two weird pixels are still in pos [0, 1] and [1, 0]

it's a bit of a shame really

hand-wavingly, normalisation makes writing a 'fast AFV' implementation more cumbersome, e.g. if you do it by using the structure of the matrix i posted, then multiply by the triangular normalisation matrix. So the fewer off-diagonal entries in the normalisation matrix the better

well, permuting rows is very cheap, at least 🙂

actually scratch what I said - it makes no difference as most of the rows are mutually orthogonal anyway. There are only 3 off-diagonal entries when you normalise, which is not a great hardship

well, all of the rows are, no? (at least as written in the spec)

but yeah, you probably can factor the 13x13 part

i meant pre-normalisation, which is where the structure is - e.g. see both versions of row 5 a few comments ago

factoring the 13x13 is how i originally found the butterflies / things I thought were sub-transforms to begin with

less importantly, did you consider doing a graph transform for the whole 8x8 block, rather than a 4x4?

consider, yes

but IIRC we decided it was going to be too slow

although admittedly we could have implemented it in less than quadratic 😉

(I'm not sure if a butterfly-ized 4x4 transform would be meaningfully faster than the one we now have)

i think i'm more sure....! are you trying to incite me to write code? 😛

c'mon, you're doing a schoolbook matrix multiply which includes multiplies by zero

yup but it's SIMDfied and very pipeline-able

IDK

I never tried 😄 it's not a significant fraction of the running time anyway

What's the status on: https://github.com/libjxl/libjxl/pull/1366

MSYS2 developer needs help to review his patch for static build: https://github.com/msys2/MINGW-packages/issues/11697#issuecomment-1133898272

any 0 duration animated should only be outputting 1 image right? jxldecoder api only gives 1 image but djxl makes an apng

coalesced decoding should give only 1 frame, non-coalesced decoding should give all layers

how would i make it so djxl only makes 1 frame png

Could be that djxl is a bit buggy and doesn't do the right thing

oh

in theory you could always interpret an APNG just as a PNG though, since PNG is forwards compatible, right?

Here's a GIF from Samsung's Smart Select (thanks <@770059673583484928> for attaching it below with Nitro!) Encode it. Losslessly, lossily, it doesn't matter. Open the resulting JXL file with GIMP. Everything's OK, right? But now try opening it with Chrome or Windows Gallery via jxl-winthumb. Well... technically it will work, but watch the animation go nuts in Chrome or view those borked frames individually in Windows Gallery. Oh, and I almost forgot: try decoding that JXL back to GIF or even to APNG or WebP. GOOD LUCK.

I suppose the implementation of animated jxl could use some more love

It's just not very appealing to work much on that - it's almost always better to use video codecs than animated image codecs, so it's a somewhat pointless exercise. We don't really want to replace GIF with JXL...

I mean, GIF is still a thing because it's pretty much supported by anything in the world, being that old. But if you're willing to give up that level of interoperability, you can just as well use av1 or whatever.

I just wanted to be consistent and have *everything* from my screenshot collection (23,318 PNGs + 3,423 JPGs + 5 GIFs) on my backup DVD in JPEG XL. Thankfully I can count all GIFs on the fingers of one hand (literally), so that's not a problem – I just didn't include them in the backup.

djxl makes the n layer 0 duration jxl an apng with n frames and some default minimum duration which I assume is wrong

OH COME ON, VCPKG STILL FAILS

with msvc22?

or do you mean the gh pipeline?

Yes, the latest commit

Try running the pipeline in your fork

is it possible to disable the palette transform in cjxl? I tried `--palette=0` but it didn't work.

Try --palette=0 -X 0 -Y 0

Otherwise it might still do channel palette

Has the experimental/fast_lossless been integrated into libjxl yet? if not, is there a plan?

Probably post 1.0

Since it requires a totally different code path, which seems to be a bit risky to add at this point if we want to stabilize things...

But yes, at some point I want the fastest lossless encode settings to do that

Hi! jpeg xl looks really cool, so I've been trying to see if I could use it for a hobby Golang project. Any idea how feasible that would be? The ideal solution would be transpiling libjxl into Go with ccgo or cxgo, but maybe that's crazy talk. On the other hand, the transpiled sqlite I've been using works great. Less ideally a cgo wrapper wiuld make sense, or even just calling cjxl directly, but there's probably more overhead to those approaches 🤔

depending on what you want to do I'd consider just writing a oneshot encoding/decoding in C/C++ and wrapping those

if you need more fine-grained interactions that could be hard

transpiling is ... probably not an option

(happy to be shown otherwise though)

I was thinking of generating thumbnails for a photo gallery / google photos style application. The idea was to generate e.g. a 1024px or 512px progressive JPEG XL, split it up by the ... progressive uhh boundaries / checkpoints (FF0A in regular jpeg?), and store each of these "levels" next to each other in a sqlite database with an index. Compared to just e.g. generating 5 separate thumbnail files on the file system as you would normally do, my hypothesis is that it would 1) take up way less space as only one full image would need to be stored, 2) be faster to read for smaller thumbnail sizes as sqlite would do fewer syscalls than reading separate files, 3) be faster to generate due to 1), and 4) be really cool 🙂

This hinges a bit on jxl operations not having a lot of overhead, which is why integrating as directly as possible is probably a good idea 🙂

I think you're best off writing a couple of specific c wrappers that would allow you to just do ~5 calls for each operation

(also I had experiences with DBs-as-a-filesystem and i can say I strongly recommend against doing it but up to you :P)

Yeah I might try that first and then fall back to just calling cjxl if I find I'm fighting C compilers too much 😅

It's a hobby project, so the worst that can happen is that I waste some time and learn a lesson 😁

Fair enough, fair enough ;)

Banking on this https://www.sqlite.org/fasterthanfs.html

Thanks for the help! 🙂

For some reason cjxl cannot read a jxl o_O

Do I really have to decode it into a different format with djxl first before I reencode it? Isn’t that dumb? What if I want to transcode a lossless image to a lossy one? And the source is jxl? Why would libjxl tools not be smart enough to decode a jxl image? 0_o

cjpeg also doesn't take jpeg input, and cwebp doesn't take webp input

Actually, mozjpeg can take a JPEG input just fine, and CWebP can take a WebP input just fine 🙂 cwebp: `Supported input formats: WebP, JPEG, PNG, PNM (PGM, PPM, PAM), TIFF`

So yeah, I do think cjxl should support JXL input 😄

Ok, but what should it do with it?

Decode to pixels? Transcode losslessly?

We don't have code yet to do lossless transcode jpegtran style

Decode to pixels, that's perfectly fine 😄

Just add that: "JXL input is supported, but lossless transcoding of JXL to JXL is currently not supported", or something similar.

If someone wants to transcode a lossless image to a lossy one. They should not have to convert the image to another format first...

Just makes no sense... regardless of what others are doing...

"X behaves like that too" doesn’t change at all the fact that it's ridiculous

Can webp even do lossless?

I don't really have an opinion on whether cjxl should have a webp decoder or not.

But you would expect it to be able to decode its own format...

It already has a jxl decoder but it isn’t smart enough to use it.

Especially with the minimal generation loss being one of its touted features, it's surprising that you have to first turn it into another intermediate format before reencoding like png or ppm

I'm not opposing this, but please note that cjxl/djxl are just example applications. Eventually, most jxl encoding would not happen using these tools, but directly in whatever application you are using (which probably still uses libjxl behind the scenes, of course)

Command line tools will be always useful. Most applications probably won't be exposing more advanced features.

Isn't image magick enough?

Agreed wb and arun

Also I hear graphicsmagick is maybe better than imagemagick

It's not a big dealio but it is mildly annoying and bizarre

Also is cjxl_ng any good? Can that accept jxl input?

The _ng tools use the api, the current/old tools were made before the api existed

Eventually cjxl_ng should become cjxl

Imagemagick is so overwhelming. In my experience, simple programs built for the task often do a better job (or provide more precise control) than complex programs that are trying to do everything.

Does cjxl_ng support lossless compression? I was unable to make it working. Or am I doing something wrong?

"-distance 0" did a lossy compression and "-distanec 0 - modular" produced a very big file, bigger than the original PNG

Hm, that's not right. Maybe file a bug report

A little correction. If I use only either "-distance 0" or "-quality 100", cjxl_ng won't produce any file. I'm using the version from package v0.7.0 ab775e2

I noticed there hasn’t been a release in some time so I have been using git. I hope jxl or something that builds on jxl takes over the world soon. Would be nice to have our phones use it instead of heif

Anyway, if you don't care too much about lossless JPEG transcoding and animation, you can use ffmpeg directly to transcode JXL to JXL 😄

Are there any good tools for windows I can recommend to people for transcoding and compacting their images with JXL?

Xnconvert

Seems pretty cool.

What is the next usecase that libjxl will be further optimized for?

How come the Windows builds do not include the actual shared lib .dll but only has the "example programs?" That's kind of weird for a library to not actually have the library.

Also for some reason cheetah is giving me smallest files.

Is there any situation where the default effort level 7 produces smaller files than the faster level 4?

Level 4 seems to produce the same quality as higher levels at a much smaller filesize

it is definitely possible, mostly because that effort level is not what people normally think of

cjxl is not updated

since 4 months

no sharpness improvement

it will be abandoned

as is not good even fo simple sceenshots

Ah I see, effort 4:cheetah is a little bit more noticeably blocky in some images. So it's trading filesize for more blockiness.

in genal cjxlng with way less effot

like default is 3

like cpu 6 aomenc but x2 faster

the quality is way better

libjxl is being updated, but apparently not enough for it to have a new "release" in a long time.

the image looked from far

look like they have better bitdepth

I am using the git version since it seems to have been a long time since the last release. At least, a long time for a piece of experimental software that aims to be the next JPEG.

seven month

New/experimental software like this typically gets new releases often, like dav1d

yes but 0.6.1 is not that good

I haven't compared 061 with latest git so I can't tell how big the changes are. Apparently not big enough to warrant a release.

like at 0.451 0.600 bpp

most of the changes since then don't effect much encoding wise

o 0.390 butter

also ringing

with xnview the quality at q80 is good

but jpeg xl in general it tends to produce an heatmap

and reduce the sharpness

at least not

but it gives less hot images if you use nornal effort

ffmpeg -i input.mp4 output.apng ffmpeg -i C:\Users\Use\Videos\e\a.mp4 C:\Users\Use\Videos\e\a.apng -f apng output.png for %i in (C:\Users\Use\Videos\e\*.apng) do cjxl_ng_main "%i" "%i.jxl" for %i in (C:\Users\Use\Documents\vvvf\*.png) do cjxl_ng_main "%i" "%i.jxl" for %i in (C:\Users\Use\Videos\l\*.jpg) do cjxl_ng_main --effort 8 --lossless_jpeg=0 "%i" "%i.jxl" for %i in (D:\video\june\phone\Camera\Camera\jpg\*.png) do cjxl_ng_main "%i" "%i.jxl" ffmpeg -i input.mp4 output.apng

Everyone wants a super efficient and responsive image codec like this to replace all legacy formats and be supported in all image viewers and web browsers and cell phones... Especially if it can losslessly crush existing jpeg.

yes but they have to garantee a big enough improvement

It's kind of odd, considering, if cjxl is supposed to be an example program demonstrating how to use the library, why doesn't it use the library? And only cjxl_ng uses it, but is less tested and experimental?

472x480

cjxl doesn't use the api

cjxl (iirc) is eventually being replaced by cjxl_ng

jon said it

even lossless jpg transcode you can get 475 to 400kb

with cjxlng

at effort 3

default one

i don't know if it also fo losslessjpgxl

Isn't it strange that cjxl didn't use the api to begin with, if it was meant to be an example of how to use the API?

Something doesn't add up somewhere

they were too bugs

yurume discovered them

and joined as sdibaska

he's very active

And if such a basic command line program as cjxl_ng can't easily use the API, and is still experimental with bugs and weird behavior, then is it even a reasonable expectation to expect web browsers to be able to adopt it?

like some images fo example this

effort 8

it doesn't even look sharp enough

no different than avif

the wood looks weird

animation are unconposite frames

Anyways sorry for asking so many questions, just thinking out loud and getting confused.

It's not really animated jxl, it's concatenated still files

Some things just don't add up...

JPEG XL thing is not lacking of low bitrate perfomance

https://ewww.io/2022/06/02/what-in-the-world-is-an-avif/

Yes, it's just a little bit perplexing to me. Like it seems unfair or unreasonable to demand or expect that other, more complex pieces of software should incorporate libjxl support if even an example program like cjxl has a hard time using the actual libjxl API.

hope for cjxl in android

and aom 3.4.0 rc1 alreasy exist june1

https://aomedia.googlesource.com/

(btw)

people will still use compression and so at 1.0 they need to finalize the encoder

at least without animation tricks

Oh now that I'm rereading that it makes more sense

So the API didn't exist at first and there was no shared lib, only cjxl/djxl it sounds like?

hello, I'm trying to fix up libvips jxl write with float data and I have a dumb question! it looks like the libjxl tools always turn on JxlColorEncodingSetToLinearSRGB() for float data I guess this assumes 0 - 255, is that right?

I ask because libvips supports both float sRGB (0 - 255, gamma, float data) and scRGB (0 - 1, linear) I suppose I should convert everything to float linear 0 - 255, but I wanted to double check I had a quick look at the docs but I couldn't find this point being discussed (probably missed it)

Sorry, one more question, I suppose non-linear float JXL is a possibility, looking at the API Is this something I should allow for?

The codec was designed first, then the api. So we had a cjxl/djxl a long time before we had an api.

The decode api is more mature than the encode api. We expect more applications to need decode than encode, including browsers.

No float is 0.0 to 1.0

great! thank you and always linear? or is it an error not to call JxlColorEncodingSetToLinearSRGB with float data?

Linear by default

But you can give data with other tf if you want

should I allow for that in my decoder? how can I tell if the person doing the encoding wrote linear data?

Decode side, you get pixel data and a (possibly synthesized) icc profile that describes how to interpret it, including the tf

ah I didn't realize it synthesised profiles if necessary that's great, thank you very much

if you look at resolution cjxl will be obsolete by a week

if you look at bpp not

you can keep it

but better using the final version

https://discord.com/channels/794206087879852103/794206087879852106/985485293991907338

we continue to chat

Fab you produce words, but I have a hard time making any sense out of them.

the question is

https://discord.com/channels/794206087879852103/804324493420920833/985516870939574324

https://discord.com/channels/794206087879852103/794206087879852106/985485293991907338

-q 92 -e 4 is about the same as -q 90 -e 7

But much faster obviously.

Efforts higher than 4 don't seem to be worth it.

Just a waste of time and CPU

Without giving any extra benefit in terms of economy of compression at all

Seems to maybe help a little bit with lossless mode maybe though?

It depends a lot on the image

E.g. if your image benefits from patches, like a screenshot with text on it, then e7 will be noticably better than e6

But if the image doesn't benefit from patches, then yes sure the search for them is just a waste of time

Higher effort _should_ give better quality per bit _in general_

And it _should_ also give more consistent results as effort gets higher

These are hard things to test though, so at the moment the only certainty is that slower settings are slower

Well if the faster settings only perform a subset of the work the higher settings do, and don't do anything novel that the slower settings don't do, then the slower settings will never produce larger files for the same fidelity

I compressed a picture of a handwritten note and it didn't seem to benefit from patches.

Using the lossy dct encoder at least

I'm aware this probably isn’t how it works and the faster settings probably are not a strict subset of the work the slower settings do.

would be nice if jpegxl.io used https://github.com/chafey/libjxl-js for crippled browsers like Apple and Firefox

They mostly do different things, slower settings are not aiming at smaller filesize but at more accurately reaching the distance target (and also trying to be smaller)

cjxl looks for recurring, highly similar patterns to encode them just once, like deriving a sprite sheet to reproduce parts of the image content. Currently, it looks for high-contrast patterns, targeting text. Patches like these won't work well on photographs in general, which don't tend to feature such strict repetition.

dumb float question: there are (roughly) the same number of floating-point values in [0.0,1.0] as [1.0,MAX], right?

yeah, for float (IEEE 754 binary32) there are 0x3f800000 numbers >= 0 & < 1 and 0x40000000 (finite) numbers >= 1

hm, that's more than i thought. the "missing" ones are subnormal?

Suggestion: something like a 'tune' flag or a "screenshot" mode and/or "synthetic" mode and "photo" mode, some way to provide the encoder with a hint about what sort of image it's encoding, to help it decide how to encode the image. Or maybe just make the encoder smart enough to be able to tell what sort of image it is compressing and make it better at avoiding wasting time on algorithms that won't improve compression density.

If the default speed is 7, and 7 is a waste of time for the majority of images (photographs), maybe the encoder should try to determine whether it's appropriate to use a potentially expensive algorithm

Like looking for tiles

I don't know if tiles are actually expensive though, just hypothetically speaking here

Shouldn't be that expensive when it's a photo

Well either way the difference in compression density between 4 and 7 seems to be marginal (but I haven't extensively tested it yet). I have noticed however that cheetah produces more JPEGesque macroblocking

how are you measuring? you shouldn't just look at filesize — cheetah will likely produce less consistent quality, so it might be similar size but it could be a worse image...

Cjxl is better using 7

Cjxlng 3 could be enough

As cjxlng uses faster tools

Though don't know when patches get enabled in cjxlng

Does the equivalent of aom have patches

Or you have to do 4 speeds slower?

cjxlng and cjxl will just have the same effort settings, no difference there

``` ClassA_8bit_CAFE_2048x2560_8b_RGB.ppm.png Encoding kPixels Bytes BPP E MP/s D MP/s Max norm pnorm BPP*pnorm Bugs ------------------------------------------------------------------------------------------------------------ jxl:3 2048 905443 3.5368867 23.042 17.424 1.72588944 0.71976041 2.545711020841 0 jxl:4 2048 905461 3.5369570 23.745 19.899 1.72588944 0.71976041 2.545761628995 0 jxl:5 2048 777557 3.0373320 10.248 19.540 1.80541682 0.77316090 2.348346369324 0 jxl:6 2048 777298 3.0363203 9.616 19.619 1.88469470 0.77710328 2.359534479215 0 jxl:7 2048 777412 3.0367656 4.342 19.388 1.71217620 0.75627939 2.296643264439 0 jxl:8 2048 791701 3.0925820 0.537 21.409 1.50374460 0.69845216 2.160020614894 0 jxl:9 2048 797286 3.1143984 0.083 17.657 1.22554004 0.68475227 2.132591408450 0 Aggregate: 2048 817087 3.1917452 3.754 19.235 1.64053283 0.73197577 2.336280134829 0 ```

these type of results are typical

note that more effort does not imply less bytes

And patches at what Speed get enabled in cjxlng

it does improve max norm / pnorm per byte

Effort sorry

same as in cjxl

7?

Squurrrll is also a name you consider

So same speed as same effort

Wonder what's the difference in tools

7 for vardct mode, 5 for modular mode, iirc

there is no difference in the actual encode/decode code between cjxl/djxl and cjxl_ng/djxl_ng

it's just that cjxl/djxl call the code directly (so their binaries don't depend on libjxl, and effectively contain a kind of statically linked libjxl), while cjxl_ng/djxl_ng use the library api (so their binaries will be very small and dynamically depend on libjxl for all the heavy lifting)

a dummy question, I have a synthetic linear float image, values can sometimes be above 1.0. I'm trying to convert it to HDR JXL using JxlColorEncodingSetToLinearSRGB() and then overwriting transfer function with JxlTransferFunction::JXL_TRANSFER_FUNCTION_PQ will converted image be displayed correctly, or do I need to apply inverse PQ to the pixels before conversion? will values of 1.0 correspond to max HDR brightness? what will happen to pixels with values above 1.0?

if you set the tf to pq, the encoder will think you are passing it data in pq

if your data is linear, why not just encode it as linear?

are you interested in lossy or lossless compression?

what happens with values above 1.0 or below 0.0 is they will be outside the sRGB gamut and/or brighter than the nominal max brightness / darker than black

Chrome only shows JXL files in HDR if they have PQ/HLG curve, so I'm trying to convert linear files to lossles PQ JXLs

and this where I'm in doubt, should input be linear, should it have preapplied PQ or inverse PQ

can't visually determine how to do it correctly because images are synthetic

if I specify PQ and pass linear data, images look like near-blacks are elevated compared to passing linear data with linear gamma

I guess I need to apply PQ curve prior to conversion, just wanted to be sure

I'm not sure where to report this, so I commented on it in the exiv2 repo for now: https://github.com/Exiv2/exiv2/issues/2233#issuecomment-1155447694

Why is stuff like `size_t ReadHybridUintClustered(size_t ctx, BitReader* JXL_RESTRICT br) {` defined in a header file?

It's not inlined, so I don't see why it can't have a stub in a header file and then be defined in the corresponding `.cc` file

or rather, why is that entire class defined in a header file?

maybe this is my lack of C++ knowledge getting to me but shouldn't that stuff be defined below in implementation files?

it was inlined in the past

ah that makes sense

I am using my eyes mostly and the zoom button :) cheetah produces smaller files actually and slightly worse quality but if you just raise the quality setting until the quality matches then the filesize also matches AND you get way faster results! :)

But I haven't tested a large corpus of images

But so far it looks like the default effort is a waste of effort for the most part. Could be way faster and still have the same compression density.

It seems that `cjxl` sometimes can't detect exif in png files. Should I fill a bug?

what's the difference between these two PNG files?

one uses zTXt and one uses tEXt, but I don't see how cjxl doesn't recognize one but does the other

since the output length is the same for both

This is the same file, but the exif is written with different tools, `exiv2` and `exiftool`.

I don't think we handle compressed text chunks in cjxl, but I might be wrong

So, it should be a "feature request" then? 😄

These observations aren't guaranteed in general. Decreasing a distance target can help avoid results that are too low-quality, but low efforts can still sacrifice bytes in both coding efficiency and producing too high-quality results.

functions defined in c++ classes are automatically inline (under the language rules), otherwise they would almost certainly be illegal due to multiple definitions. whether they're actually inlined is implementation-specific

if i were to guess, i would say that gcc probably inlines these kinds of functions too aggressively. it could be a factor in lto being more effective on c++ than c: if a medium-sized function is used once per file in 3 files, it should probably be inlined; if it is used once per file in 100 files, it should probably not be inlined

also, the inline specifier is neither necessary nor sufficient for gcc to actually inline. putting the function in a header file simply means that gcc *can* inline if it wants to. otherwise, gcc cannot inline it without lto

example of gcc inlining even without `inline`: https://gcc.godbolt.org/z/3P5Yxzn8n

yep but compilers are *more likely* to inline things that are `inline`

and also, it was actually `__attribute__((always_inline))` or whatever the spelling is 😛

Hi, I'm not sure if this is the channel to ask this, but I have encoded an apng file with distance 0.0, but when I decoded to a list of png, there seemed to be a loss of information. I assume this is not expected, right?

how do you decode?

iirc djxl only decodes back to `apng`

I decoded just with: djxl img.jxl img.png

also how did you compare the frames?

It says, at djxl --help, that it just decodes to png, jpg and ppm/pfm

that should probably be updated

Using open-cv with python, it worked with other cases: cv2.imread(img.png, cv2.imread_unchanged)

can you post the sample?

The .jxl file right?

no, the original apng

The file is 70MiB, discord doesn't allow this size, is there anywhere else I can send?

try running this: ``` ffmpeg -i original.png -c rawvideo -f hash - 2>/dev/null cjxl -d 0 original.png interim.jxl djxl interim.jxl test.png ffmpeg -i test.png -c rawvideo -f hash - 2>/dev/null ```

and let me know what output you get

Switching original.png by my apng?

yes

ffmpeg -i IMG_MONOCHROME2.apng -c rawvideo -f hash - 2>/dev/null cjxl IMG_MONOCHROME2.apng interim.jxl djxl interim.jxl IMG_MONOCHROME2.png ffmpeg -i test.png -c rawvideo -f hash - 2>/dev/null SHA256=543bd0f74847bf10d83c116cb058639864212568fc81ac02fdbcec41408ab448 JPEG XL encoder v0.6.1 a205468 [AVX2,SSE4,Scalar] Read 1890x2457 image, 64.4 MP/s Encoding [VarDCT, d1.000, squirrel], 4 threads. Compressed to 249739 bytes (0.007 bpp/frame). 1890 x 2457, 0.06 MP/s [0.06, 0.06], 1 reps, 4 threads. JPEG XL decoder v0.6.1 a205468 [AVX2,SSE4,Scalar] Read 249739 compressed bytes. Decoded to pixels. 1890 x 2457, 0.94 MP/s [0.94, 0.94], 1 reps, 4 threads. Allocations: 36637 (max bytes in use: 4.561156E+09)

Change that last ffmpeg command, it has the wrong input filename

ffmpeg -i IMG_MONOCHROME2.apng -c rawvideo -f hash - 2>/dev/null cjxl IMG_MONOCHROME2.apng interim.jxl djxl interim.jxl IMG_MONOCHROME2.png ffmpeg -i IMG_MONOCHROME2.png -c rawvideo -f hash - 2>/dev/null SHA256=543bd0f74847bf10d83c116cb058639864212568fc81ac02fdbcec41408ab448 JPEG XL encoder v0.6.1 a205468 [AVX2,SSE4,Scalar] Read 1890x2457 image, 54.0 MP/s Encoding [VarDCT, d1.000, squirrel], 4 threads. Compressed to 249739 bytes (0.007 bpp/frame). 1890 x 2457, 0.05 MP/s [0.05, 0.05], 1 reps, 4 threads. JPEG XL decoder v0.6.1 a205468 [AVX2,SSE4,Scalar] Read 249739 compressed bytes. Decoded to pixels. 1890 x 2457, 0.90 MP/s [0.90, 0.90], 1 reps, 4 threads. Allocations: 36637 (max bytes in use: 4.561156E+09)

after playing with it a little, i think gcc (12) treats functions defined in classes as if they were declared/defined with inline, both for language and for optimization purposes

why are you getting no output from that last ffmpeg command? Also, try cjxl -d 0 -e 2 (when not doing lossless, you cannot expect the hash to match)

interestingly, clang *doesn't*: `class foo { public: static int f() { ... } }` and `class foo { public: static inline int f() { ... } }` are sometimes optimized differently

You're right, I think it's because each frame is written separately in its own file, like so: img-00.png, ...

it shouldn't be

> Encoding [VarDCT, d1.000, squirrel], 4 threads. you didn't set `-d 0`

Oh

This is a really old cjxl/djxl

yea, I was about to mention

The one that doesn't know about apng yet

well then, that would do it <:kek:857018203640561677>

Also, I specified the bits per sample of 10, thus , since my original image is 10 bits, but png only supports either 8 or 16, so that could also be a problem

yeah, I was somewhat more familiar with clang (well, llvm, I think the inliner is there) so that's what I talked about 😛

Yes, I was using the 0.6.1 version commit, specifically

Is it better to use a recent commit?

Yes, best to use the latest git version

Haha well yes it has been too long since we made a release

Will you jump to the 1.0.0 in the next release?

that's the plan

Out of curiosity, with the current release it's not possible to for example re-encode a lossless jxl to a lower quality copy because cjxl can't read in jxl files? (without going jxl -> png -> jxl)

Sure this is possible, just have to use imagemagick or ffmpeg or gimp or vips or ... 😅

or gwenview

I think there is a bug, decoding a .jxl multi-frame image to .apng (original is also .apng)

Can I send the output?

sure

if you've updated to git master libjxl and there's still that issue, feel free to provide a sample

first try the same thing I suggested earlier

this, I mean https://discord.com/channels/794206087879852103/804324493420920833/986673916305088512

It didn't finish decoding

Or writing

wdym it didn't finish? post output

(Sorry, wasn't able to post earlier) % djxl img.jxl img.apng JPEG XL decoder v0.7.0 89215e7 [AVX3,AVX2,SSE4,SSSE3,Emu128] Read 47746120 compressed bytes. Decoded to pixels. ./lib/jxl/image_bundle.cc:34: JXL_CHECK: metadata_->color_encoding.IsGray() == c_current.IsGray() [1] 4053 illegal hardware instruction (core dumped) djxl img.jxl img.apng

what's the source of your djxl binary? was it compiled with mingw-w64?

if I had to guess, it's most likely compiled with MinGW, which incorrectly emits aligned instructions for avx2 when there's no alignment guarantee

you have to compile highway with `-Wa,-muse-unaligned-vector-move`

that's done with clang, so I believe so

Looks like a bug, assert is failing. Is the image grayscale?

Pretty sure it is, yes, but I can double check

I compiled it with cmake, like the main readme shows

Does it work if you decode to other formats like jpeg or pgm?

No, same error message appears

I can send the file through online file sharing if you want to test

The os I'm working on is Linux

at least you can trust it as much as you trust me 😛

can you upload a sample?

It's too big for discord, where do I upload?

Thank you. Here https://fileupload.ggc-project.de/download/62b3969e414f2915/#uA1nlC4QulQgL90dHLFRRQ

<@388756576824983553> there's also `https://0x0.st`

which is very helpful

I can reproduce the assert failure

For sharing?

yea

`curl -F'@file.png' https://0x0.st`

uploads it from the CLI, and it returns a link to the file

file size is <256 MB, it disallows some stuff like `.exe` and `.jar` but otherwise it works

it only lasts temporarily, between 30 days and a year depending on file size

and if you upload the same file again it returns the same link since it keeps track of the hash

Nice, thanks

also, I reproduced your bug

I'll open a bug report

https://github.com/libjxl/libjxl/issues/1517

it is grayscale, and it's a bug in the APNG encoder, since if you discard the decoded pixels it exits without issue

I reproduced it on a very small sample and opened the above issue

FYI, SIGILL is just how our asserts die

why not SIGABRT?

uh, looking at the code again, I get the impression that it should be SIGTRAP https://github.com/libjxl/libjxl/blob/c4262565e8de611f593bf2c6f4a24aeb55d38afb/lib/jxl/base/status.h#L127-L141

Built-in Function: void __builtin_trap (void) This function causes the program to exit abnormally. GCC implements this function by using a target-dependent mechanism (such as intentionally executing an illegal instruction) or by calling abort. The mechanism used may vary from release to release so you should not rely on any particular implementation.

well

maybe we should replace it with a call to `abort()`

or maybe `raise(SIGTRAP)`

(or, yeah, `abort` will do)

isn't raise Unix-like specific?

probably

Trying to build libjxl statically is a bit more difficult than I though: `ld.lld: error: attempted static link of dynamic object libz.so.1.2.11 clang-13: error: linker command failed with exit code 1 (use -v to see invocation)` It's the only thing that fails.

I guess you need a libz.a

you can always add `-Wl,-Bdynamic -lz -Wl,-Bstatic`

which will dynamically link libz.so

but you can't dynamically link into a static binary anyways

Recently, my AVIF Qt plugin got approximately 10% faster by avoiding unnecessary conversion between pixel formats. Beside RGBA, I get pixels as BGRA (on little endian) or ARGB (on big endian) from libavif. JPEG XL Qt plugin could be faster too when libjxl API supports more pixel formats. I think something like that is planned for 1.0, isn't it?

well you need to link *some* things dynamically

like libc.so

I figured a 'static binary' on linux usually means except for some basics like libc and libz

Yea, and they're also planning on supporting planar

I'm not sure if it's a 1.0 milestone though

Yeah, that's about it. It's only zlib that is being a bit annoying, it'd be nice if it could be built statically since it is downloaded anyway.

Feel free to make a header-only api update pull request for this. It should be easy enough to implement any output pixel format, so I think once we know which ones are needed and what the api should be, it shouldn't take too much effort to add things there.

it is possible, and sometimes valuable, to make binaries that are completely static

Would it be possible for example to extend JxlPixelFormat to contain new char channel_order[8]; member? (decoding API breaking change) It would allow generic description of various formats. You can mention that only certain combinations are supported now, but later you can extend the list. It would be used this way: pixel_format.num_channels = 5; pixel_format.channel_order[0] = 'C'; pixel_format.channel_order[1] = 'M'; pixel_format.channel_order[2] = 'Y'; pixel_format.channel_order[3] = 'K'; pixel_format.channel_order[4] = 'A';

is this interleaved only

I don't know what preferences have those who use planar. Maybe they would like to have multiple output buffers.

i think the idea/hope is to have an API which can be generalized to planar output

see also https://github.com/libjxl/libjxl/issues/993

Maybe planar is better added by extending the function to set buffers for extrachannels to also work for individual color channels

That would be convenient/comfortable to use. But will it be faster to get data that way than getting pixels as RGBA first and converting on application's side afterwards?

In jxl the internal data is planar RGB for lossless/non-xyb, planar XYB for lossy/xyb

It is converted to interleaved only at the end, when doing color conversion and conversion to integers

CMS color conversion takes an interleaved row and produces an interleaved row, but that's done with floats so we can still choose in the conversion to ints whether to keep it interleaved or go back to planar

Basically we could just have 3 pointers, 3 row strides, and 3 pixel strides for the final step from interleaved floats to output pixelformats

So in planar each pointer would point to a different buffer and the pixel strides would be 1, while in interleaved they would point to different offsets in the same buffer and the pixel stride would be 3 (for rgb) or 4 (for rgba).

<@179701849576833024> <@604964375924834314> does that make sense?

at first glance, it seems to

In some cases, there is a padding at the end of each row (except the last), so the new row starts at aligned address.

oh, right

but that's what the row stride is for, no?

yes, actually I like row stride term more than alignment.

my current idea for this is to add `enum JxlColorPlane` with values like `ALL, RED, GREEN, BLUE, LUMA, CB, CR, CYAN, MAGENTA, YELLOW, EXTRA`

and you'd keep the current SetDecoderOutputBuffer for backward API/ABI compat

but you'd add an extra SetDecoderOutputBufferPlane, which takes an argument of enum JxlColorPlane

if you pass *ALL* you'll receieve packed/interleaved output

which is the current default

if you pass RED, GREEN, BLUE, etc. you'll receieve only those planes

the decoder will reject LUMA, CB, and CR unless this is a reconstructed JPEG, or this is grayscale, in which case it will allow LUMA

You might also simply not want to allow YCbCr

Does this make sense to anyone else? Should I open an issue for this proposal?

sounds plausible to me, but i know very little about colors. i assume `ALL=0`

i'm not sure how important it is to disallow nonsensical combinations, such as requesting RED and CB and YELLOW. maybe such cases could also be supported, but just run slower. it could theoretically be useful to get, say, LUMA, RED, GREEN, BLUE. in either case, maybe it would be useful to have an API to query which combinations are supported.

also, what would be the behavior if you call both SetDecoderOutputBuffer and SetDecoderOutputBufferPlane? do you get both ALL and whatever other colors?

actually, your proposal does not seem to cover the order of interleaved channels. how would that be integrated? would it use JxlColorPlane?

RGB and CMY are just the same thing in jxl, the only difference is in the icc profile

it doesn't, it would be for planar only

I see, so maybe you wouldn't have CMY, just ALL, RED, GREEN, BLUE, EXTRA, and optionally Y, Cb, or Cr

maybe Y you'd always have for the purpose of grayscale

SetDecoderOutputBuffer would become SetDecoderOutputBufferPlane(ALL) externally

I figure it would error out if you tried to call SetDecoderOutputBufferPlane in invalid ways

that's one option, another option is if you set a plane incompatible with the previous options, then nothing happens

I'm getting what I think is a bug in libjxl, but I'm not sure

if I attempt to encode a 3840x2160 image that's 16-bit RGBA (64-bit total), it fails to set the basic info unless I set the codestream level to 10 first

but I'm not sure how my client is supposed to be able to know that it's supposed to do this before I set the basic info.

`JxlEncoderSetCodestreamLevel` doesn't support `-1` yet

and setting the basic info includes a level verification check

which will fail

I feel like the only way to work around this is to set the level to 10 no matter what

in this case, level10 is required because `alpha_bits > 12` but I don't want to duplicate all of the same checks that libjxl already has

basically, I'm in a catch-22

I call JxlEncoderGetRequiredCodestreamLevel requires basic info to be set to provide any meaningful info

JxlEncoderSetBasicInfo includes a level check, which will cause it to fail if level10 is required and level5 is the current setting

it sounds like what I actually have to do is set the level to 10, then populate JxlEncoderSetBasicInfo, and then *after that* set the level to what it's actually supposed to be

I think we just need to implement SetCodestreamLevel -1

and let that be the default

that way you can populate the basic info without setting it first, and then check afterward

https://github.com/libjxl/libjxl/pull/1538

An issue will make sure that it gets discussed with all the people who understand that domain -- these discord discussions can be missed by some of our experts.

https://github.com/libjxl/libjxl/issues/1542

Hello, I am currently working on my bachelor thesis where I am comparing image compression codecs on a system based on the QNX operating system. Unfortunately the proprietary compiler for QNX only supports C++99 Do you think it would be possible to somehow compile libjxl using C++99 for QNX?

this is a bit of a crazy idea, but... have you tried compiling libjxl to webassembly and webassembly to c with wasm2c?

give a developer a browser, everything looks like wasm?

from https://stackoverflow.com/questions/5050349/is-there-a-way-to-compile-c-to-c-code, https://github.com/JuliaComputingOSS/llvm-cbe seems to be maintained

well, sort of maintained

oh I didn't know of llvm-cbe

i'm pretty sure both the original c++ implementation and several after that were implemented using translation to C

yes

it *has* been more than 20 years though 😉

libjxl doesn't use exceptions, so theoretically it should be amenable to C translation. the trouble is that nobody cares about such things anymore because everyone (sensible) uses gcc or llvm for new platforms anyways

https://stackoverflow.com/questions/34168433/am-i-able-to-use-c11-in-qnx says that c++11 was available on qnx in 2015

https://www.qnx.com/developers/docs/7.0.0/#com.qnx.doc.qnxsdp.migration/topic/cpp_apps.html says "We support only C++11 and C++14 with libc++. Using older or new standards (C++98, C++03, C++1z (17)) may result in compilation errors. If these are absolutely required, it might be possible to do with some toolchain changes using libstdc++."

yeah thanks for the help but the system I'm working on only has a really outdated license of QNX that really only supports c++99

I have looked into it more and it seems I will have to port parts of the system to linux to get it to work

I also want to add avif and heif to the comparison as well so I guess I would have run into the same problems anyway with these libraries

fwiw, i don't think there exists a c++99. there is c99 and c++98

and c89 which is also basically c90

does cjxl save encoder settings in the file

has someone already asked this

you can _guess_ particular settings, but in general no such metadata exists to my knowledge

i would like to have the encoder version and parameters

might be even more useful if/when lossless jxl recompression is added

i think jxl developers tried very hard to minimize header size, even (in hindsight) at significant expense to minimal parsers; adding a bunch of metadata by default would defeat the purpose imo

it might be okay to have some off-by-default option to store encoder settings; that might also satisfy some people who want to know whether some image was compressed using the lossless setting or not

otoh this is arguably dubious info to begin with, as explained in the issue. if you want to know if an image could be losslessly recompressed to be smaller, the proper way to do it is to just try to do it and see if it gets smaller, not to make some guess based on the previous parameters

agreed, many compression formats did have some sort of encoder metadata in one way or two, hoping to be useful, and I haven't seen any tool that took advantage of them

hello

is it possible to disable the build of jxl_emcc.exe ?

https://github.com/libjxl/libjxl/commit/3171add7cf986934d15706e586720fdeb9de5f90

<@208917283693789185> and with cmake ?

After that commit the `jxl_emcc` executable is no longer build (unless you target Emscripten and build without `-DJPEGXL_ENABLE_TOOLS=0`). Sorry for not being clear.

thank you

We could invent an isobmf box for encoder version/settings, or invent an exif/xmp field for it...

No guarantees that it will exist or be correct, of course

you could also just have an extension in the codestream too

if you wanted

That's also an option, but I guess a new box would be easier to parse

jxlinfo can tell if JXL is possibly lossless by checking if the original profile is used. What exactly is original profile? Does it mean that XYB color transformation was not used?

that's exactly correct

"uses original profile" means XYB is not used

"possibly lossless" in this case means it's encoded using settings that can be used in lossless mode

most importantly, modular mode is used without XYB

although it's not signalled whether the encoder modified any data when it did the encode, which is why it's *possibly* lossless

if it's done with VarDCT but the original profile is used, that's still guarnateed lossy since VarDCT cannot be used for bitexact lossless mode

has there been any decoder improvements since 0.6.1?

i guess i can keep waiting until 1.0...

pretty sure it's much faster now than it was

the API is a lot better, I can say that for certain

"A new release needs to be maintained," says who? What? No it doesn't. I don't think anyone expects 0.6 to be maintained after 0.7 or 1.0 comes out. Seems like a terrible excuse. Seems like a very bad idea to wait so long without making a release.

All that people want right now is a good reliable decoder library that satisfies all of the needs of common web browsers (probably also things like progressive decoding and lqip)

The decoder is more important than the encoder as far as adoption goes right now, at least at this point.