JPEG XL

Heh, the last commit strongly suggests the private repo is in GitHub

Thanks, did a quick search and that's: https://discord.com/channels/794206087879852103/822105409312653333/844555374999765022

I don't want to promise anything, but I think somewhere around next week we'll have a github repo 🙂

Will the sync to GitLab stop then?

nah, they'll be mirrored AFAIU

Cool

and I think we'll keep mirroring it to the private gitlab 'til we figure out how to run the full CI on github

probably github, but I think we'll either migrate the bugs or just disallow opening new ones and close the old ones as they get fixed

BTW, why move to github when started in gitlab? 👀

because of stuff that I don't fully understand but it makes it easier to accept external contributions

it started in gitlab because at the time github didn't have free private repos and ISO needed that

I ran into a bug in the Chromium development tools that was caused by branch name renaming -_-

made me lose one hour or so

like what is the thing more difficult to get right?

why is the development a bit slow

other than the decoder

and the patches

talking about the lossy encoder

what is the problems you run up with?

You don't have to convince me

Just a lot of complicated logic

It's not hard per se, just complex to get right

It looks like a question but I have no idea how to parse it

When you go higher than oddly specific q 63.52 -> you mean set a higher quality?

You'll get jpg looking artificial -> you mean you somehow get more artifacts at a higher quality setting than at a lower one?

At the moment at least there is no difference between jpg or png input if you pick a specific lossy quality: it will start from the pixels and do the same thing

Only lossless jpg is something else

I suppose we could do something at the dct coefficients level too to do lossy jpg recompression, but we currently don't and there are no plans to implement that anytime soon

New heuristics is an experimental thing that is for now just for testing

We aim to keep the meaning of d X or q X the same when we do optimizations

yes i know; usually you make the table

to choose an heuristic

the one with those colored squares

At faster speeds, more things are heuristics based so there is less guarantee that the quality target is actually reached

At slower speeds, more things are search based so there is more guarantee that the target is reached

BUT the target is only an objective metric, a good one but still, no objective metric is perfect.

yes but digiting only q 65 s 9

is simpler to do

than a whole long command

So reaching the objective metric target is not a guarantee that subjectively it will be perfect

ok

For now, default speed, default options is the best tested and most reliable way to encode

as long it looks good with png

Just cjxl -q 80 or whatever target you like

-Q 94.3 --dots=0 --intensity_target=194 --use_new_heuristics

I want to add something faster than -s falcon, what should I call it?

there are no animals faster than a falcon

-s airplane ?

-s helicopter?

-s peregrine? (that could get confusing though)

I was thinking airplane for -e 2 and maybe later rocket for -e 1 or something

<@!532010383041363969> opinions please 🙂

I think that the choice of non-animal names would create more confusion

`-s tesla`

Then we would have to rename current options though

I said helicopter to open up airplane and rocket for further evolution

photon-in-vacuum

Alcubierre-drive

Or shift names so the hawk is still the fastest

Shifting names might be confusing, at least if everything changes meaning

Maybe if it's only falcon that shifts and there's an animal between falcon and cheetah it could be ok

Still, I prefer to keep falcon to tortoise as they are, and have new names for new things

Yep, but if making this changes, it is better to do it earlier while JXL is not so much used and also I am not sure that the names are very often chosen for scripts and other things (so there is little chance that it will break something)

-s unicorn? 😛

Are unicorns fast?

Pegasus?

Shadowfax? That's Gandalf's horse, should be quite fast

I guess it depends on the mythology/fantasy world you subscribe to 😛

like with harry potter you can go -s phoenix since they teleport

We also need names for the other fast speeds: 0(?), 1, 2 (even if they do not exist, so as not to have problems later)

phoenix can be 0, unicorn 1 or 2, and I don't know about the middle 😛

Pegasus is also ok, the only problem is knowing which of these creatures is faster Shadowfax is complicated, not everyone and everywhere remembers this name (in other languages it may be in translation)

pegaso >> unicorn

damn 1 moment early

https://github.com/libjxl/libjxl keep an eye here! 😛

For VarDCT, s 3 is already about as fast as it gets, so I will propose to shift that to s 2 and make s 3 something in between the current s 3 and s 4

For Modular, a faster s 2 is possible

If you just want to use jxl as a temporary local storage format (a mildly compressed ppm with embedded icc profile, basically) then s 2 will be nice

(or maybe s 1)

How come you people uploaded a new build to Github?

What advantages does doing stuff on Github give over Gitlab?

tldr: CLA

also it gives us an excuse to not put it under wg1/, but that's minor 😛

and I think it also has some "more internally approved" ways to publish binaries for releases, but don't quote me on that

-s 1/2 supersonic, -s 0/1 light? <:Thonk:805904896879493180> (not animals, but easier to remember) At least it is something not technological, for example the speed of vehicles, trains, planes, rockets became faster over time and the speed of mythological creatures is difficult to understand

Just Sonic? He is fast enough.

-s hedgehog

sanic

Yea

The names of superheroes or game characters is cool, but then there can be problems, since they usually have copyrights and stuff

But why not "sonic" as a reference to audio waves? Technically they're faster than animals... 😉

Because there is another Sonic that people usually imagine and there will be confusion with him (also the game character Sonic can be faster than light)

What's the utility of the animal names anyway? I don't find them intuitive myself.

To give an intuitive sense of the speed of their numerical equivalents

numerical is fine, animals are a bit more fun

Sonic is both sound and animal. Also can't be copyrighted

Oh, I've got an idea for the fastest speed name

Supersonic <:megapog:816773962884972565>

Sonic is copyrighted, sonic isn't

hypersonic

Orbital veloticty

Hypersonic is mach 5+. That would be reserved for `-s 1`

Lightspeed

Ludicrous speed

Alcubierre-drive (FTL)

Space stone

Police telephone box

😛 nice reference

-s 2 (supersonic), -s 1 (hypersonic), -s 0 (light)

-s 12 (rock), 12 years of encoding and progress will move one percent

And is a progression in a while true loop

it'll come closer to 100% but will quite never hit it

-s 2 (airplane), -s 1 (rocket), -s 0 (light)?

I've seen Windows progress bars that do that

Zeno paradox progress bars

Windows 98 installation

Was a geometric succession 1/2 1/4 1/8

If there were animal names already, I would prefer not technological things, but rather characteristics/descriptions of speed or something natural like a storm, hurricane (but it doesn't fit, just an example) https://discord.com/channels/794206087879852103/804324493420920833/846781980682354760 Like the speed of an airplane may be different depending on the model and design and some animals may be faster, rockets can also be for example homemade and the speed of all of this is not obvious

-s thunder, -s lightning?

https://c.tenor.com/EVJo-FdHf9AAAAAM/zeus-thunder.gif

Yep, something like that

https://c.tenor.com/evwQTrVO080AAAAM/queen-bohemian-rhapsody.gif

There is also variation in animals. I'd think if the animal ordering is obvious, then 'rocket is faster than falcon' should be obvious under similar reasoning.

Rockets can be like this

Also, not all airplanes can be faster than a falcon

Kittens can be like this.

🥳 Is there a plan to migrate issues too?

Yes, but in some categories of animals the maximum speed is relatively unchanged if they are healthy, unlike technical things, which are always progressing, once cars were slower than horses, trains could not be supersonic, etc.

Maybe in the future we have artificially-enhanced kittens that run faster than falcons fly.

A typical airplane can go faster than a falcon now already, and it is unlikely to get slower in the future

But maybe falcons will evolve to compete with airplanes?

https://tenor.com/view/sam-wilson-falcon-gif-10597370

By growing cabins to facilitate human transportation?

A sleeping falcon is also much slower than a tortoise

https://tenor.com/view/mcconnell-turtle-sprint-gif-7799343

That's some proper motivational thought... Have you ever thought about entering philosophy?

We can go off of connotation and not denotation

Most people associate these animals with speeds

so is the new github repo getting commits as they are written, instead of syncing occasionally?

yup

Why do people want the full commit history?

I don't.

I think for `git bisect` / debugging purposes.

BTW, is it necessary to add myself to `AUTHORS`, even for small changes?

I don't think it's *necessary*, no

Ah OK, fwiw, there's a welcome bot for that. See e.g. <https://github.com/emscripten-core/emscripten/pull/14224#issuecomment-843783370>.

I don't know though 😛 I'll ask today during the team meeting

as far as I understand, AUTHORS is for the purpose of authorship in the copyright sense. Consider it like a book. If you write a chapter, you're an author. If you fix a typo, you're not. In between those extremes there is a spectrum and basically you can choose yourself as far as I'm concerned.

<:Stonks:806137886726553651>

We just want an up to date repo that's synced in real time rather than having to wait a week or two for batches of commits

You got it since yesterday

I think <@794205442175402004> was asking whether people wanted the full commit *history*; not just the new commits now.

I don't think the full commit history is needed

If someone needs all the commits or whatever they contained for something they can ask

So if Github has become main, might as well change it here?

yes please!

__**Ye Ultimate Updated (Once Again) One-Liner for Compiling JPEG XL in Bash:**__ `mv libjxl/build/tools/ ~; rm -rf libjxl; git clone https://github.com/libjxl/libjxl.git --recursive; cd libjxl; git revert -n 96e1f432595669ab9912934ec4d8b4d24d3f8eab; mkdir build; cd build; cmake -DCMAKE_BUILD_TYPE=Release -DJPEGXL_ENABLE_DEVTOOLS=ON -DBUILD_TESTING=OFF ..; cmake --build . -- -j$(nproc); mv -n ~/tools/* tools/; rm -r ~/tools/`

`git revert -n 96e1f432` 😆

at some point you'll start getting conflicts 😛

(you know you can do parallel recursive clone?)

Joke's on you, I've just updated it 😉

I mean at some point we'll change something in c/djxl that will make the revert not work

Better ASCII art? 🙂

One-hour color fullscreen ASCI animation before start of each encoding

Weak.

Here's mine.

```export CFLAGS="-O3 -march=native" CXXFLAGS="-O3 -march=native" export CC=/usr/bin/clang && export CXX=/usr/bin/clang++ git clone https://gitlab.com/wg1/jpeg-xl.git --recursive cd jpeg-xl && mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-O3 -march=native" -DCMAKE_C_FLAGS="-O3 -march=native" -DJPEGXL_ENABLE_PLUGINS=ON -DJPEGXL_ENABLE_GIMP_SAVING=1 -DJPEGXL_ENABLE_DEVTOOLS=ON -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_TESTING=OFF -DJPEGXL_WARNINGS_AS_ERRORS=OFF -DJPEGXL_FORCE_SYSTEM_BROTLI=1 -DJPEGXL_ENABLE_SJPEG=OFF .. cmake --build . -- -j 16 sudo make install```

I doubt you get much out of either -O3 or -march=native 😛

It mostly helps in modular 😛

oh, it does?

that's very useful to know

how much?

also, decoding or encoding?

Yeah, the benefit is not negligible. It's been a while since I've tested encoding, but I guess I could rebuilt the latest without compiler opts and profile the difference.

does it help encoding speed mostly?

'cause in that case I don't care *as much* 😛

Btw, I can't compile JXL with Clang and LTO in MinGW for some reason (and maybe PGO would help as well) If encoding takes weeks, even a small speed gain would be helpful

surprising, I remember trying LTO and succeeding but not on win

but yeah, an LTO + PGO build for the modular encoder is likely a very good idea

just make sure to profile on a mix of -I 0 -s 9 -E 3 graphics and -I 1 -s 9 -E 3 non-graphics files 🙂

eh, not bad

decoding?

`0.80 MP/s [0.72, 0.72]` wait, what?

Oops, wrong one.

You know, perhaps I should make sure I copy paste the right ones <:Thonk:805904896879493180>

Wait, a minute. <:Thonk:805904896879493180> `time /home/bluezakm/jpeg-xl_no_opt/build/tools/cjxl /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9.png /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s7_noopt.png -d 0.0 -s 7` `time /home/bluezakm/jpeg-xl_no_opt/build/tools/cjxl /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9.png /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s7_noopt.png -d 0.0 -s 7` Something seems to have messed up my numbers, because I was getting the same times. That's why.

isn't that the same build? 😛

Yep. It was just the fact that I decided to do a `sudo dnf upgrade` in the background that resulted in the different numbers.

ahah ok

Wrong numbers sorry 😛 That doesn't matter: looks like it is margin of error anyway :/ O3+march znver2 ```time /home/bluezakm/jpeg-xl_opt/build/tools/cjxl /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9.png /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9_opt.jxl -d 0.0 -s 7 J P E G \/ | /\ |_ e n c o d e r [v0.3.7 | SIMD supported: AVX2] Read 6668x3840 image, 67.2 MP/s Encoding [Modular, lossless, squirrel], 8 threads. Compressed to 10028832 bytes (3.133 bpp). 6668 x 3840, 0.74 MP/s [0.74, 0.74], 1 reps, 8 threads. real 0m35.035s user 0m44.726s sys 0m0.697s``` ```time /home/bluezakm/jpeg-xl_no_opt/build/tools/cjxl /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9.png /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9_noopt.jxl -d 0.0 -s 7 J P E G \/ | /\ |_ e n c o d e r [v0.3.7 | SIMD supported: AVX2,SSE4,Scalar] Read 6668x3840 image, 68.0 MP/s Encoding [Modular, lossless, squirrel], 8 threads. Compressed to 10028832 bytes (3.133 bpp). 6668 x 3840, 0.72 MP/s [0.72, 0.72], 1 reps, 8 threads. real 0m35.869s user 0m45.426s sys 0m0.747s```

you can do multiple reps (--num_reps=5) to get more consistency (and perhaps also --num_threads=0)

Eh, I don't think it would be really necessary looking at these numbers. I'm more interesting in decoding anyway, like you 😛

I think we did enough manual optimization that -O3 barely helps in the hot parts for decoding 😄

Small difference for decoding though: ```time /home/bluezakm/jpeg-xl_opt/build/tools/djxl /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9_opt.jxl --num_reps=5 Read 10028832 compressed bytes [v0.3.7 | SIMD supported: AVX2] No output file specified. Decoding will be performed, but the result will be discarded. Decoded to pixels. 6668 x 3840, geomean: 35.92 MP/s [35.57, 35.97], 5 reps, 8 threads. Allocations: 8339 (max bytes in use: 9.718959E+08) real 0m3.577s user 0m26.298s sys 0m1.697s``` ```time /home/bluezakm/jpeg-xl_no_opt/build/tools/djxl /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9_noopt.jxl --num_reps=5 Read 10028832 compressed bytes [v0.3.7 | SIMD supported: AVX2,SSE4,Scalar] No output file specified. Decoding will be performed, but the result will be discarded. Decoded to pixels. 6668 x 3840, geomean: 34.44 MP/s [34.19, 34.52], 5 reps, 8 threads. Allocations: 8339 (max bytes in use: 9.718937E+08) real 0m3.729s user 0m27.480s sys 0m1.678s```

can you do it with 0 threads?

I wonder what the compiler is doing, that 0.3% might be something worth doing manually...

O3+march```ime /home/bluezakm/jpeg-xl_opt/build/tools/djxl /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9_opt.jxl --num_reps=5 --num_threads=0 Read 10028832 compressed bytes [v0.3.7 | SIMD supported: AVX2] No output file specified. Decoding will be performed, but the result will be discarded. Decoded to pixels. 6668 x 3840, geomean: 5.24 MP/s [5.22, 5.24], 5 reps, 0 threads. Allocations: 8129 (max bytes in use: 9.464126E+08) real 0m24.465s user 0m24.170s sys 0m0.276s```No optimizations(release profile, which is O3)```time /home/bluezakm/jpeg-xl_no_opt/build/tools/djxl /home/bluezakm/Pictures/SNCE8P_2021-03-13_21-45-41_s9_noopt.jxl --num_reps=5 --num_threads=0 Read 10028832 compressed bytes [v0.3.7 | SIMD supported: AVX2,SSE4,Scalar] No output file specified. Decoding will be performed, but the result will be discarded. Decoded to pixels. 6668 x 3840, geomean: 5.02 MP/s [4.99, 5.02], 5 reps, 0 threads. Allocations: 8129 (max bytes in use: 9.464126E+08) real 0m25.554s user 0m25.263s sys 0m0.272s``` 4,38% is certainly not negligible...

very interesting

I'll try to see a profile of it at some point

O3+march haswell ```Read 12792752 compressed bytes [v0.3.7 | SIMD supported: AVX2] No output file specified. Decoding will be performed, but the result will be discarded. Decoded to pixels. 3359 x 2267, geomean: 2.97 MP/s [2.87, 2.97], 5 reps, 0 threads. Allocations: 1104 (max bytes in use: 3.081844E+08)``` O3 ```Read 12792752 compressed bytes [v0.3.7 | SIMD supported: AVX2,SSE4,Scalar] No output file specified. Decoding will be performed, but the result will be discarded. Decoded to pixels. 3359 x 2267, geomean: 2.89 MP/s [2.81, 2.90], 5 reps, 0 threads. Allocations: 1104 (max bytes in use: 3.081844E+08)```

No problem. Here are my perf profiles in the meantime

I'm not sure if those are useful without a binary

ah, I thought they were the perf.data files

mh, ok, that's enough info anyway

it likely optimizes the weighed predictor

which is not really a surprise tbh

there's a few things I already know we could do better there, just never has been at the top of the "what should I do next" list

> ISN'T A ONELINER IN THE FIRST PLACE > no `cjxl` ASCII art > still points to GitLab instead of more recent GitHub > still uses `jpeg-xl` directory instead of `libjxl`

oss-fuzz has found a -ve shift in libjxl (undefined behaviour), shall I open a bug report? (it's not a very interesting bug, but should probably be fixed)

... issue here https://github.com/libjxl/libjxl/issues/29

I wonder how it can happen at all 😄

hehe ... the wonder of C (we all love it really)

I'd have thought that it was mathematically impossible for that number to be negative, really xD

oh well, <@!794205442175402004> will figure it out

Must be somehow uninitialized, I suppose.

Found the bug

I can confirm that PR #35 resolves issue #29. Tested with both unminimized (which gave `-5`) and minimized (i.e. the one in that issue) testcases from OSS-Fuzz.

ah the unminimized probably just did it on a larger image, where the hshift would go to 5 after squeeze, then became 0 due to buggy inv palette, and then became 0-5 = -5 after inv squeeze (instead of the 5-5 = 0 it should have been)...

another oss-fuzz report, sorry https://github.com/libjxl/libjxl/issues/37

https://i1.wp.com/gifrific.com/wp-content/uploads/2015/03/Liar-Liar-Drinking-Water-Oh-Come-On-in-Courtroom.gif

<@!310374889540550660> OSS-Fuzz issue 34186 and 34269 can be reproduced using their Docker images. It's a floating-point division by zero, which is a no-op unless you actively checking them.

Something is wrong with my discord app, it shows incorrect previews...

github repo doesn't have tags pushed

you mean the old versions?

please file an issue 🙂

done.

thanks!

The fastest animal would be --speed tartigrade-shot-from-gun

-s unicorn is great -- there is a utf symbol for it U+1F984

if we add airplane and rocket, it will reduce the cuteness of the animal categories

human constructs are still rather inferior to what biology has created -- it is a common mistake to celebrate them over animals 😄

we should ideally add 🐞🦀🦆 and other cute animals and shift the others around

we could add swift between cheetah and eagle, but swift is such a boring and confusing name in computer environments

For now I have called the two new speeds thunder and lightning

Feel free to do a renaming 😁

😄

I'm fine with those

https://c.tenor.com/evwQTrVO080AAAAM/queen-bohemian-rhapsody.gif

Now just have to make lightning actually do something useful 😅

Maybe just YCoCg + Gradient predictor + RLE? That might still beat fast png encoding...

there is something that is more frightening than thunder and lighting, it is 'huutajat'

https://www.youtube.com/watch?v=iv4IoQXWSXQ

it is a choir from the town where I was born

it cannot unfortunately be recorded, shouting doesn't work through a video as well as it works in person

https://www.youtube.com/watch?v=HtpQI07MlP4

Reminds me of https://en.m.wikipedia.org/wiki/Geographical_Fugue

I once participated in a choir that did that one, it's quite fun

Der Popocatepetl liegt nicht in Kanada, sondern in Mexiko, Mexiko, Mexiko. Kanada, Malaga, Rimini, Brindisi, Kanada, Malaga, Rimini, Brindisi. Ja! Athen, Athen, Athen, Athen, Nagasaki, Yokohama, Nagasaki, Yokohama,

so I now have an (apparently working) JPEG to JXL nginx plugin transcoder 😄

it just takes any JPEG you are about to serve, and transcodes it on the fly to JXL if the client supports it

of course it takes a nonzero amount of time, so one would need caching or a (relatively) slow network for this to be a win

The opposite would also be nice, going forward. Store jxl'd jpeg on server, serve directly if supported, convert back to jpeg otherwise

shouldn't be too hard either

Jxl to jpeg should in theory be faster too, maybe

very likely

Of course if 99% of visitors need the jpeg, it will hurt latency

But from the server storage pov, and going towards a future where the majority can use the jxl, it would be the best thing to store the jxl and restore jpeg when needed

(and then at some point you can start using arbitrary jxl and do a lossy decode to q85 or whatever jpeg as a fallback for the minority who needs one - or could use webp as a fallback then, maybe)

I imagine for many use cases, the bandwidth savings are more significant than storage savings

depends on how often people access files

for CDNs, likely yes

for things like facebook, probably not

I'm thinking for a site with little-to-no user content

then yes 😛

<@&803357352664891472> wake up samurais, you have a PR to merge 😉 https://github.com/libjxl/libjxl/pull/45

Ugh, this group ping enabled-by-default 'feature'...

Ok I am not going to disable that one, but please don't abuse it for non-urgent stuff

Ok, I'll remember that.

Thanks for quick review!

Important bugs (not necessarily security), maybe big news like apple or adobe saying something about jxl, I dunno. Things you think all core devs would certainly like to get a notification for 😅

abuse of pinging will result in retaliatory abuse *via* pinging 😛

<@!794205442175402004> maybe we could make a channel that is rate-limited for lower priority things one might want to bring to our attention?

Its "natural" rate is limited anyway, most of the traffic already goes to other channels

Rate limiting is useful to avoid discussions start there

Suggestions for a channel name?

contact-devs?

Added it

I made the fast lossless encoding quite a bit faster, especially in the HDR case where it was unnecessarily slow: https://github.com/libjxl/libjxl/pull/47

probably 0.5.0, I guess

0.4.x is a separate branch where only bugfixes get cherry-picked

the git version shouldn't be that risky, we don't make changes in the main branch that break stuff

But why use new heuristics? They are not developing it

Options that are not in the default (non-verbose) --help of cjxl are experimental. They could disappear at any time, and they are not guaranteed to do anything useful or even work.

That's not the intention. The goal is to have a "no manual tweaking required" encoder where you basically only have to choose a fidelity target, an effort, and maybe progressiveness, and it does something that makes sense.

The experimental options are for experimentation, to try things out in order to make the default behavior better.

basically cjxl was overriding some settings in the specific case of .gif to .jxl with no options, but not in the case of .gif to .jxl with options (like -q 100)

overriding the settings was not particularly useful, it's better to let the encoder figure it out instead of assuming things like a specific predictor being best and whatnot

With these changes `-s 3` could also be better on non-photographic images? https://github.com/libjxl/libjxl/pull/47

nope

-s 2 probably is though

yes but losing accuracy at q70 compared to older version of the encoder

isn't good for the encoder

what are you talking about, Fabian? this is about lossless

like if you make new heuristics and you have regression in medium qualities like they looks less accurate or more distorted blurred less beautified

is not always better

jxl needs also to look good to the users

even if most will not notice as they don't know the accuracy jpeg xl can give

I'm pretty sure we didn't change much recently in the encoder

but the 12052021 version

at

for %i in (C:\Users\User\Documents\bn2*.jpg) do cjxl -j -s 6 -q 63.52 --epf=2 -p --gaborish=1 --patches=0 --dots=1 --use_new_heuristics %i %i.jxl

has great accuracy

great beautify effect

like for jxl art but even for other images it greatly reduces file size

but the problem is whatever encoder you use

if you go up

you don't get same accuracy

it looks less precise

don't use new heuristics

like is modular that could be optimized

i'll encode at less quality

q 67 and q 66

suprisingly q 66 it does a far good job

but it blurs totally the triangle at bottom

particularly q 75 blurs in up the photo

the center is totally blurred in q75

i think if with a newer build it looks sharper at -q 78,97356 -s 7 --use_new_heuristics --dots=1 --gaborish=0

it's a sign that is more fixed the problem

obviously you could try to get a q70 or q 66 sharp picture but it would be too much work

save it for release

or for improving low bpp

I THINK IT DOESN'T SOLVE ANYTHING

Q 78 is still high quality

because is new heuristics so is like doing a q 82 with more modular effort

so it's an illusion

medium quality still looks blurred

This is maybe more for <#840831132009365514> ?

I removed some duplicates from 12:56

ouch nasty libjxl crash possibly found by oss-fuzz (write after free) github doesn't seem to have a "keep this private" option for issues, is it OK to open it there anyway?

I think it's ok to open it publicly, wdyt <@179701849576833024> ?

It's probably something we will then fix quickly anyway and the fix will be public anyway

probably OK for now (til we have a proper security process)

you can also email it I guess, but issue is likely good

hello, OK to report security issues on github libjxl? there doesn't seem to be a "keep private" option in the new issue page

sure

A zero day exploit targeting specifically chrome with the enable-jxl flag enabled is probably not a huge problem? Considering likely the only thing they can do is make a website that makes a tab get killed...

ok, opened here: https://github.com/libjxl/libjxl/issues/66

could probably use it to run arbitrary asm code in a chrome tab

well ... possibly anyway heh

Thanks for the find!

https://i.kym-cdn.com/photos/images/newsfeed/001/401/347/312.jpg

I've just updated my compiling one-liner. Now you won't loose any of the additional files you've kept in `libjxl/build/tools` (e.g. all of those `.jxl`, `.png` or `.jpg` you've processed, `jxl_from_tree` designs etc.) **Make sure you don't have `tools` directory in your home folder!** https://discord.com/channels/794206087879852103/804324493420920833/847204142589018132

Whoaaaa, I only wanted to make some slight modifications to the command from `README.md`...

Another thing is I'm not knowledgeable enough to know `cmake`'s settings and I just trust devs to give the right command that I won't modify

It's a decoder-only library. In many applications you don't need an encoder...

I guess because if you link dynamically, you can probably just depend on libjxl and link to the full library. The decoder-only libjxl_dec would be more for something like a game that uses jxl sprites and wants to do static linking to have a self-contained executable without dependencies.

🤔

static libs increase binary size, dynamic ones not to significant degree. though i think it could just discard encoding stuff which isn't used at all?

probably needs sufficiently smart compiler/linker for that

What do you think of such commits? https://github.com/ziemek99/libjxl/commit/7b4c2dda9eeafe690a76cbf5b22c7a3607c71093

I'm asking you for an opinion, because I don't want to make pull request only to be rejected...

no reason not to 🙂

Also, not really important, but ```=0|1 force enable/disable``` but, "enable" is usually `1`, although `0` comes first

more clearly it's something like ```=0|1 force disable/enable``` or ```=1|0 force enable/disable```

I only followed core devs' convention

It's in the descriptions of all the other options, not just this commit/option

feel free to change nonsensical stuff in the CLI

😛

That's the effect I wanted. I put the edited gaborish option under unedited ones, now you can't discern which one was changed and which ones were "original".

Also ``` -N max_d, --near-lossless=max_d [modular encoding] apply near-lossless preprocessing with maximum delta = max_d``` but it is not clear what the maximum and minimum numbers could be (like 1, 1000, 0.01, -1?)

Yep, but I'm talking about options in general, not this commit (it just reminded me of these descriptions)

There's probably nothing a force-push won't do 😉 Better now? https://github.com/ziemek99/libjxl/commit/1c81d8d4070c7220315e6378e32f35e93c93765d

Yep, because I think it's better to make more changes that apply to one thing than to create a lot of commits with very small changes (such as adding a word, missing a comma, etc.)

It's not like I wanted to make multiple commits, it simply was outside of my *scope* (pun intended)

I made the first commit before this message

I'm also against "micro-commits", I just didn't think I could do it while I'm at it

Yes, if they are necessary, change for the sake of change is also not good

But this change should actually make sense, you were right about pointing it out

pro tip: even if you have multiple commits in your local branch, they'll be squashed when the PR is accepted

How will be the new squashed commit named?

I think the one that does the squash decides

I'm taking a look at some options to expose in Squoosh and could do with some advice. `middleout` - is this worth exposing? Does it only apply to progressive images? `progressive_ac`/`qprogressive_ac` - are these worth exposing? If so, what's the difference between them? `progressive_dc` - worth exposing? If so, what range of values?

`middleout` IMHO it's an option the most worth exposing from that list. Essentially it reorders AC coeffs so they get enhanced from the middle in a spiral pattern instead of usual top-down progression.

Middleout applies to all images, even lossless modular ones, and does what <@456226577798135808> just said

qprogressive_ac is the default progressive AC

(what you get with -p)

progressive_ac is a different scan script, one that is closer to what libjpeg does, I think

Progressive_dc=1 makes sense, higher values probably don't work well

Currently neither djxl nor chrome have a way to show previews of the progressive dc loading though

I would like to speak to the manager of C++! ``` In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/memory:671: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/new:326:12: error: 'operator delete' is unavailable: introduced in macOS 10.12 return __builtin_operator_delete(__ptr, __a1); ^ ```

I'm trying to build the lib on macOS 11, via jpegxl-sys.

Ugh

ugh.

https://tenor.com/view/supervisor-speak-to-the-manager-manager-bubble-gum-princess-adventure-time-gif-9822847

*ahem* <@178524721023811584>

hii!

i cant believe youve discovered my secret identity as the manager of c++

YOU CAN'T HIDE ANYMORE, MUAHAHAHAHAHAAAA

you may think it to be ISO/IEC JTC1 SC22 WG21 but really it was i all along

clang and gcc are but pawns in my shadow war of mysterious motive

I _knew_ it

<@794205442175402004> do you think that such small tweaks *really* require adding myself to the authors list?

not the contributors list, only the authors list

Sorry, my mistake. Fixed the message.

I'm not a lawyer but I guess there's a good reason for being exhaustive on the authors list

So ugg I updated my system and it broke cjxl, it seems to have to do with openexr

Can't compile

I see, thanks

welp the patch fails

should probably try libjpeg-xl-git

Both fail

o. rip

also its odd, none of the code that the patch tries to change is present in codec_exr.cc

the patch tries to change `OpenEXR::Int64` to `uint64_t` but the file has `ExrInt64`

This patch works with upstream on gitlab

If it says ExrInt64 then it is the version that should already be fixed. The old OpenEXR needs OpenEXR::Int64 and the new OpenEXR needs uint64_t, so to work with both versions libjxl now sets ExrInt64 to the appropriate type for the version of OpenEXR that you're using and uses that.

<@!132637059327328256> Your error message says OpenEXR::Int64 so it looks like you were compiling an older libjxl from before it was changed to ExrInt64.

damn the aur package points to the gitlab repo

It looks like the master branch on the gitlab repo says ExrInt64 so it should work. https://gitlab.com/wg1/jpeg-xl/-/blob/main/lib/extras/codec_exr.cc#L115

you might want to cleanbuild 🙂

both libjpeg-xl and libjpeg-xl-git point to the gitlab repo

The gitlab and github repos are in sync, so the main branch is the same.

Oh gitlab has both a main branch and a master branch, and they are different... <:WTF:805391680538148936>

that *might* be my fault

And why do automated GitHub checks fail after merging with main but not in pull requests?

More exhaustive checks are done after merge, too much effort to run them at every pull request update

Oh, msan builds are passing again, nice

it's actually nondeterministic...

you have no idea how much I'm hating that bug 😛

I dunno about a description

but it corresponds to something like being visually lossless when zoomed

-d 1.0 is strictly speaking not quite visually lossless, it is 1 unit of just-noticeable-difference so basically in the worst part of the image, half of the people should just barely see a difference when flipping between original and compressed

half of a just-noticeable-difference should be a not-noticeable-difference 🙂

of course in practice it depends a lot on how much effort you are spending on spotting a difference, and even -d 2.0 can be "visually lossless" according to some definitions

if you define "visually lossless" as "confidence interval of MOS score is overlapping with the confidence interval of the MOS score of the original", then probably for some images even -d 3.0 is "visually lossless"

-d 2.92394 -s 8

should be good for web use

but i don't know what compression achieves

is good

33065 kb

s 9 is still not particularly on this (low bitrate)

but for medium quality compression density has improved

by very much

so i think even for artwork

what CFLAGS you're using

in `/etc/makepkg.conf`

iirc archlinux recently changed their stuff and new compilation flags made things fail for me so I'm using something less rigid

mhhh, weird, it works for me...

it did fail for me too when I tried transfering CFLAGS from `/etc/makepkg.conf.pacnew` to `/etc/makepkg.conf`

Oh, I noticed that <https://github.com/libjxl/libjxl/pull/24> landed after <https://github.com/libjxl/libjxl/pull/23> and <https://github.com/libjxl/libjxl/pull/16>. Should I open a PR to add myself to `AUTHORS` for copyright purposes?

I like this distance description from Jyrki Alakuijala. > -d 4.1 means that you accept errors that are 4.1x what I consider just-noticeable-error. > at -d 1.0 you would get exactly what I consider just-noticeable-errors. And this from old version jxl > Usage: cjpegXL_jxl.exe [OPTIONS] > 100 = mathematically lossless. Default for already-lossy input (JPEG/GIF) > 95 = visually lossless (flicker-test). > 90 = visually lossless (side by side). Default for generic input.

OK, but I'll need to learn something first. I started with "Fetch and merge" in my fork. What should I do next? Edit AUTHORS in my fork and make another PR?

`git commit AUTHORS --amend`

And what if I (try to) contribute only from a web browser? 🙂

On github you can create edits in pull requests directly from the web interface

I've never used it myself though.

I couldn't figure out how to modify an existing PR. So I created a new one. I hope it's how it should be. If not, let me know…😜

you don't need to modify the commit, github will happily squash multiple commits if you have >1

Cool, a PR for better lossy palette? https://github.com/libjxl/libjxl/pull/108/files

Hope pingo dev can provide some code, pingo lossy palette implement(RGB24, RGBA32) is work very well on pixel art content. https://css-ig.net/pingo

<@826537092669767691> is also very familiar with that kind of thing

In jxl you can do funkier stuff than just a color palette though, you can also have delta palette entries which is kind of a game changer

It means you don't have to resort to dithering to do slow gradients in palette mode, you can use deltas and get actual smooth gradients.

Perhaps the Rust version of JXL will motivate new experiments

no pressure there, huh?

😛

No rust no adoption

I'm not sure what to start with on that rust reimplementation - both a fully featured, but possibly not so fast decoder and just a simple JPEG-to-JXL transcoder seem pretty interesting...

<@!826537092669767691> <@!239702523559018497> any opinions there? 😛

Both are interesting from a spec validation pov - a full decoder is more useful than a partial encoder, but both are good to find spec bugs

I'd love a Rust version

I'm stretched thin between too many projects, so don't wait for me to write one

From security perspective of course the most important part is parsing in the decoder

unfortunately I'm not exactly overflowing with free time either 😆 but it's fun, so I do try to find the time 🙂

But OTOH for deployment server-side, decoders are irrelevant.

So for me it'd be important to have an encoder which can guarantee it won't leak uninitialised memory in the file, or do anything silly like that

fair enough, fair enough

Just a JPEG transcoder may be fine for a start

since you're the Rust expert here 😛 - are you aware of any library that is able to do low level parsing of JPEGs?

well, I guess crate is the correct word

https://lib.rs/jpeg-decoder

yeah, I had seen that one, but IIRC it doesn't expose things like DCT coefficients or quantization tables

I've hacked this one to build https://lib.rs/crates/cloudflare-soos

I believe I even opened an issue about that

It exposes its source code 😄

ok, fair enough 😛

I guess a JPEG recompressor would be a much easier task than a full-featured decoder too...

Should there be any Rust coder who wants to do part of a decoder (or encoder), I am happy to (try to) explain any part of the spec / reference code

me too, but I can also help with the coding itself (if you're willing to deal with my lacking Rust knowledge)

so you could say your Rust knowledge is... rusty 😎

yes 😛

Hasn't been tempered yet .... rusty means it has decreased over time, rather than needing to be built up

I did actually write some Rust code ~3 years ago

I don't remember *what* I wrote, I thought it was this https://github.com/edomora97/task-maker-rust but apparently not

ah, it was a rewrite of this https://github.com/algorithm-ninja/cmsocial in rust but it never got to see the light of day 😄

Finally we get delta palettization to produce occasionally better results than jxl:d1, i.e., lower bpp, lower butteraugli max, lower p norm

for example the graphics example image from squoosh.app falls into this category

also images with some gradients have worked

next up with Iulia we will possibly try to add k means clustering of deltas -- that should improve the delta palettization further

Delta palette is a different animal where different compromises make sense

middle-out is great -- but if there are very frequent updates then the visual activity will be confusing -- with middle-out you need Chrome to throttle (like it naturally does) the progressive updates, if you make a video of each tile update (the way progression is sometimes demonstrated), the viewer will get hypnotized or has an epilepsy stroke half way down the image.

the middle-out episode from Silicon Valley is more inappropriate than our memories of it

possibly it would be good to call middle-out middle-out only inofficially, and call it center-first in the official documentation/binary

It is funny that we had dithering in pik -- it got removed at some stage because of decoding speed optimizations and no obvious reduction of quality -- but I anticipated that it will get back, now happy to have it again 😄

Well it's not in libjxl atm, afaik, but it's an easy add since <@179701849576833024> has written the code for it

Did you just read everything you missed during your month of absence?

🤔 <https://docs.google.com/document/d/1mdvt5Q7987wfFwLypDFx7CslQMWmx7R8ytJKSMKKD80/>