JPEG XL

Info

rules 57
github 35276
reddit 647

JPEG XL

tools 4225
website 1655
adoption 20712
image-compression-forum 0

General chat

welcome 3810
introduce-yourself 291
color 1414
photography 3435
other-codecs 23765
on-topic 24923
off-topic 22701

Voice Channels

General 2147

Archived

bot-spam 4380

jxl

Anything JPEG XL related

OkyDooky
2023-02-14 06:51:45
This is what it looks kike in FluffyChat\:
2023-02-14 06:52:13
Screenshot\_2023-02-13\_22-51-08.png
2023-02-14 06:52:19
[Edit](https://discord.com/channels/794206087879852103/794206170445119489/1074946030723932181): This is what it looks like in FluffyChat\:
2023-02-14 06:52:21
screenshot.webp
2023-02-14 06:55:06
ありがとうございます。 Also, \:cringe\:. That looks god-awful.
Traneptora
2023-02-14 10:00:09
2023-02-14 10:00:17
in case you were wondering
zamfofex
2023-02-14 10:06:09
Meanwhile. 🤔
OkyDooky
2023-02-14 10:09:48
How do you not even have a font with katakana glyphs? (<@171381182640947200>)
2023-02-14 10:11:59
On the spam issue, there is ["Mjolnir for all" in beta](https://matrix.to/#/!KizEhDboJpFmMVzCTs:matrix.org/$r7plW1a0-cPCew_hA11japAL-sYhemPNIFnXUo4b9lQ?via=matrix.org&via=kde.org&via=anonymousland.org) which might help with moderation, perhaps worth trying
zamfofex
How do you not even have a font with katakana glyphs? (<@171381182640947200>)
2023-02-14 10:14:50
I only have DejaVu Sans installed. Asian language characters are broken in many websites to me, including Twitter and YouTube. (Though I can’t understand them, so it doesn’t really matter to me.)
Traneptora
2023-02-14 10:20:13
It's helpful to have them render properly since it at least lets you recognize which language is being written
2023-02-14 10:20:21
even if you can't understand the characters
2023-02-14 10:20:46
I can tell that OkyDooky is speaking in Japanese, for example
2023-02-14 10:20:57
because the hiragana is rendering
w
2023-02-14 10:21:03
> anime pfp > doesnt really matter to me
zamfofex
Traneptora It's helpful to have them render properly since it at least lets you recognize which language is being written
2023-02-14 10:24:04
That is actually fair enough. I feel like I’m actually reasonably decent at discerning Chinese from Korean from Japanese by just looking at it. Korean has more straight edges and right angles, and Japanese is curvier. Chinese is just Han, so oftentimes the shapes look more complex.
w
2023-02-14 10:25:01
how i discern: japanese contains hiragana and katakana, characters are chinese characters. korean has straight up circles
Traneptora
2023-02-14 10:25:31
>straight up circles <:PauseChamp:755852977574510632>
zamfofex
w how i discern: japanese contains hiragana and katakana, characters are chinese characters. korean has straight up circles
2023-02-14 10:26:29
That’s fair enough, knowing (at least roughly) the set of kana, but I suppose I was just trying to characterize them in an abstract way.
w
2023-02-14 10:27:32
it was originally more like knowing what Chinese characters are and seeing weird symbols in between
OkyDooky
2023-02-14 11:52:52
another thing special about Korean is that they use halfwidth spaces to separate words, which neither Japanese nor Chinese do
Demiurge
2023-02-14 12:33:03
what's the algebraic relationship between -d and -q values?
2023-02-14 12:33:27
oh, meant to ask this in the <#804324493420920833> channel.
Eugene Vert
Demiurge what's the algebraic relationship between -d and -q values?
2023-02-14 12:49:51
```c++ double distance = args->quality >= 100 ? 0.0 : args->quality >= 30 ? 0.1 + (100 - args->quality) * 0.09 : 53.0 / 3000.0 * args->quality * args->quality - 23.0 / 20.0 * args->quality + 25.0; ``` https://github.com/libjxl/libjxl/blob/main/tools/cjxl_main.cc#L677
Jyrki Alakuijala
...but, wouldn't this kind of make everyone think twice about adopting a next-gen replacement format? I mean, if they can get HDR (future-proofing) and can increase the quality by a lot without increasing the filesize, then they have most of the benefits that the new format offers (that they would care about).
2023-02-14 01:59:13
jpeg1 will never be quite as good as jpeg xl, but it might stretch to be better than avif or webp -- with enough long time horizon it is rewarding to go all the way to (jpeg xl) even when there are great intermediate stops (jpegli)
Deleted User
2023-02-14 06:48:27
I'm trying to compress a boatload of png's in one of my image directories to jpegxl. Was trying for an effect similar to oxipng where it goes over all the files in the directory and compresses them. However due to technical incompetence I have been unable to achieve such a result. `cjxl *.png *.jxl -q 100 -e 9 --brotli_effort 11` this is what I had used.
yoochan
2023-02-14 07:02:01
could you give us some clues about your ability to write batch, powershell, or bash scripts ? are you on windows or linux ?
Deleted User
yoochan could you give us some clues about your ability to write batch, powershell, or bash scripts ? are you on windows or linux ?
2023-02-14 07:10:29
>ability to write batch, pwsh, or bash Absolutely ZERO. I'm on windows.
yoochan
2023-02-14 07:15:47
well... we can try remotely... but I must confess I don't have a windows available 😄
2023-02-14 07:16:24
could you already convert a single file ? with cjxl ?
_wb_
2023-02-14 07:17:17
I don't know how to do that in windows, in linux you can do `for i in *.png; do cjxl $i $(basename $i .png).jxl -q 100 -e 9; done`
2023-02-14 07:17:26
Or a more parallel variant of that
yoochan
2023-02-14 07:18:02
I'm stackoverflowing to try to find the equivalent in .bat 😄
2023-02-14 07:19:50
isn't bash available on windows now ?
2023-02-14 07:23:00
```for %%F in (*.png) do ( cjxl %%F "%%~nF.jxl" -q 100 -e 9 --brotli_effort 11 )``` to put into a `my_convert_script.bat` file
2023-02-14 07:24:48
or ```for %%F in (%*) do (``` on the first line if you want to process all arguments passed to the script instead of all files who match *.png
2023-02-14 07:25:57
but I'm completely blind...
Deleted User
yoochan could you already convert a single file ? with cjxl ?
2023-02-14 07:42:22
yeah that was easy
w
2023-02-14 08:03:01
what i do is just do it in python
Traneptora
2023-02-14 09:02:13
shell all the way
2023-02-14 09:02:41
or GNU parallel
2023-02-14 09:04:02
``` find dir -type f -name '*.png' -print0 | parallel -0 cjxl -d 0 {} {.}.jxl ```
2023-02-14 10:00:29
how does one *encode* an ANS stream?
veluca
2023-02-14 10:07:41
eeeehhhhh
2023-02-14 10:07:46
you mean a jxl-ans stream?
Traneptora
2023-02-14 10:22:55
yea, the kind in JPEG XL
veluca
2023-02-14 11:23:09
well, the general idea is that you just do what the decoder does, but in reverse
Fraetor
2023-02-14 11:49:30
This is a nice script I wrote to bulk convert images to jxl. Use the --help option to see how to use. (Licence CC0)
Traneptora
2023-02-15 12:02:24
well I thought with ans it was a bit more complicated
veluca
2023-02-15 12:39:28
not *really*, inverting what the decoder does is not 100% trivial but it's doable
2023-02-15 12:40:54
https://github.com/libjxl/libjxl/blob/main/lib/jxl/enc_ans.cc#L1633 https://github.com/libjxl/libjxl/blob/main/lib/jxl/enc_ans.h#L58 those two functions are the core of it
2023-02-15 12:42:19
the only nontrivial part is the `reverse_map` thingy, which is built in such a way that `reverse_map[lookup(state).sym][lookup(state).offset] = state % TABLE_SIZE` IIRC
Demiurge
Jyrki Alakuijala jpeg1 will never be quite as good as jpeg xl, but it might stretch to be better than avif or webp -- with enough long time horizon it is rewarding to go all the way to (jpeg xl) even when there are great intermediate stops (jpegli)
2023-02-15 07:33:00
JPEG is already better than webp, especially with XYB
2023-02-15 07:33:27
It's progressive, it looks better, it doesn't have forced chroma resampling etc
2023-02-15 07:34:00
It will support HDR soon too
2023-02-15 07:36:11
webp is pure failure. The only thing it has going for it, is that the compression artifacts look smoother, and it has a nice lossless mode stapled onto it but inheriting most of the severe limitations of webp.
w
2023-02-15 07:38:08
it has browser adoption
Demiurge
2023-02-15 07:40:16
Something really needs to be done about webp, either by changing the format to improve its shortcomings and to justify its existence and the cost of linking another library and all of the harness code and unit tests etc... or it just needs to be nuked from orbit so we can all pretend like it never happened and we're never stuck with these dumb files on our computer that nobody can open.
2023-02-15 07:41:13
Maybe HDR support can be added to webp similar to how it's being added to jpeg
w
2023-02-15 07:41:28
the lossless mode is already decent for screen content
2023-02-15 07:41:35
better than png
Demiurge
2023-02-15 07:42:26
Either way it's insane the way browsers are reacting to JXL, the format with the most promise and potential and outside third-party enthusiasm, and saying "it's not much of an improvement!" after adopting a format like webp
2023-02-15 07:43:22
yeah, lossless webp is by far better than webp and it's usually better and way faster than JXL too. Aside from the lack of features.
w
2023-02-15 07:43:47
and tbf webp is already more than 10 years old
Demiurge
2023-02-15 07:43:50
both PNG and JXL have a lot more features and uses than lossless webp
zamfofex
2023-02-15 07:47:58
Imma be honest, most use cases for people wanting to use images on the web don’t really involve any of the flagship additional features of JPEG XL. And lossy compression (at least from what I have seen) is usually something people don’t really like, and want to try to avoid for most cases (with the exception of photographs sometimes). When it happens automatically (e.g. Twitter), people are usually either indifferent or negative about it, because the artifacts are usually not good when it comes down to anything except photography.
_wb_
2023-02-15 07:55:54
Avoiding lossy compression on the web is not an option.
2023-02-15 07:56:31
That would bump up the median page weight from 2 megabytes to 15 megabytes
afed
2023-02-15 08:10:16
since webp is video based, it's still good for lossy previews with flats, text, graphs because it has less ringing artifacts compared to jpeg for example youtube also uses some filtering when resizing and this makes thumbnails cleaner and flatter which gives much better compression for webp, but jpeg is not very like this
2023-02-15 08:10:33
webp (40398) https://i.ytimg.com/vi_webp/RYMnIGxxqU0/sddefault.webp
2023-02-15 08:10:35
jpeg (59664) https://i.ytimg.com/vi/RYMnIGxxqU0/sddefault.jpg
zamfofex
_wb_ Avoiding lossy compression on the web is not an option.
2023-02-15 08:29:34
That’s fair, I suppose. Note that I didn’t say I think people *never* want to use it, just that it’s not something that is not appropriate for most use‐cases. I think lossy compression has its place, but I also feel like when it is done automatically without choice, it’s not something people usually appreciate. (And editing+recompressing introducing increasingly more artifacts is something that tends to damage media such as memes or other widely popular images over time, as their widespread sharing tends to distance itself from the original source over time.)
Demiurge
Imma be honest, most use cases for people wanting to use images on the web don’t really involve any of the flagship additional features of JPEG XL. And lossy compression (at least from what I have seen) is usually something people don’t really like, and want to try to avoid for most cases (with the exception of photographs sometimes). When it happens automatically (e.g. Twitter), people are usually either indifferent or negative about it, because the artifacts are usually not good when it comes down to anything except photography.
2023-02-15 09:08:45
the features of JXL that I think actually matter a lot are: lossless legacy JPEG transition, progressive/middle-out loading, and (hopefully) high quality colors and shadows, although currently libjxl struggles with both...
2023-02-15 09:11:13
But HDR support is important even though it's not really common right now... In the meantime until HDR viewing becomes more common I think libjxl needs to continue to improve on color and shadow until it can distinguish itself clearly above other codecs.
2023-02-15 09:13:01
It's probably already maybe slightly better than other codecs when it comes to accurately preserving color? But it seems to be noticeably behind when it comes to preserving detailed shadows.
2023-02-15 09:49:51
Hmm, I was doing some reading about spline detection. It might really help actually, if splines were detected and then subtracted from the image before DCT encoding.
2023-02-15 09:50:29
It would probably more than double the amount of time it takes to encode an image though, maybe.
2023-02-15 09:53:31
But, possibly substantial bitrate savings... Not sure if it would help for modular mode though.
2023-02-15 09:54:40
Is there any reason why DCT is theoretically better than Haar/Squeeze transform?
_wb_
2023-02-15 09:55:14
spline detection would definitely be something for the higher effort settings, but yes, especially for some of the image content jxl currently is a bit weak at (like anime), it could potentially help a lot
Demiurge
afed since webp is video based, it's still good for lossy previews with flats, text, graphs because it has less ringing artifacts compared to jpeg for example youtube also uses some filtering when resizing and this makes thumbnails cleaner and flatter which gives much better compression for webp, but jpeg is not very like this
2023-02-15 09:59:43
Yeah, the webp is smaller and a lot better looking than the JPEG here... but I wonder how much of that is because they're using a bad JPEG encoder...
_wb_
2023-02-15 10:08:07
for modular mode it would also help, though for lossless it will likely only help if the spline fit is perfect or very close to perfect
2023-02-15 10:09:33
webp/avif benefit from directional prediction which helps to save bits on (locally) straight edges; jpeg and jxl don't have that
2023-02-15 10:10:39
splines are in principle more expressive though, since they can also help for curved edges, and they're not limited to a fixed set of angles
Demiurge
2023-02-15 10:15:25
You mean they are only more expensive if you use them for detecting conic sections.
2023-02-15 10:15:42
But probably not for straight lines
2023-02-15 10:16:30
Also I'm pretty sure computers are pretty fast with conic sections, it's just another polynomial to them after all...
2023-02-15 10:16:42
In theory.
2023-02-15 10:17:54
Also I don't think splines can help if there is a border without a "line" like two different colored shapes against each other with nothing in between them.
2023-02-15 10:18:31
Splines can only do actual lines right? If there's no line then it isn't any help?
2023-02-15 10:19:38
If there isn't an actual thin line drawn on the border, but it's just two solids against eachother
_wb_
2023-02-15 10:30:23
yeah, jxl splines only help for actual lines
2023-02-15 10:34:50
for borders between two solids, they _could_ also help in some cases, but it seems tricky. If one of the solid colors is either white or black, then in principle you could make a somewhat thick spline at the border to get the border right, and then the solid areas can be filled in a (mostly) DC-only way.
2023-02-15 10:38:44
(that would work because colors brighter than white or darker than black get clamped anyway, so the spline adding white to pixels that are already white does not cause trouble — in the general case though, since splines are drawn in an additive way, not 'replace', you couldn't do a border between two solids of arbitrary color that way)
2023-02-15 10:43:32
in any case, it's the "actual lines" that are the costliest for both DCT and lossless compression (for lossless, prediction will work well if they're straight lines but not so much if they're curved). Hard borders between solid colors are also somewhat costly but it's not as bad, and such parts are probably better done with modular patches than with splines.
DZgas Ж
2023-02-15 11:05:58
cjxl can encode ssim?
_wb_
2023-02-15 11:16:03
Libjxl uses heuristics that (implicitly) optimize mostly for butteraugli 3-norm, and at e8+ it can also do actual optimization for ba
DZgas Ж
_wb_ Libjxl uses heuristics that (implicitly) optimize mostly for butteraugli 3-norm, and at e8+ it can also do actual optimization for ba
2023-02-15 11:17:21
this means that cjxl can only do butteraugli
Demiurge
_wb_ for borders between two solids, they _could_ also help in some cases, but it seems tricky. If one of the solid colors is either white or black, then in principle you could make a somewhat thick spline at the border to get the border right, and then the solid areas can be filled in a (mostly) DC-only way.
2023-02-15 01:28:19
Come to think of it, you COULD have 2 parallel splines of each color on the border to help give the border sharper definition.
2023-02-15 01:29:44
But I am not sure if that would actually help reduce bitrate.
2023-02-15 01:30:31
If the splines are subtracted from the image during preprocessing and added during postprocessing
2023-02-15 01:31:23
It might just increase entropy or fail to reduce entropy before the DCT step.
2023-02-15 01:32:00
If it doesn't reduce entropy and make the DCT more efficient then it's a waste of time
2023-02-15 01:33:48
the whole point of splines is to remove signals that are difficult for DCT
2023-02-15 01:34:21
And sharply defined edges really ARE bad for DCT
2023-02-15 01:34:59
whether there's an actual separate border line between the border or not
OkyDooky
2023-02-15 06:50:46
My concern was more on the "marketing" front, since people like to stick to what's familiar, unless you give them something way shinier that overly meets their beeds (as far as they're concerned; not necessarily in reality). So, my worry was that this might suck away enthusiasm for JXL and increase indifference, or even resistance, towards it; kind of like what was discussed earlier about h.264 versus its designated successors. But, after thinking a bit, this may actually be a saving grace, since it can give something for people to focus on, while buying time for the primary reference encoded for JXL to develop to 1.0 and above, as well as build some enthusiasm for media codec improvements and innovation; and pointing it towards JXL ("wow! I didn't know you could do that with a JPG!" "Well, it's all thanks to JXL." "JXL?" "Yup. It's a new format that offers all these new features and improvements, and even more! In fact, you can get even more savings and better quality than what you saw with that JPG! It's the future!" "Wow! Golly gee willikers, Media Spokesman!" "And how, little Timmy." ...something like that). (<@532010383041363969>)
2023-02-15 06:57:30
Considering only photographs would need near-instant encoding speeds (like for camera apps), I think it would be fine for someone to export their anime illustration from Clip Studio Paint (or whatever) at a slower rate and preserve the quality better. Since, as Pashifox pointed out, people like that usually complain quite loudly when even a single pixel gets "misrepresented" or blemished by some kind of forced image processing, like on Twitter. Actually, places like Pixiv kind of drove me to look into next gen formats, because having 17MB PNGs or bloated 7-12MB JPGs for a regular high-res image, that shrinks to 1.xMB or 700-800KB when renecoded to JPG (again), is just stupid. (<@794205442175402004>)
2023-02-15 06:57:57
[Edit](https://discord.com/channels/794206087879852103/794206170445119489/1075489364139851906): My concern was more on the "marketing" front, since people like to stick to what's familiar, unless you give them something way shinier that overly meets their needs (as far as they're concerned; not necessarily in reality). So, my worry was that this might suck away enthusiasm for JXL and increase indifference, or even resistance, towards it; kind of like what was discussed earlier about h.264 versus its designated successors. But, after thinking a bit, this may actually be a saving grace, since it can give something for people to focus on, while buying time for the primary reference encoded for JXL to develop to 1.0 and above, as well as build some enthusiasm for media codec improvements and innovation; and pointing it towards JXL ("wow! I didn't know you could do that with a JPG!" "Well, it's all thanks to JXL." "JXL?" "Yup. It's a new format that offers all these new features and improvements, and even more! In fact, you can get even more savings and better quality than what you saw with that JPG! It's the future!" "Wow! Golly gee willikers, Media Spokesman!" "And how, little Timmy." ...something like that).
Demiurge
2023-02-15 09:42:32
Yeah, if anything it will just increase attention to JXL since it's made by JXL devteam and the library is, I assume, distributed with or depends on libjxl?
2023-02-15 09:43:30
Also I really hope the psycho improvements to jpegli are also improving jxl encoding too
2023-02-15 09:44:44
since jxl encoding could still use psycho tuning. In fact all lossy codecs can always be improved
2023-02-15 09:45:44
except maybe x264, that's probably pretty optimal now
2023-02-15 09:49:02
But that's sorta just a meme
improver
2023-02-15 09:49:40
jxl as a format is strictly a supperset if you limit yourself to the most popular colorspaces/pixel formats
Demiurge
2023-02-15 09:50:39
Yeah so therefore I hope that jpegli improvements are also jxl improvements
2023-02-15 09:51:02
I don't know if they share the same psy-rd code though
improver
2023-02-15 09:52:29
i wouldn't assume that it would be direct translation, however i believe that some of the new things discovered during jpegli research could benefit jxl eventually
2023-02-15 09:54:33
not to mention giving general faith to underlying algorithms what are shared among them ("jpeg ways of doing things are actually not so old and bad")
2023-02-15 09:55:36
i mean, i personally think that it's actually pretty cool that such an old format can still get improvements from research after all these years
Demiurge
2023-02-16 12:52:18
I mean it would make sense if the jpegli tuning applied to jxl as much as it applies to jpeg, especially on the faster jxl speed settings
2023-02-16 12:53:24
since making good quantization and noise shaping decisions when doing a dct transform is involved in both formats
OkyDooky
2023-02-16 02:01:31
screenshot.webp
2023-02-16 02:03:01
On Element it shows (edited) after the message (which works similarly to Discord), but apparently when it is bridged to Discord, the old message will not be edited - it instead posts a new message as shown in the screenshot.
2023-02-16 02:27:58
Well, that's annoying. Thankfully I just changed "beeds" into "needs," so there shouldn't be any major confusion. But. Thanks for bringing that to my attention.
zamfofex
2023-02-16 02:37:36
I feel like if it highlighted changes in the edited comment, it would be clearer, but I felt like the linked “edit” made it clear enough that something had changed at least.
Dolphin
2023-02-17 04:47:11
Hey, could anyone tell me how histograms + context clustering is done in VarDCT? As far as I understand MA context trees aren't used in VarDCT, so I struggle to understand how histograms are split (like per DCT coef.) . I want to try alternative distribution modelling for lossy mode, but don't know how compatible it is with jxl.
_wb_
2023-02-17 05:02:36
The lowest freq ("DC", but actually an 1:8 image that includes DC for 8x8 blocks and DC + lowest freq AC for bigger blocks) gets modular encoded so there MA ctx trees are actually used.
2023-02-17 05:04:24
For the high freq coeffs ("AC"), there is a somewhat complicated big and mostly fixed context model with thousands of contexts, but then context clustering will map them so the actual number of histograms is much smaller (at most 128 or something like that)
2023-02-17 05:04:59
(MA ctx trees also get clustered so effectively they're not trees but DAGs)
Dolphin
2023-02-17 05:11:17
So how are the contexts picked? Is it just coordinates in a DCT block that determines which context to use, or is there some choice logic based on the already decoded values in the block (although the order of decoding inside a block can be arbitrary?)?
_wb_
2023-02-17 05:35:29
Block type, coeff position, amount of nonzeroes left, the value of the quantized dc coeff and adaptive quant weight all help determine the preclustered ctx
Traneptora
2023-02-17 11:09:33
some people were interested in this: https://github.com/mpv-player/mpv/commit/7607432127d5aa4e2a6e8cc05ea112c19aa9ff7f
2023-02-17 11:10:14
note that the default distance is 1.0. I'm expecting that people who change that to 0 in their config will also change the effort in their config to something realtime like fjxl (e=1)
gb82
2023-02-17 11:52:16
`time cjxl input-pic.png test_e4.jxl -j 1 -e 4 --brotli_effort 1`: 2.07s user 2.02s system 401% cpu 1.019 total `time cjxl input-pic.png test_e3.jxl -j 1 -e 3 --brotli_effort 1`: 1.96s user 2.13s system 417% cpu 0.981 total
2023-02-17 11:52:21
yeah looks like basically nothing
afed
2023-02-18 07:23:48
even for lossless, though e4 is not as fast as e1, but it's not slower than png encoding and has very good compression https://discord.com/channels/794206087879852103/803645746661425173/1069395940294795304
Traneptora some people were interested in this: https://github.com/mpv-player/mpv/commit/7607432127d5aa4e2a6e8cc05ea112c19aa9ff7f
2023-02-18 07:26:39
<https://github.com/mpv-player/mpv/blob/master/DOCS/man/options.rst>
Traneptora
2023-02-18 11:02:12
good catch, I missed the documentation update
DZgas Ж
2023-02-18 12:41:03
so
2023-02-18 12:41:15
how install libjxl in google colab?
2023-02-18 12:43:25
literally
_wb_
2023-02-18 02:01:38
Try `apt-cache search jxl`?
DZgas Ж
_wb_ Try `apt-cache search jxl`?
2023-02-18 02:21:53
_wb_
2023-02-18 07:47:49
What distro is this?
DZgas Ж
_wb_ What distro is this?
2023-02-18 08:40:59
Google Colab use Ubuntu
_wb_
2023-02-18 08:55:09
https://packages.ubuntu.com/source/lunar/jpeg-xl
2023-02-18 08:55:43
I guess it doesn't use latest Ubuntu but an older one?
DZgas Ж
_wb_ https://packages.ubuntu.com/source/lunar/jpeg-xl
2023-02-18 11:15:18
<:Thonk:805904896879493180>
2023-02-18 11:16:35
I don't think you should worry about it. it doesn't matter anymore. it's just that there is such the thing.
gb82
2023-02-18 11:22:24
Hello, quick question -
2023-02-18 11:22:39
2023-02-18 11:22:49
2023-02-18 11:23:27
Upon using both of these bash scripts, the one that uses mozjpeg works fine & the other doesn't. Is there anything I'm doing wrong?
sklwmp
2023-02-19 12:39:15
quoting the variables doesn't fix anything? if so, nothing i can tell at first glance...
gb82
2023-02-19 01:01:37
Current error is ```log_test/input-pic-Q85.jpeg': File name too long @ error/blob.c/OpenBlob/3569. identify: unable to open image 'enc-jpegli:xyb:q85```
2023-02-19 01:01:48
even though I'm fairly sure the filename isn't too long
2023-02-19 01:31:21
you might be right, I don't know how I'd fix that though bc afaik everything looks ok
2023-02-19 01:33:02
even without the `mv` line it fails on convert or ssimu2 i think
2023-02-19 01:33:20
`log_test/input-pic-Q85.jpeg-ssimu2.png' @ error/convert.c/ConvertImageCommand/3342. thread 'main' panicked at 'Failed to open distorted file: IoError(Os { code: 2, kind: NotFound, message: "No such file or directory" })',`
Demiurge
2023-02-19 11:57:19
In variable substitution, it's a good idea to enclose variable names in curly braces and to use quotes to avoid splitting into separate words.
2023-02-19 12:00:50
```bash benchmark_xl --input=$infile --codec=jpeg:enc-jpegli:xyb:q$1 --save_compressed --output_dir=$outdir benchmark_xl --input="${infile}" --codec=jpeg:enc-jpegli:xyb:q"${1}" --save_compressed --output_dir="${outdir}" ```
2023-02-19 12:02:06
the curly braces are not usually necessary, but the double quotes usually ARE necessary if you want to prevent word-splitting, which can turn an expanded variable into multiple separate words/args that was meant to be one word, like a file name
2023-02-19 12:03:40
Basically, if you want something to be treated as 1 word and not separate words, use double quotes, not doing so makes your script very fragile
2023-02-19 12:18:48
Otherwise the shell will split the result into separate words before execution.
2023-02-19 12:19:27
if there are any word separators like spaces
spider-mario
2023-02-19 12:32:27
or newlines (yes)
2023-02-19 12:36:21
here is how to loop over the results of a `find` command in a newline-safe way: ```bash while IFS='' read -d '' filename; do # do something with "$filename" done < <(find -print0 ...) ```
2023-02-19 12:36:53
(if there is a glob expression that does whatever is desired then `for filename in *.jpg` works fine)
2023-02-19 12:37:30
(preferably with `shopt -s nullglob`)
2023-02-19 12:38:56
oh, and when available, `--` is good practice as well
2023-02-19 12:39:01
in case filenames start with `-`
Demiurge
2023-02-19 12:47:00
Yes
sklwmp
gb82
2023-02-20 06:25:19
The issue was that benchmark_xl was outputting to stdout, corrupting the output of the function.
gb82
2023-02-20 06:34:51
Ahh gotcha, let me try that!
ator
2023-02-20 02:31:57
I have created a website for playing board games online, and have been looking for ways to improve the quality of graphic assets without needlessly inflating file sizes. It's a mix of image types - cards with text on varyingly plain or textured backgrounds, and illustrations or photos. Maps in different styles, usually with a lot of line art and small text. Lossy WebP is completely unusable. The chroma subsampling completely ruins colored text and line art, even with the "sharp yuv" filtering. The color shifts it has by enforcing 16-235 video color range is also unacceptable. AVIF erases all textures by aggressively smoothing out low-contrast details, unless using a high enough quality setting where it's no better than plain JPEG. I've also noticed some very odd random compression artefacts where AVIF adds strange lines and dots when compressing images of text. I was really looking forward to being able to eventually use JPEG-XL... <@532010383041363969> So, I've been experimenting with jpegli and I am extremely impressed! The results so far have been outstanding. The way jpegli manages to compress with so much less ringing and mosquito noise is absolute magic. Well done! I do have a few examples where jpegli adds some odd artefacts similar to AVIF. If you're interested I can share some of these cases.
Jyrki Alakuijala
2023-02-20 03:52:25
I'm interested in all examples where jpegli is misbehaving
2023-02-20 03:53:04
if you can post them as issues in libjxl then everyone can participate in the process
HLBG007
Jyrki Alakuijala I'm interested in all examples where jpegli is misbehaving
2023-02-20 03:59:17
Where is a binary encoder for jpegli?
ator
HLBG007 Where is a binary encoder for jpegli?
2023-02-20 04:00:25
You can use benchmark_xl or cjpeg_hdr as built with libjxl.
HLBG007
ator You can use benchmark_xl or cjpeg_hdr as built with libjxl.
2023-02-20 04:01:36
I want encode my own images with jpegli not benchmark jxl
ator
HLBG007 I want encode my own images with jpegli not benchmark jxl
2023-02-20 04:02:17
`benchmark_xl --save_compressed --output_dir=. --codec=jpeg:enc-jpegli --input=*.png`
Demiurge
2023-02-20 04:02:28
does cjpeg_hdr let you control the quality?
username
HLBG007 I want encode my own images with jpegli not benchmark jxl
2023-02-20 04:04:46
benchmark_xl has jpegli embeded in it, there is currently no standalone binary for jpegli
HLBG007
ator `benchmark_xl --save_compressed --output_dir=. --codec=jpeg:enc-jpegli --input=*.png`
2023-02-20 04:07:31
That looks interesting. But documentation about all these features would be nice. Then others can benefit from it too.
_wb_
2023-02-20 04:16:50
I think we should make a cjpegli/djpegli for convenience
ator
2023-02-20 04:34:03
cjpeg_hdr is almost a cjpegli (but with only 'd' parameter)
Demiurge
2023-02-20 04:35:24
Well if libjxl becomes a libjpeg replacement/provider, then it should provide cjpeg/djpeg/jpegtran binaries as well hopefully
ator
2023-02-20 04:38:55
I personally prefer separate tools, easier to know what you're actually using that way
Demiurge
2023-02-20 04:44:07
The idea is that it will be a drop in replacement. Why would you have a need for two jpeg encoders on your system?
2023-02-20 04:45:23
You could always rename the binaries on your system or install them into a prefix if you really want to
improver
2023-02-20 04:45:28
because these encoders are doing things in different ways that ain't universal
ator
Demiurge The idea is that it will be a drop in replacement. Why would you have a need for two jpeg encoders on your system?
2023-02-20 04:46:13
Because just like with mozjpeg, you want to use different encoders in different situations. mozjpeg is great for low quality jpegs, but at q95 or more it's worse than stock libjpeg.
Demiurge
2023-02-20 04:46:42
The only thing that isn't universal is the iccv4 colorpsace, but even without xyb it is still vastly superior to other encoders.
2023-02-20 04:47:31
low quality jpeg looks grotesque.
improver
2023-02-20 04:47:31
encoder choices are never universal
Demiurge
2023-02-20 04:47:40
even with mozjpeg
improver
2023-02-20 04:48:35
"vastly superior" also doesn't override differences of compatibility of outputs produced
Demiurge
2023-02-20 04:51:36
What differences?
improver
2023-02-20 04:52:39
requirement of different colorspace processing, when jpeg decoders historically were bad at it
Demiurge
2023-02-20 04:53:13
That is optional though.
2023-02-20 04:53:27
And even without that, it still makes mozjpeg look like trash
improver
2023-02-20 04:54:31
it's a weird thing that something what used to "look good" before now "is made" to "look like trash"
Demiurge
2023-02-20 04:54:42
Yeah, it's pretty amazing.
2023-02-20 04:55:03
q90 mozjpeg looks like trash next to non-XYB jpegli
improver
2023-02-20 04:55:29
these aren't intellectually sound ways to discuss these things tbh
Demiurge
2023-02-20 04:56:18
well, we are talking about how it subjectively looks. If you want me to sound more erudite and sephisticated I would say that there is less ringing and mosquito noise and the color looks more clear and true.
2023-02-20 04:56:57
but that sounds more dry and boring to me :)
improver
2023-02-20 04:58:12
if you want adaptation of a thing you gotta offer it as a choice to be sampled first, not shove it down people throats
Demiurge
2023-02-20 04:58:44
Here, see for yourself. https://nitter.slipfox.xyz/jyzg/status/1622890389068718080
2023-02-20 04:59:24
This is q90 mozjpeg vs jpegli without XYB
improver
2023-02-20 04:59:55
do you consider this specific selection of images a sufficient benchmark for the performance of the codec in all of its use cases, even private ones?
Demiurge
2023-02-20 04:59:59
Even without XYB it completely blows mozjpeg away
2023-02-20 05:00:29
There might be some "killer samples" that jpegli performs poorly at.
2023-02-20 05:00:38
People should test.
improver
2023-02-20 05:01:32
yes. people should be able to test. they should be given a binary separate from libjpeg's cjpeg
username
2023-02-20 05:01:36
I did notice in a test image I did that some parts of it where lower quality with jpegli compared to mozjpeg
Demiurge
2023-02-20 05:01:46
But all jpeg encoders ane drop in replacements of each other. even mozjpeg conflicts with turbojpeg and all others.
improver
2023-02-20 05:02:05
i consider mozjpeg conflicting more of an issue than a good thing
Demiurge
2023-02-20 05:02:25
The best way to test is to make it easy to build statically-linked tools.
improver
2023-02-20 05:02:53
not all of people are well-versed in compiling their own tools
2023-02-20 05:03:35
the set of people who are experts of looking at images and knowing compilation magics is overlapping but not the same
Demiurge
2023-02-20 05:03:47
then also make it easy to download statically-built tools*
improver
2023-02-20 05:04:23
how about... via package manager?
Demiurge
2023-02-20 05:04:41
packagers are typically brain dead in my experience.
improver
2023-02-20 05:04:53
not always
2023-02-20 05:05:15
don't let your bad experiences of some packagers cloud your judgement of all of them
Demiurge
2023-02-20 05:05:24
You don't want to depend on them and it's not their job to distribute testing/development builds anyways
2023-02-20 05:05:41
Their job is to distribute stable and production-ready software
improver
2023-02-20 05:05:59
not to mention that different named binary could be put in /usr/bin/ manually and still be more comfy to benchmark than one replacing your main cjpeg binary
Demiurge
2023-02-20 05:06:12
And at that point you want to make it easy for people to install libjxl as a drop in replacement for the system libjpeg lib
2023-02-20 05:07:33
The best way for people to test the jpeg encoder is to have a link to download the latest statically built tools on github based on the latest commit
2023-02-20 05:07:43
And even better if you have some noobfriendly GUI
2023-02-20 05:07:51
But I don't see that happening.
improver
2023-02-20 05:07:54
also arguably libjxl is in fact "testing/development builds" right now
2023-02-20 05:07:59
despite that it's still being packaged
Demiurge
2023-02-20 05:09:39
Yes, and packagers are often very bad at their job. Often refusing to distribute updated builds. Or they try to hack and patch old versions themselves while lacking familiarity with the code.
improver
2023-02-20 05:09:58
citation needed regarding "often"
Demiurge
2023-02-20 05:10:09
debian
improver
2023-02-20 05:10:13
some exceptions don't make a rule
Demiurge
2023-02-20 05:11:07
debian is hardly an exception
2023-02-20 05:11:18
They are, for some reason, the most popular linux distro
2023-02-20 05:11:24
despite being the most braindead
2023-02-20 05:11:27
https://www.jwz.org/blog/2016/04/i-would-like-debian-to-stop-shipping-xscreensaver/
2023-02-20 05:11:31
Here's your citation
improver
2023-02-20 05:12:10
imo xscreensaver's behavior here is braindead, i wouldn't expect my software to have ticking expire time
Demiurge
2023-02-20 05:13:03
Even Arch Linux which prides themselves for being close to upstream and rolling release, often has poorly written build scripts written by people who are not familiar with the software being installed or compiled.
improver imo xscreensaver's behavior here is braindead, i wouldn't expect my software to have ticking expire time
2023-02-20 05:13:16
That was added out of sheer frustration with debian's shitshow
improver
2023-02-20 05:13:27
if they chose to use shitty versions of things then honestly it's on debian
Demiurge
2023-02-20 05:14:09
they keep trying to patch old versions themselves instead of just building the latest release from source.
2023-02-20 05:14:44
And then a bunch of debian users write to jwz reporting bugs that were already fixed
improver
2023-02-20 05:14:57
i'm trying to figure out how this connects to stuff we've discussed before
2023-02-20 05:15:17
"they're doing shitty job so lets expect them to not package this at all"?
Demiurge
2023-02-20 05:16:26
well, debian is already packaging libjxl, probably because a lot of third party software on linux uses it. Like kimageformats
2023-02-20 05:16:36
But that doesn't mean we should expect them to do a good job
improver
2023-02-20 05:17:09
yeah. but, like, if you make it harder for them to do a good job, then it's more likely or less that they'll do it?
2023-02-20 05:18:45
though for a tool like cjpegli at least initially i wouldn't expect it to be packaged and it probably shouldn't be until it matures better
2023-02-20 05:19:34
but i don't see why one would want to make additional friction to use a tool or install it alongside your cjpeg
Demiurge
2023-02-20 05:20:10
Debian is still distributing 0.7 for example
2023-02-20 05:20:21
And there have been security fixes in 0.8.1
improver
2023-02-20 05:20:48
yeah. debian being debian. and?
Demiurge
improver but i don't see why one would want to make additional friction to use a tool or install it alongside your cjpeg
2023-02-20 05:21:31
That's how it already is with mozjpeg... try installing mozjpeg in linux...
2023-02-20 05:21:44
It will conflict with turbojpeg
improver
2023-02-20 05:21:58
yeah. but why it has to be that way?
Demiurge
2023-02-20 05:22:04
Unless it was installed in a separate prefix
2023-02-20 05:22:12
It doesn't have to be.
improver
2023-02-20 05:22:12
i used it exactly that way
Demiurge
2023-02-20 05:22:53
But it actually is nice to have it that way if you don't like needlessly reduced quality
improver
2023-02-20 05:23:06
yeah. but it should be an option
Demiurge
2023-02-20 05:23:12
It's nice if everything automatically uses the best jpeg encoder
improver
2023-02-20 05:23:41
until it's widely established what is 'the best jpeg encoder' you can't rush it
2023-02-20 05:25:14
not to mention that it's libjpeg what is the most important for things using jpeg encoding, as they are more likely to link to that instead of invoking cjpeg
Demiurge
2023-02-20 05:25:19
Well, most linux systems and package managers are not that cleverly designed Theoretically if they were more cleverly designed they could allow multiple different versions of the same library to be installed at the same time.
2023-02-20 05:25:33
There is 1 linux distro I am aware of that is designed this way
HLBG007
ator `benchmark_xl --save_compressed --output_dir=. --codec=jpeg:enc-jpegli --input=*.png`
2023-02-20 05:25:42
Its not working in windows >out\\build\\x64-Debug\\tools\\benchmark_xl.exe --save_compressed --output_dir=. --codec=jpeg:enc-jpegli --input=*.png benchmark_xl v0.9.0 0eebc174 [AVX2,SSE4,SSSE3,Unknown] tools\benchmark\benchmark_file_io.cc:190: JXL_FAILURE: directory doesn't exist tools\benchmark\benchmark_xl.cc:982: JXL_CHECK: MatchFiles(Args()->input, &fnames)
Demiurge
2023-02-20 05:32:13
https://www.gobolinux.org/
improver
2023-02-20 05:33:15
it's pretty cool as an experiment ngl
2023-02-20 05:34:29
meanwhile, you could do libjpegli and optionally provide libjpeg wrapping that
2023-02-20 05:36:49
or even have that as a symlink but that'd probably be less clean. imo glue code being different than main stuff would allow exposing more knobs to tune main thing
Traneptora
2023-02-20 06:07:31
there's value in having libjpeg.so compatibility even if you dont' intend to replace system libjpeg
2023-02-20 06:07:47
for example, mozjpeg being slower than turbojpeg makes them recommend against using it as a system libjpeg
2023-02-20 06:08:14
but being a drop-in replacement lets applications choose which one they want to use
2023-02-20 06:08:25
without actually having to change *any* code
2023-02-20 06:09:01
well, any C code. they'd have to change buildscripts but you know what I mean
improver
2023-02-20 06:09:03
that also could mess up ones wanting to link to both things at once though
Traneptora
2023-02-20 06:09:27
well different SONAMEs can fix that
2023-02-20 06:09:34
identical looking API still has value
improver
2023-02-20 06:09:59
wouldn't symbol conflicts mess things up
2023-02-20 06:10:39
and i agree that identical looking API does have value, just unsure if i'd do that as a main thing
_wb_
2023-02-20 07:19:59
Mozjpeg has a cjpeg -revert option that makes it behave like libjpeg-turbo
2023-02-20 07:20:19
We could do that too, I imagine
2023-02-20 07:22:56
(though jpegli is not a fork of libjpeg-turbo like mozjpeg, so it probably is tricky to make it behave _exacty_ like libjpeg-turbo — maybe it will still be a bit more precise in its dct implementation, or something)
afed
2023-02-20 07:27:33
yeah, it might be useful to have efforts and even slower modes, if it gives something for jpeg, I think at least more accurate butteraugli
2023-02-20 07:32:43
and I don't think it matters to be exactly the same as libjpeg-turbo if jpegli can be comparably fast as libjpeg-turbo
zamfofex
Demiurge https://www.gobolinux.org/
2023-02-20 07:34:05
What about Guix? 😄
2023-02-20 07:35:34
See: <https://guix.gnu.org>
Traneptora
HLBG007 Its not working in windows >out\\build\\x64-Debug\\tools\\benchmark_xl.exe --save_compressed --output_dir=. --codec=jpeg:enc-jpegli --input=*.png benchmark_xl v0.9.0 0eebc174 [AVX2,SSE4,SSSE3,Unknown] tools\benchmark\benchmark_file_io.cc:190: JXL_FAILURE: directory doesn't exist tools\benchmark\benchmark_xl.cc:982: JXL_CHECK: MatchFiles(Args()->input, &fnames)
2023-02-20 08:52:46
replace `--input=*.png` with `--input=whatever.png`
2023-02-20 08:52:51
where `whatever.png` is the name of the input file
2023-02-20 08:53:29
not sure how to control the quality though
Demiurge
2023-02-20 09:49:36
If you want to have both on your system there is nothing stopping you from putting jpegli in a different folder/prefix or renaming the tool binaries or downloading the latest static build if that ever becomes available one day.
2023-02-20 09:53:03
Since it would be nice to eventually have it installed as the system libjpeg in the future, it would be nice if it provided the same tools with the same familiar tool names and options.
ator
Traneptora not sure how to control the quality though
2023-02-20 09:53:55
you can control the quality by changing the codec parameter: --codec=jpeg:enc-jpegli:d2.0 --codec=jpeg:enc-jpegli:Q75 (encode to same target size as libjpeg's quality 75) --codec=jpeg:enc-jpegli:xyb:yuv420:d1.0 (can combine options)
Demiurge
2023-02-20 09:54:34
At least by default, if it's being installed on the system.
Traneptora
2023-02-20 09:57:52
thanks
gb82
2023-02-21 01:31:05
anyone know how to build ssimulacra2_rs with image support?
2023-02-21 01:31:22
https://github.com/rust-av/ssimulacra2_bin the docs only have instructions for videos
2023-02-21 01:46:46
nevermind
sklwmp The issue was that benchmark_xl was outputting to stdout, corrupting the output of the function.
2023-02-21 01:51:23
using this script, nothing has changed & it still fails
2023-02-21 02:19:32
works on my desktop though ... although I think ssimulacra2_rs doesn't understand jpegli images & gives them low scores <:FeelsReadingMan:808827102278451241>
2023-02-21 02:58:24
on laptop, not even benchmark_xl is working
2023-02-21 02:59:04
``` /usr/sbin/../lib64/gcc/x86_64-pc-linux-gnu/12.2.1/../../../../include/c++/12.2.1/bits/stl_vector.h:1123: std::vector::reference std::vector<jpegli::JPEGHuffmanCode>::operator[](std::vector::size_type) [_Tp = jpegli::JPEGHuffmanCode, _Alloc = std::allocator<jpegli::JPEGHuffmanCode>]: Assertion '__n < this->size()' failed. [1] 240379 IOT instruction (core dumped) benchmark_xl --input=pixel_ref.png --codec="jpeg:enc-jpegli:xyb:q80"```
sklwmp
gb82 on laptop, not even benchmark_xl is working
2023-02-21 04:49:35
I know what's going on
2023-02-21 04:49:56
benchmark_xl was compiled with `-D_GLIBCXX_ASSERTIONS` which is the default on Arch
2023-02-21 04:49:59
it causes some bugs
2023-02-21 04:50:06
i really cant find the time to file a proper issue though, if somebody could do that, it would be great
gb82 works on my desktop though ... although I think ssimulacra2_rs doesn't understand jpegli images & gives them low scores <:FeelsReadingMan:808827102278451241>
2023-02-21 04:50:28
maybe the color profile?
gb82
sklwmp maybe the color profile?
2023-02-21 06:08:59
yeah, I rebenched with rgb but I'm getting weird results
2023-02-21 06:09:08
sklwmp
2023-02-21 06:14:47
yeah, that's weird...
gb82
2023-02-21 06:25:32
here's the script
2023-02-21 06:25:54
ah, wait I think I know the problem
2023-02-21 06:28:50
yep. two different images. used the wrong jpeg output log <:PepeHands:808829977608323112>
sklwmp i really cant find the time to file a proper issue though, if somebody could do that, it would be great
2023-02-21 06:31:09
I'll do it, if you can direct me to where
2023-02-21 06:32:12
https://github.com/libjxl/libjxl/issues/2207
2023-02-21 06:32:32
nvm, looks like someone (you?) got it
sklwmp
2023-02-21 06:33:22
yea, i finally got around to doing it
2023-02-21 06:33:48
although my mind cant form the words properly, it's like a word vomit of a description, but good enough to describe the issue
gb82
2023-02-21 07:23:31
that's more like it
2023-02-21 07:24:32
looks like jpegli is indeed better, at higher quality in particular. I'll send the source image in a sec
2023-02-21 07:28:40
here's the source. original was 5470x3656 16-bit PNG
fab
2023-02-21 09:54:33
https://es.m.wikipedia.org/wiki/JPEG_XL
2023-02-21 09:55:21
Jon First Wikipedia of JPEG XL with completed translation
2023-02-21 10:08:50
daniilmaks
fab
2023-02-21 10:51:57
hmmm yes, that's a nicely smooth blur
HLBG007
gb82 that's more like it
2023-02-21 03:55:38
the picture does not look like jpegli is 20% better than mozjpeg.
ator
2023-02-21 04:07:17
http://ccxvii.net/jpegli/showcase.html here's a comparison of jpegli (not xyb), mozjpeg, and libjpeg at quality 80-ish jpegli has more banding but significantly less ringing and mosquito noise
2023-02-21 04:08:20
jpegli doesn't like bright lines on red backgrounds though ... see the bright spots around the "STORMS" word
HLBG007
ator http://ccxvii.net/jpegli/showcase.html here's a comparison of jpegli (not xyb), mozjpeg, and libjpeg at quality 80-ish jpegli has more banding but significantly less ringing and mosquito noise
2023-02-21 04:21:20
Oh, I was sure that jpegli is the future of jpeg encoders 😕
afed
2023-02-21 04:33:31
by my testing jpegli is good on q90+, but on lower qualities it might be worse or have other distortions, i think it needs to use additional strategies for lower quality though, on q90+ jpegli can be better by like 30-40%, like i need to bump size by 30%+ to get about the same quality from mozjpeg/libjpeg (but some images still need tuning or have some issues)
ator
2023-02-21 04:34:45
d2 is about when jpegli starts behaving weird. at d1 it's close to perfect in my tests.
HLBG007
afed by my testing jpegli is good on q90+, but on lower qualities it might be worse or have other distortions, i think it needs to use additional strategies for lower quality though, on q90+ jpegli can be better by like 30-40%, like i need to bump size by 30%+ to get about the same quality from mozjpeg/libjpeg (but some images still need tuning or have some issues)
2023-02-21 04:35:04
Is this with the xyb?
afed
HLBG007 Is this with the xyb?
2023-02-21 04:42:29
without xyb, not for all images, but with q90+ jpegli is almost always better, except for some strange blockiness on some color tones as I showed earlier it's less noticeable at q90+, but still https://discord.com/channels/794206087879852103/803645746661425173/1062890996146385006
Traneptora
2023-02-21 04:45:16
only problem with XYB is that it requires an ICCv4 profile which is not supported by firefox
2023-02-21 04:45:31
otherwise it works great
HLBG007
ator d2 is about when jpegli starts behaving weird. at d1 it's close to perfect in my tests.
2023-02-21 04:45:39
I'm working only with Q is d2 full of bugs?
Traneptora
HLBG007 I'm working only with Q is d2 full of bugs?
2023-02-21 04:47:00
quality just maps to distance
2023-02-21 04:49:09
d2 corresponds to approximately q79
username
Traneptora only problem with XYB is that it requires an ICCv4 profile which is not supported by firefox
2023-02-21 04:50:34
uhh firefox supports it fine, well recent versions atleast have the support turned on
HLBG007
2023-02-21 04:50:46
Ok q79 not working with jpegli https://twitter.com/jonsneyers/status/1627605185743794179
Jyrki Alakuijala
2023-02-21 04:56:09
one possibility is to use Q80, not q80 -- then it uses jpegli with the BPP of q80 from libjpeg
2023-02-21 04:57:00
with Q80 you will ask it to produce a better quality with the same bit budget instead of the same quality with a lower bit budget
2023-02-21 04:57:19
however it is slow to encode like that since it needs to run libjpeg and then iterate
2023-02-21 04:58:27
we have ideas how to improve the lower qualities -- the main idea is to re-optimize the quantization matrices for different qualities (for example for d4, and optimize another matrix for d2 and better, interpolate in between)
2023-02-21 04:58:57
I anticipate 3-5 % improvement in low qualities once it is completed
2023-02-21 04:59:35
better is to just use d1 to compress images 🙂
2023-02-21 04:59:47
perhaps go to d1.4 in an emergency
ator jpegli doesn't like bright lines on red backgrounds though ... see the bright spots around the "STORMS" word
2023-02-21 05:05:34
There are subtle things for red handling in xyb, all of that breaks down with usual ycbcr
HLBG007
Jyrki Alakuijala one possibility is to use Q80, not q80 -- then it uses jpegli with the BPP of q80 from libjpeg
2023-02-21 05:07:45
The differents between Q and q is maybe not understanding by different people
Jyrki Alakuijala we have ideas how to improve the lower qualities -- the main idea is to re-optimize the quantization matrices for different qualities (for example for d4, and optimize another matrix for d2 and better, interpolate in between)
2023-02-21 05:08:44
The question is then why not an uppercase D too
afed
2023-02-21 05:12:10
because libjpeg has only q?
Jyrki Alakuijala
2023-02-21 05:13:40
Q80 == libjpeg (or mozjpeg, I'm not sure?) run with quality 80, measuring how many bytes are used, then running iteratively jpegli to produce an image with the same number of bytes
HLBG007
Jyrki Alakuijala Q80 == libjpeg (or mozjpeg, I'm not sure?) run with quality 80, measuring how many bytes are used, then running iteratively jpegli to produce an image with the same number of bytes
2023-02-21 05:14:12
q80 == ?
afed
2023-02-21 05:14:42
https://github.com/libjxl/libjxl/pull/2146
Jyrki Alakuijala
2023-02-21 05:14:43
q80 == quantization that leads to same quality like libjpeg as measured by butteraugli on jyrki31 test corpus
2023-02-21 05:15:05
ssimulacra2 doesn't like jpegli that much
2023-02-21 05:15:39
we will figure it out later perhaps, now we are still following the butteraugli for the first solution as we have experience that butteraugli can be used for tuning codecs
2023-02-21 05:16:07
it could be some simple thing like the banding
2023-02-21 05:16:42
usually ssimulacra2 has been more practical difference metric, but we don't have experience of it in codec design
2023-02-21 05:17:05
also the benchmark_xl's aggregation mechanism doesn't work with it -- since the metric aggregation is geometric, not arithmetic
2023-02-21 05:17:35
and ssimulacra2 can produce a value of 0 or -10 or 0.0001 etc, and those will just completely confuse geomeans
2023-02-21 05:18:21
running nelder mead on such aggregates doesn't work, optimization will find 'clever' solutions that don't really work
HLBG007
Jyrki Alakuijala There are subtle things for red handling in xyb, all of that breaks down with usual ycbcr
2023-02-21 05:23:22
I saw that webpv2 looks better with YIQ colorspace. Have you make tests with this colorspace for jpegli?
Traneptora
username uhh firefox supports it fine, well recent versions atleast have the support turned on
2023-02-21 05:35:50
there was a test earlier here that had issued with iccv4
2023-02-21 05:36:09
I admit I haven't tested it myself tho
gb82
2023-02-21 05:37:06
opening an xyb jpeg in Firefox displays it normally iirc
Traneptora
2023-02-21 05:37:18
normally as in correctly?
HLBG007
2023-02-21 05:37:29
<@532010383041363969> There should not be only one colorspace. The encoder tests through which one compresses best.
gb82
Traneptora normally as in correctly?
2023-02-21 05:38:21
yeah
2023-02-21 05:38:54
here it is, if discord doesn't recompress
username
2023-02-21 05:39:10
discord doesn't recompress anything
gb82
2023-02-21 05:39:10
ignore the preview
username discord doesn't recompress anything
2023-02-21 05:39:27
doesn't it recompress the preview always?
username
2023-02-21 05:39:42
well the previews aren't the file itself
2023-02-21 05:39:51
they are just generated previews
gb82
2023-02-21 05:39:58
oh so it will never recompress the underlying file?
username
2023-02-21 05:40:40
yeah the underlying file stays the same for the most part except sometimes metadata gets stripped out but that's about it
w
2023-02-21 05:41:12
it only always recompresses when uploaded from mobile
Traneptora
2023-02-21 05:41:25
firefox mobile
2023-02-21 05:42:11
chrome mobile
gb82
2023-02-21 05:42:15
Traneptora
2023-02-21 05:42:42
ill check desktop firefox when I get to work
gb82
2023-02-21 05:42:57
works for me
Traneptora
2023-02-21 05:44:40
I wonder if there's a coIor space which is mathematically equivalent to XYB but looks closer to RGB when misinterpreted
2023-02-21 05:44:58
so images like these don't become pink messes
2023-02-21 05:45:17
or at least, YCbCr
2023-02-21 05:46:47
perhaps because X is 1/32 the scale of Y and B, approximately
afed
Traneptora firefox mobile
2023-02-21 05:51:58
https://bugzilla.mozilla.org/show_bug.cgi?id=1412350
_wb_
Traneptora perhaps because X is 1/32 the scale of Y and B, approximately
2023-02-21 06:57:08
That scale thing is different in xyb jpeg anyway, since jpeg uses the same scale for all components
2023-02-21 07:14:46
Making it closer to rgb when misinterpreted is probably possible, but it will still be quite different and then I think maybe an obviously wrong color is better than something quite wrong but not obviously.
Jyrki Alakuijala
Traneptora I wonder if there's a coIor space which is mathematically equivalent to XYB but looks closer to RGB when misinterpreted
2023-02-21 08:38:19
We searched for an expression of XYB in YCbCr instead of RGB space, unfortunately such expression doesn't exist -- we didn't think, but used an optimization system, I also tried some manual exploration, but it is a no
2023-02-21 08:39:56
could be that we missed something -- but I did my best for about a week with a help of a person smarter than me (Zoltan!) and I convinced myself that it is too difficult for me
Traneptora
2023-02-21 09:35:19
hm, but maybe what jon said is right
2023-02-21 09:35:26
where you'd rather have obviously wrong than subtly wrong
ayumi
Traneptora firefox mobile
2023-02-21 11:07:05
IIRC after Android version of Firefox 68 (the last one based on "Fennec" codebase), Mozilla disabled `about:config` in the release versions. If you use a build that has `about:config` enabled and you set `gfx.color_management.enablev4` to `true`, it will work:
2023-02-21 11:09:27
Traneptora
ayumi IIRC after Android version of Firefox 68 (the last one based on "Fennec" codebase), Mozilla disabled `about:config` in the release versions. If you use a build that has `about:config` enabled and you set `gfx.color_management.enablev4` to `true`, it will work:
2023-02-21 11:13:01
how would I use such a build?
ayumi
2023-02-21 11:39:20
There are a few options: 1. You can use the Beta or Nightly build that Mozilla distributes. Those already have `gfx.color_management.enablev4` set to `true`, however you will still need to set `gfx.color_management.mode` to `1` or `2` (platform default) to make it work. 2. If you insist on using a version build from the release branch, there is one offered by F-Droid, however due to how F-Droid works, it always lags behind official Mozilla releases. It also does not use the official Mozilla branding and IIRC has few other changes. You will also need to change the `gfx.color_management.mode` pref to `1` or `2`. 3. You could also build the browser yourself, in which case you can just remove/comment `mobile/android/app/mobile.js:243` to make the it use the default `2` value, enabling colour management.
gb82
2023-02-22 08:55:08
https://github.com/oupson/jxlviewer/issues/21
2023-02-22 08:55:19
Anyone know the answer to this?
OkyDooky
2023-02-22 11:28:56
The Fennec version on F-Droid is good, since it is basically vanilla Firefox. Another version, one tuned for security and privacy is Mull. I might try to see if that works with the above flag prompt. (<@853026420792360980>)
2023-02-22 11:32:51
Since there was some talk above about AV1 vs h.264, would something like JXL's progressive decoding be possible in a video codec? Unless I misunderstood, it seems that both a thumbnail and lower resolution/lower quality versions of the same image can be served from the same JXL file. If that's true, and if that could be applied to a video codec, that could potentially elimate most of the need for transcoding to different versions like YouTube does. That woild benefit them, and also Peertube instances.
improver
2023-02-22 11:52:19
you'd need to split out preview parts and rest of content parts
2023-02-22 11:52:27
and you'd lose inter-frame coding
2023-02-22 11:52:32
imo not worth it
ayumi
2023-02-22 11:52:33
You do not do progressive decoding with video*, because either client that only wants low quality frames would still need to download the higher quality bits too, or client wanting higher quality frames would need to buffer the whole low quality video first.
2023-02-22 11:55:17
*MPEG-5 Part 2 kind of does the first variant, but its does it for backward-compatibility reasons.
OkyDooky
2023-02-23 01:13:02
Alright. I'll keep thinking of something. There's probably a way to design a video codec to allow multi-quality level options for streaming in a viable way. It probably would be bigger than a regular video (like h.26x or VPx/AVx), but would be smaller and (hopefully) faster to encode than 4-5+ separate versions, so that and it being one file would still make it worth it for hosting and streaming purposes. Ah, then there'd probably need to be an audio codec that could do that, too. Idk.
2023-02-23 01:15:36
Regarding JXL's non-hypothetical progressive dexosing feature\: could this be dynamically resampled on the fly? I'm thinking in the context of games, where JXL could effectively replace mipmaps and other LOD methods for texture maps.
2023-02-23 01:15:56
dexosing=decoding. Oops.
ayumi
2023-02-23 01:33:44
AFAIK (and please correct me, if I am wrong) you can currently progressively encode 1:32, 1:16 and 1:8 in the DC frame and 1:4, 1:2 and 1:1 in the AC frame. I think that the codec itself allows for more, but without access to the standard, I am not sure.
2023-02-23 01:34:35
Also, IIRC the current decoder API will not give you anything lower than 1:8.
gb82
2023-02-23 07:00:21
Bit of testing going on in the AV1 server rn - how is it possible that the system time difference is *this big* between JXL & AVIF?
_wb_
gb82 Bit of testing going on in the AV1 server rn - how is it possible that the system time difference is *this big* between JXL & AVIF?
2023-02-23 07:34:39
maybe memory allocations?
gb82
2023-02-23 07:35:18
it is being argued that this is a reason for AVIF's superiority, bc much longer system times don't scale on CDNs
_wb_ maybe memory allocations?
2023-02-23 07:35:32
Is this a weakness of JXL?
_wb_
2023-02-23 07:36:07
i think it's a weakness of libjxl
gb82
2023-02-23 07:36:36
on the bright side, user times are fantastic
_wb_
2023-02-23 07:37:40
nothing specific in the bitstream requires allocating lots of memory, but current libjxl encoding was pretty much made without caring much about memory footprint
2023-02-23 07:39:10
(in particular it tokenizes the entire image to do global context clustering; this is quite memory-intensive and could be avoided at a small cost in compression)
gb82
2023-02-23 07:39:53
Gotcha. So the flexibility of the codec would allow different implementations in the future, while right now it is an encoder-specific weakness?
_wb_
2023-02-23 07:40:23
https://github.com/libjxl/libjxl-tiny is already doing something else right now
2023-02-23 07:43:24
it's the model we're using for the purposes of hardware encode, where you also want to avoid memory bandwidth (and where you want to do as much as possible in SRAM instead of DRAM)
gb82
2023-02-23 07:45:05
Would that be for use in cameras or smartphone ISP's to base their implementations on?
_wb_
2023-02-23 07:46:22
for very fast lossy encodes, better things than libjxl e3 are possible — for lossy encodes in software though, especially when going down to lower qualities, I would recommend libjxl e5+
gb82
2023-02-23 07:47:28
Also, morbid curiosity - why can JXL be used directly in cameras & AVIF cannot?
_wb_
2023-02-23 07:51:04
both could be used in principle, I assume. Not sure when cameras with av1 _encode_ hardware will arrive, but when they do, it should in principle be easy to also use it for still images, in the way Apple does with its hevc hardware encoder on iPhones (tiled)
2023-02-23 07:52:10
but I think that at "camera quality" (which is libjpeg q95+), the advantage of avif is likely very questionable
gb82
2023-02-23 07:52:46
every benchmark I've done favors JXL near what is considered visually lossless by ssimulacra2
w
2023-02-23 07:57:01
im sure most of the time what people choose is what is most convenient
2023-02-23 07:57:26
and people includes the hardware manufacturers
gb82
2023-02-23 07:58:25
we saw a lot of flagship Android phones use HEVC for videos, & iPhones use HEIC + HEVC for a bit now
_wb_
2023-02-23 08:03:39
even default libjpeg-turbo (which is similar to jpeg hardware encoding) at q95+ 4:4:4 is almost as good as s9 libaom avif (which is likely even an overestimate of how good hardware avif encoding would be), or even better, depending on which metric you ask.
2023-02-23 08:09:15
at camera quality, and using hardware encoding (or very fast software encoding) I would estimate that avif can only bring a 5-10% compression improvement over jpeg at best (and a small regression at worst), while jxl can bring a ~45% compression improvement over jpeg
gb82
2023-02-23 08:09:49
I've seen some test case images where AVIF shits the bed
_wb_
2023-02-23 08:10:36
so if I were a camera vendor, I wouldn't be rushing to switch to avif — the gains just aren't there, while the interop regression is very much there
gb82
2023-02-23 08:13:24
gotcha, makes sense. I know from video encoding that av1 < h265 < h264 for super high bitrate
_wb_
2023-02-23 08:14:11
iPhones are only able to use HEIC because Apple controls both the hw and software side, can make stuff work by converting heic to jpeg on the fly when needed, but even then I think they're getting plenty of complaints about both quality and interop
gb82
2023-02-23 08:33:58
yeah, the college board had some issues with HEIC that one time
2023-02-23 08:34:12
https://www.theverge.com/2020/5/20/21262302/ap-test-fail-iphone-photos-glitch-email-college-board-jpeg-heic
2023-02-23 10:00:23
As wb explained, that may not be beneficial in any way except causing issues like the one above
_wb_
2023-02-23 10:02:33
unless they significantly lower the fidelity, there wouldn't really be much advantage of using avif by default. Not to mention it would be way too slow, since the encode would have to be done in software...
BlueSwordM
gb82 Bit of testing going on in the AV1 server rn - how is it possible that the system time difference is *this big* between JXL & AVIF?
2023-02-23 05:14:46
BTW, please do not always force 4:4:4. There is no need to do so.
_wb_
BlueSwordM BTW, please do not always force 4:4:4. There is no need to do so.
2023-02-23 06:28:05
no need because avifenc does 444 by default on png input, or what do you mean?
HLBG007
2023-02-23 06:44:38
I have updated squoosh with avif/jxl from today git https://dominikhlbg.github.io/ to compare open this in another tab https://squoosh.app/
2023-02-23 07:03:35
the new avif/aom/jxl version 02 2023 (http://dominikhlbg.github.io) the old avif/aom version 05 2021 / jxl 01 2022 (https://squoosh.app)
2023-02-23 07:07:27
What do you thinking about the quality changes between the years of developing? 🙂
2023-02-23 07:07:48
wp2 is not compileable
2023-02-23 07:15:06
the source code i put later online
Traneptora
2023-02-23 07:19:27
Hm, it appears XYB jpeg is messing with the colors
2023-02-23 07:19:38
2023-02-23 07:19:48
view these both in browser, and swap between them
2023-02-23 07:20:17
I lied, it's my image viewer that's not handling XYB jpeg properly
2023-02-23 07:22:21
nvm lol
HLBG007
Traneptora
2023-02-23 07:37:12
what did you use to encode the xyb picture?
gb82
BlueSwordM BTW, please do not always force 4:4:4. There is no need to do so.
2023-02-23 08:29:41
Why? Wouldn't it go 420 otherwise?
BlueSwordM
_wb_ no need because avifenc does 444 by default on png input, or what do you mean?
2023-02-24 12:14:40
Because it keeps the chroma subsampling, yes.
gb82 Why? Wouldn't it go 420 otherwise?
2023-02-24 12:15:39
No.
Traneptora
HLBG007 what did you use to encode the xyb picture?
2023-02-24 02:02:57
``` benchmark_xl --save_compressed --output-dir=. --codec=jpeg:enc-jpegli:xyb:q90 --input=lenna.png ```
_yummersdeluxe_
2023-02-24 05:18:01
Hey Jon, i hope it doesn't bother you to ask, but how do you feel about what Google did in regards to removing JXL support in Chrome? I've been following JXLs development for a while and that announcement was honestly disheartening to see, i still do believe JXL is the future and i hope more and more tools support it to apply more pressure on Google to backtrack on their decision
Deleted User
2023-02-24 06:32:39
media could provide such pressure, but there is nearly no interest ...
gb82
Hey Jon, i hope it doesn't bother you to ask, but how do you feel about what Google did in regards to removing JXL support in Chrome? I've been following JXLs development for a while and that announcement was honestly disheartening to see, i still do believe JXL is the future and i hope more and more tools support it to apply more pressure on Google to backtrack on their decision
2023-02-24 07:44:09
he explained a bit ago
2023-02-24 07:46:23
https://discord.com/channels/794206087879852103/803574970180829194/1078583611432910901
_wb_
2023-02-24 08:03:31
See also: https://www.reddit.com/r/jpegxl/comments/113lria/comment/j9nyy83/?utm_source=share&utm_medium=web2x&context=3
Jyrki Alakuijala
gb82 he explained a bit ago
2023-02-24 01:08:55
I believe Chrome removed JPEG XL support, while Google Research is continuing to develop JPEG XL -- I believe that no one from Google is attempting to speak with the voice of Google about JPEG XL
gb82 gotcha, makes sense. I know from video encoding that av1 < h265 < h264 for super high bitrate
2023-02-24 01:10:52
This may also need to be retested with a 'jpegli' like approach -- of course there is a pretty hard wall there as 264 decoders don't always give 10 bits of dynamics...
gb82 Bit of testing going on in the AV1 server rn - how is it possible that the system time difference is *this big* between JXL & AVIF?
2023-02-24 01:12:06
strace the runs?
HLBG007
Jyrki Alakuijala I believe Chrome removed JPEG XL support, while Google Research is continuing to develop JPEG XL -- I believe that no one from Google is attempting to speak with the voice of Google about JPEG XL
2023-02-24 02:33:14
Why Google Research do not develop an owm browser? I have already a name: Chrome Science Research - The web browser of future technology
Peter Samuels
2023-02-24 02:35:25
Google Ultron
gb82
Jyrki Alakuijala This may also need to be retested with a 'jpegli' like approach -- of course there is a pretty hard wall there as 264 decoders don't always give 10 bits of dynamics...
2023-02-24 05:25:00
sure, but for fine detail retention at very high bitrates, AV1 is unequivocally worse right now
Jyrki Alakuijala I believe Chrome removed JPEG XL support, while Google Research is continuing to develop JPEG XL -- I believe that no one from Google is attempting to speak with the voice of Google about JPEG XL
2023-02-24 05:26:02
Ur right, I should correct myself that the Chromium team is the obstacle, not Google in its entirety
HLBG007
Jyrki Alakuijala I believe Chrome removed JPEG XL support, while Google Research is continuing to develop JPEG XL -- I believe that no one from Google is attempting to speak with the voice of Google about JPEG XL
2023-02-25 08:03:40
How did you manage to convince Apple and Microsoft with Brotli? https://caniuse.com/?search=brotli Why did you not succeed with jpegxl? https://caniuse.com/?search=jpegxl
_wb_
2023-02-25 08:06:38
Chrome does not have a general-purpose compression team afaik. They do have a codec team (who did vp8/vp9/av1/webp/avif).
2023-02-25 08:08:03
Once Chrome supports something and it gets adoption, Firefox and Safari eventually have to follow. Edge already follows automatically, being a chromium-derived browser.
HLBG007
_wb_ Once Chrome supports something and it gets adoption, Firefox and Safari eventually have to follow. Edge already follows automatically, being a chromium-derived browser.
2023-02-25 08:21:15
Didn't Microsoft already have brotli inside when they were still developing themselves?
2023-02-25 08:30:33
I am missing measurements how much watt is needed when encoding and decoding jpegxl/avif/mozjpeg.
Demiurge
2023-02-26 12:33:31
The Google codec team is frankly acting like shameless clowns, forcing the half-baked webp format through without addressing any criticism, while outside experts were practically begging them not to introduce a new format that wasn't a clear improvement. Now outside experts from a whole bunch of different companies are all begging Google to enable their excellent and mature JXL support by default and they respond by unilaterally deleting all of the code, with once again no discussion or addressing anyone's concerns. Simply telling us that "AVIF is faster than JXL" (lmao) and that there isn't enough room for both JXL and their own special baby to co-exist, or in other words they don't want their precious child to have to compete with JXL.
2023-02-26 12:35:11
Literally the same people who lead the project to develop webp and AVIF and get them fast tracked are telling us to be conservative when adopting JXL, a format that has far more merit.
2023-02-26 12:39:13
And they have (ab)used their power as the "Chrome Codec Team" to push their mediocrity on the world and shut down an alternative, simply because they are afraid people will use it instead of the format they helped create. And the other parts of Google and the Chromium team have been pretty silent. Do these clowns have to explain their decisions to anyone?
2023-02-26 12:39:57
How come no one else on the Chromium team asks why this is happening?
2023-02-26 12:40:43
Surely there are more people in charge of Chromium development and direction than Jim Bankowski.
2023-02-26 12:44:21
Maybe I'm getting too emotional about this but it seems like a totally shameless and petty thing.
2023-02-26 12:45:25
What did their "partners" tell them when they wanted to force webp on everyone?
zamfofex
Demiurge How come no one else on the Chromium team asks why this is happening?
2023-02-26 12:47:46
I feel like Chrome developers are more interested in keeping their jobs. Maybe they are interested in JPEG XL, but they might not feel as passionate about it, not enough to warrant risking their positions as paid developers of the project.
Demiurge
2023-02-26 12:48:18
They're listening to their "partners" now and that's why they're removing JXL? Conveniently these partners don't have any names either. Meanwhile everyone on the issue tracker (real people with names) were waiting for it to be enabled by default.
2023-02-26 12:49:50
I doubt that being leader of the codec team gives him enough power to put peoples jobs at stake who aren't in the codec team.
username
2023-02-26 12:51:48
it might also be that supporters of JXL in google would rather wait before attempting to show their support for JXL or request it's inclusion back into chrome as doing so right after it being removed doesn't sound like the best idea as the answer they are going to get is likely "no" if they ask immediately.
Demiurge
2023-02-26 12:52:16
I just wonder why Jim doesn't have to explain this decision to anyone else, he's not in charge of Chromium development, only the codec team.
username
2023-02-26 12:54:07
it's likely others have no opinion on JXL and are focused on other things since they aren't apart of the team that decides codec related things
Demiurge
2023-02-26 12:54:23
It looks like he has an extremely personal interest in the decision, like JXL gave him the bad touch