|
Jyrki Alakuijala
|
2022-12-29 09:27:27
|
I'll hope to come up with a mitigation today without ruining DZgas image as a side effect 🙂
|
|
2022-12-29 07:24:08
|
https://github.com/libjxl/libjxl/pull/2007 makes a small improvement on DZgas' image and fixes the previous degradations -- it should be a substantial improvement on anime again, but uses some more bits (moderate if thinking about worst case based on my observations)
|
|
|
Traneptora
|
|
andrew
should I just do a straightforward port of jxlatte or something as a starting point?
|
|
2022-12-30 01:50:29
|
jxlatte follows the spec pretty closely so I'd recommend reading the spec instead of porting it
|
|
|
BlueSwordM
>JPEG2000 <:monkaMega:809252622900789269>
|
|
2022-12-30 02:03:45
|
the signature box is similar to the J2k signature box
|
|
2022-12-30 02:04:22
|
in particular 0x0D0A870A is also used in JPEG 2000
|
|
|
_wb_
|
2022-12-30 06:42:05
|
The first four bytes are the size of the signature box (it's big-endian uint32 saying 12). Then the j2k one has "jP " and the jxl one has "JXL ". The final four bytes I am not sure actually where they come from. Some identifier reserved for the JPEG committee maybe?
|
|
|
yurume
|
|
andrew
at this point I've realized that it's not feasible for me to write a "from scratch" Rust implementation of JPEG-XL from the spec in a reasonable amount of time
|
|
2022-12-30 06:58:51
|
to my knowledge someone in this server did have a significant progress
|
|
|
Tirr
|
2022-12-30 09:12:54
|
I'm writing a jxl decoder from scratch in Rust, modular decoding isn't finished yet though
|
|
2022-12-30 09:14:52
|
entropy decoding is complete at least
|
|
|
|
Jyrki Alakuijala
|
2022-12-30 10:14:48
|
so many good news
|
|
2022-12-30 10:23:02
|
interestingly scalar and simd versions seem to encode to about 1.5 % different sizes
|
|
2022-12-30 10:23:22
|
smells like a substantial bug is lurking there somewhere
|
|
2022-12-30 10:24:55
|
is there an easy way to convert a single function from simd to scalar while retaining highway conventions?
|
|
2022-12-30 10:27:37
|
I start to think that to solve the DZgas' image (red green) properly, I need to rethink how that image is received in the eye, how the compression curve (gamma-like) is dealing with it, looking at how I compensate for it in the adaptive quantization and be a bit more precise there
|
|
|
|
veluca
|
|
Jyrki Alakuijala
is there an easy way to convert a single function from simd to scalar while retaining highway conventions?
|
|
2022-12-30 10:57:34
|
run it with a descriptor that is a HWY_CAPPED(float, 1)
|
|
2022-12-30 10:57:39
|
or something like that
|
|
2022-12-30 10:57:48
|
or, compile the whole thing only enabling support for EMU128
|
|
|
lithium
|
2022-12-30 03:11:03
|
If I want preserve original image ICC profile on djxl decode png image,
I should specify which parameter?
> png(Adobe RGB (1998)) -> jxl -> png(preserve Adobe RGB (1998))
> --icc_out=FILENAME
> --orig_icc_out=FILENAME
|
|
|
|
Traneptora
|
2022-12-30 05:38:28
|
I'd think that cjxl should have a `--use_original_profile` option but I don't see it
|
|
|
|
Jyrki Alakuijala
|
2022-12-30 06:10:24
|
https://github.com/libjxl/libjxl/pull/2010 should make the DZgas red-green image somewhat ok
|
|
|
DustinBrett
|
2022-12-30 07:41:25
|
Howdy all, I recently added JPEG XL support to my desktop environment in the browser. If anyone is interested I could share a link assuming this is the right place for that.
|
|
2022-12-30 07:43:20
|
I also wrote an article about it last night. https://dev.to/dustinbrett/adding-jpeg-xl-qoi-support-to-my-website-os-3oni
|
|
|
|
Jyrki Alakuijala
|
2022-12-30 11:55:10
|
what is a Website OS? making a webserver be like PC ?
|
|
|
OkyDooky
|
2022-12-31 03:42:07
|
hi folks, I'd like to batch migrate a ton of my photos to JXL, and for that I kinda need the assurance that the EXIF metadata (or anything similar) would be preserved *and* readable by desktop Linux apps, like gthumb, GIMP, Shotwell, Nautilus, etc. So far, I've been [unable](https://github.com/Exiv2/exiv2/issues/2398) to figure out if those apps would be able to display that metadata automagically with the current outputs of cjxl, if exiv2 was not [neutered in Fedora](https://bugzilla.redhat.com/show_bug.cgi?id=1979565) ; is there an easy way to determine if it's just the library being fubar, or if the apps would need further work to support it too (i.e. if I need to be filing bug reports)?
|
|
|
pandakekok9
|
2022-12-31 04:13:55
|
The browser is literally an operating system
|
|
2022-12-31 04:14:04
|
:P
|
|
|
|
DustinBrett
|
|
Jyrki Alakuijala
what is a Website OS? making a webserver be like PC ?
|
|
2022-12-31 06:16:17
|
It's a bit of a stretch but the name is daedalOS so I treat it like an OS. Basically it's a desktop environment in the browser. I've tried to simulate the "OS" experience as best I can. And I continue to add stuff most weeks.
|
|
2022-12-31 06:19:28
|
Thanks for summarizing. It can do some less "very very simple" stuff like reading JPEG XL, converting audio/video (ffmpeg) photo (imagemagick) and various other things (git, python, etc), usually with WebAssembly involved.
|
|
|
sklwmp
|
|
DustinBrett
Howdy all, I recently added JPEG XL support to my desktop environment in the browser. If anyone is interested I could share a link assuming this is the right place for that.
|
|
2022-12-31 06:54:23
|
Try <#803574970180829194> for software support
|
|
|
DustinBrett
It's a bit of a stretch but the name is daedalOS so I treat it like an OS. Basically it's a desktop environment in the browser. I've tried to simulate the "OS" experience as best I can. And I continue to add stuff most weeks.
|
|
2022-12-31 06:54:33
|
On another note, your software is amazing!
|
|
2022-12-31 06:55:12
|
The animations are really quite spot on; it feels professionally-made and was really enjoyable to tinker around with.
|
|
2022-12-31 06:56:40
|
Sometimes I forget I'm in a browser and not actually in Windows.
|
|
|
|
lithium
|
|
Traneptora
I'd think that cjxl should have a `--use_original_profile` option but I don't see it
|
|
2022-12-31 08:20:59
|
`v0.7.0 36ece47` cjxl and djxl can preserve original image ICC profile,
but when I change to `v0.8.0 5853ad9`, djxl decode png can't preserve original ICC profile.
> Unknown argument: --use_original_profile
> cjxl -d 0.5 -e 7 --num_threads=12 --use_original_profile=1
> JPEG XL encoder v0.8.0 5853ad9
|
|
|
|
Traneptora
|
|
lithium
`v0.7.0 36ece47` cjxl and djxl can preserve original image ICC profile,
but when I change to `v0.8.0 5853ad9`, djxl decode png can't preserve original ICC profile.
> Unknown argument: --use_original_profile
> cjxl -d 0.5 -e 7 --num_threads=12 --use_original_profile=1
> JPEG XL encoder v0.8.0 5853ad9
|
|
2022-12-31 11:48:10
|
I didn't mean that it has that option, but rather that it should
|
|
|
|
lithium
|
|
Traneptora
I didn't mean that it has that option, but rather that it should
|
|
2022-12-31 01:23:20
|
sorry I made a mistake...
|
|
|
|
Traneptora
|
2022-12-31 02:45:24
|
no worries
|
|
2022-12-31 02:45:36
|
I might be able to get that added
|
|
2022-12-31 02:45:55
|
<@461421345302118401> do you have a sample?
|
|
2022-12-31 02:46:04
|
of a PNG with adobe RGB profile
|
|
2022-12-31 02:51:03
|
actually nvm, I'll use the bench
|
|
|
|
lithium
|
2022-12-31 04:23:51
|
png rgb24
icc profile: Adobe RGB (1998)
|
|
2022-12-31 04:25:01
|
|
|
2022-12-31 04:25:14
|
and icc profile file
|
|
|
|
Traneptora
|
2022-12-31 04:26:44
|
thanks
|
|
|
|
lithium
|
|
Traneptora
thanks
|
|
2022-12-31 04:28:34
|
Thank you for your help 🙂
|
|
|
|
Traneptora
|
|
lithium
`v0.7.0 36ece47` cjxl and djxl can preserve original image ICC profile,
but when I change to `v0.8.0 5853ad9`, djxl decode png can't preserve original ICC profile.
> Unknown argument: --use_original_profile
> cjxl -d 0.5 -e 7 --num_threads=12 --use_original_profile=1
> JPEG XL encoder v0.8.0 5853ad9
|
|
2022-12-31 08:04:41
|
https://github.com/libjxl/libjxl/pull/2012
|
|
|
pshufb
|
|
DustinBrett
I also wrote an article about it last night. https://dev.to/dustinbrett/adding-jpeg-xl-qoi-support-to-my-website-os-3oni
|
|
2022-12-31 11:40:35
|
this is crazy cool
|
|
|
|
DustinBrett
|
|
sklwmp
On another note, your software is amazing!
|
|
2023-01-01 01:30:02
|
Thanks very much!
|
|
|
|
OkyDooky
|
2023-01-01 07:55:53
|
So, you cobbled this DE together yourself? \:O
|
|
2023-01-01 09:12:12
|
yes, that's what my links said; my question however was if there is a way to easily determine if the *only* thing preventing those apps from showing EXIF metadata from JXL images is the broken/neutered BMFF support in exiv2, or if there is more that needs to be reported on those apps, and how to test that without essentially having to know how to compile exiv2 and overwrite the system's version (and hopefully not need to recompile all apps with it?)...
(<@456226577798135808>)
|
|
|
|
Jyrki Alakuijala
|
2023-01-01 10:14:12
|
Happy New Year Everyone! Let's make it a JPEG XL Year 😉
|
|
|
Demiurge
|
2023-01-01 11:08:05
|
The year of JXL
|
|
2023-01-01 11:08:12
|
Efficient image coding
|
|
2023-01-01 11:08:21
|
Needs a new release
|
|
|
Fraetor
|
|
yes, that's what my links said; my question however was if there is a way to easily determine if the *only* thing preventing those apps from showing EXIF metadata from JXL images is the broken/neutered BMFF support in exiv2, or if there is more that needs to be reported on those apps, and how to test that without essentially having to know how to compile exiv2 and overwrite the system's version (and hopefully not need to recompile all apps with it?)...
(<@456226577798135808>)
|
|
2023-01-01 11:17:28
|
I think the best way to do that is to do as you suggest, and compile a ISOBMFF supporting version of exiv2 and install it as your system library.
|
|
2023-01-01 11:24:33
|
Most programs should be dynamically linked, so it shouldn't be necessary to recompile them.
|
|
|
|
Demiurge
|
2023-01-02 12:06:25
|
Most programs ought to be statically linked if you ask me... except maybe in situations where it's convenient or makes sense to have dynamic linking..
|
|
2023-01-02 12:07:20
|
Static linking is way simpler and more reliable.
|
|
|
|
Traneptora
|
|
Demiurge
Static linking is way simpler and more reliable.
|
|
2023-01-02 12:14:43
|
requires the same library to be loaded into RAM many times, and requires rebuilding anything that depends on it if a bug gets fixed
|
|
|
|
Demiurge
|
2023-01-02 12:16:08
|
rebuilding dependents is done most of the time anyways and the RAM usage thing is a myth since the entire library does not get loaded into memory, just the parts that are actually being used.
|
|
|
|
Traneptora
|
2023-01-02 12:16:38
|
no it's definitely not done most of the time anyway
|
|
2023-01-02 12:17:03
|
and usually the whole library is required
|
|
2023-01-02 12:17:28
|
the ram usage thing is not at all a myth
|
|
2023-01-02 12:18:30
|
try measuring the ram usage of a statically linked ffmpeg binary. in any given execution most of the encoders and decoders are not used
|
|
2023-01-02 12:18:46
|
but they are still loaded by ld.so into RAM
|
|
|
|
Demiurge
|
2023-01-02 12:21:33
|
In some situations dynamic linking like with ffmpeg makes sense
|
|
2023-01-02 12:21:50
|
And similar things like that
|
|
2023-01-02 12:21:55
|
But most of the time static is better
|
|
|
|
Traneptora
|
2023-01-02 12:22:04
|
no it's not
|
|
2023-01-02 12:22:27
|
try updating zlib and enjoy watching your whole system recompile
|
|
|
|
Demiurge
|
2023-01-02 12:27:35
|
For really big things that are meant to be modular with a lot of third party crap, yeah dynamic linking is more convenient. For a system abstraction and compatibility library like SDL, it makes no sense to use static linking for that.
|
|
2023-01-02 12:27:56
|
But for most everything else nothing wrong with static linking and recompiling
|
|
|
|
Traneptora
|
2023-01-02 12:46:23
|
except the whole recompiling part of it
|
|
2023-01-02 12:46:30
|
and how it doesn't actually accomplish anything
|
|
|
|
OkyDooky
|
2023-01-02 03:14:38
|
aw shucks, it seems to me this is really going to be a problem for adoption on desktop Linux
|
|
2023-01-02 05:32:00
|
received some clarification on the matter, and ugh, this is such a shitshow... https://bugzilla.redhat.com/show_bug.cgi?id=1979565#c29
|
|
|
|
Deleted User
|
|
Jyrki Alakuijala
Happy New Year Everyone! Let's make it a JPEG XL Year 😉
|
|
2023-01-02 09:48:41
|
March is still the target for a v1.0 release, right? ;)
|
|
|
|
Jyrki Alakuijala
|
2023-01-02 11:03:07
|
Good question 🙂
|
|
2023-01-02 11:05:31
|
it will be difficult to prioritize effectively between wasm, quality, fast lossless improvements, 1.0, possibly industry asks, etc.
|
|
2023-01-02 11:06:42
|
I think the main change in 1.0 is the longer term back-porting support for security fixes
|
|
2023-01-02 11:08:13
|
this can be important for the community -- the current security model may require some users to take unrelated changes in addition to the security fixes which can be irritating -- I'm not an expert on this so I may be mistaken here...
|
|
2023-01-02 11:09:18
|
for the jpeg xl team, back porting security fixes may be a small slowdown (think 1 % less efficiency for the team after 1.0)
|
|
|
uis
|
|
Traneptora
but they are still loaded by ld.so into RAM
|
|
2023-01-02 11:18:12
|
No, they are mapped. Or mmaped.
|
|
|
|
Traneptora
|
2023-01-03 12:18:17
|
which effectively loads it
|
|
|
|
OkyDooky
|
2023-01-03 01:44:58
|
I chose Fedora because it's the only somewhat stable leading edge distro that gives me a vanilla GNOME experience every six months, not because I like that they need to adhere by US laws
|
|
2023-01-03 01:45:26
|
I have wanted to find an alternative for many years, and there is none
|
|
|
|
Xiaojing
|
2023-01-03 08:08:56
|
I have met some roadblocks trying to compile Jxl. Right now I am having trouble building and installing highway. May I seek some help?
|
|
2023-01-03 08:09:54
|
Right now when I install highway, it says hwy_test.lib is not there…
|
|
2023-01-03 08:10:17
|
Yes
|
|
2023-01-03 08:15:11
|
I think the files are saved in debug, but the program kept searching in release
|
|
2023-01-03 08:21:03
|
Thank you! I will check this out
|
|
|
zamfofex
|
2023-01-03 09:30:04
|
Hello, everyone! I have recently been investing time learning more about JPEG XL, because I found it amazing! ✨
I don’t know if this is the right channel for it (sorry if it isn’t!), but I made a browser extension that uses jxl.js to display JPEG XL images on arbitrary websites.
I know it is kind of a useless thing while there isn’t enough adoption for it by websites (and that once there is, it’ll be obsoleted because by then browsers will have had to have implemented it themselves), but I worked on it mostly just for fun!
In case anyone thinks it sounds interesting, I can share a link to it. (It’s on the Chrome webstore, and the review is pending for Firefox’s addon website, since Mozilla is a bit more thorough about acceptance.)
I have been experimenting using j40, and compiling it to Wasm was effectively trivial, but it seems to be nonfunctional then, failing to work on every image I’ve tried. (I can share the work I have done regarding it insofar if anyone feels like they might be able to help me debug it.)
|
|
2023-01-03 10:26:15
|
It seems Mozilla just recently approved my extension! 🎉 It would be nice if jxl.js didn’t load the fallback images on `<picture>` elements, though, I feel.
|
|
|
|
Jyrki Alakuijala
|
|
Xiaojing
I have met some roadblocks trying to compile Jxl. Right now I am having trouble building and installing highway. May I seek some help?
|
|
2023-01-03 01:26:38
|
please make the problems know to the highway developer Jan -- github.bom/google/highway, file an issue there
|
|
|
|
OkyDooky
|
2023-01-03 03:10:58
|
<@171381182640947200>\: you should definitely publicize it on the r/Firefox subreddit (heck, on the Chrome/Chromium-related ones too?) once it gets approved on the Mozilla add-ons website, too, and suggest people try it out and install it to show demand; you'd probably get a lot of people willing to participate like that, and I suspect this might help JXL's market demand story
|
|
2023-01-03 03:11:59
|
Thanks for the experiment! 🙂 There seems to be a dependency problem though\:```
$ sudo dnf install exiv2-0.27.5-4.1.x86_64.rpm
Problem: conflicting requests
nothing provides exiv2-libs(x86-64) = 0.27.5-4.1 needed by exiv2-0.27.5-4.1.x86_64
```
(<@456226577798135808>)
|
|
|
Peter Samuels
|
2023-01-04 03:08:57
|
stands for bullshit
|
|
|
|
yurume
|
2023-01-04 03:16:18
|
how on earth is BMFF proper considered potentially patent-encumbered
|
|
|
|
_wb_
|
2023-01-04 04:37:34
|
ISOBMFF is basically just the same container mechanism as QuickTime (.mov) and it is very simple and similar to PNG chunks but without the CRC. Just a uint32 box size, then a 4-byte box name, then payload.
|
|
2023-01-04 04:42:54
|
QuickTime is from 1991, so if there are patent concerns: it's more than 20 years old so any patents would be expired anyway. Though I don't really understand how you could patent something as simple as such a container mechanism — and if you could, then PNG would be very likely infringing on it too.
|
|
|
|
yurume
|
2023-01-04 04:44:01
|
when was the 64-bit extension to the container introduced?
|
|
|
|
_wb_
|
2023-01-04 04:44:22
|
Maybe the confusion arose because ISOBMFF containers _can_ be used for patent-encumbered payloads like HEVC. But a tool like exiv2 wouldn't need to look at such payloads anyway...
|
|
|
yurume
when was the 64-bit extension to the container introduced?
|
|
2023-01-04 04:47:21
|
Probably later since in 1991 it would be quite hard to imagine the need for it, but I dunno. Anyway I don't think there's anything patented/patentable in ISOBMFF. For HEIF, Nokia does claim patents though...
|
|
|
|
Quikee
|
2023-01-04 04:52:56
|
LOL - so the whole issue is that exiv2 has a warning that there might be patents for ISOBMFF, which is not true and Fedora doesn't like that warning.
|
|
|
|
yurume
|
2023-01-04 04:58:36
|
I agree to quikee, exiv2 should revise that wording
|
|
|
|
_wb_
|
2023-01-04 05:03:09
|
It's in any case not useful to have warnings like that. Absolutely everything "might be patent encumbered", since you never know if a patent troll will at some point make ridiculous claims about anything at all. It's better to only have a warning when there are actual known patent issues (like with HEVC), not when there are no known issues (like with PNG or JXL) or even when there are known patent trolls but they haven't tried to attack FOSS software implementations yet (like with AVIF).
|
|
|
|
Demiurge
|
|
Traneptora
and how it doesn't actually accomplish anything
|
|
2023-01-04 06:35:51
|
Makes stuff more reliable. Ever had pacman break on you while using Arch?
|
|
2023-01-04 06:36:05
|
Pacman really ought to be statically linked
|
|
2023-01-04 10:45:13
|
I mean broken as in, an incomplete upgrade involving pacman and its dependencies. This happened to me a few weeks ago and even using the arch install iso I could not fix it with pacstrap
|
|
2023-01-04 10:46:23
|
Since the pacman binary is not statically linked, if any of its dependencies are the wrong version then pacman becomes very very very broken
|
|
2023-01-04 10:49:32
|
To clarify I was able to fix it, but not even pacstrap worked
|
|
2023-01-04 10:51:04
|
It was a few weeks ago but it was a crypto lib
|
|
2023-01-04 10:51:07
|
probably gpg
|
|
2023-01-04 10:51:33
|
Because pacman could not dynamically link all of its dependencies on startup pacman was completely inoperabel
|
|
|
novomesk
|
2023-01-04 12:33:54
|
Fedora seems to have libavif and libjxl already. I am curious how the lawyer is able to explain, that parsing metadata via libavif and libjxl is OK, but same functionality in EXIV2_ENABLE_BMFF enabled exiv2 is suddenly not OK.
It is inconsistency of decisions.
Since libavif and libjxl are already there, these are precedent decisions (to use languare of lawyers).
|
|
2023-01-04 12:53:38
|
I hope they will not remove libavif and libjxl in order to reach full consistency of decisions...
|
|
|
|
Traneptora
|
2023-01-04 01:18:30
|
The only reason this is blocked on legal is that exiv2 has a line of text in its documentation that says "usage of this may be patent-encumbered"
|
|
2023-01-04 01:18:48
|
so they booted it to the legal team which refuses to actually give a straight answer
|
|
|
BlueSwordM
|
|
novomesk
Fedora seems to have libavif and libjxl already. I am curious how the lawyer is able to explain, that parsing metadata via libavif and libjxl is OK, but same functionality in EXIV2_ENABLE_BMFF enabled exiv2 is suddenly not OK.
It is inconsistency of decisions.
Since libavif and libjxl are already there, these are precedent decisions (to use languare of lawyers).
|
|
2023-01-04 04:45:51
|
Even better: AVIF uses HEIF as a base container...
|
|
|
|
_wb_
|
2023-01-04 04:48:26
|
libavif is certainly at least as much a "patent risk" as exiv2-with-isobmff-parsing
|
|
2023-01-04 04:50:06
|
Does fedora have ffmpeg?
|
|
|
Sam
|
2023-01-04 06:26:10
|
Has there been any though to making a Chromium fork (or another browser fork) that is compatible with JPEG XL?
It could also open a door to some other interesting browser tweaks (e.g. LibRedirect https://github.com/libredirect by default, HTMX https://htmx.org/ compatibility?) And yet, at it's simplest, it could start with a simple browser fork that is compatible with JPEG XL?
|
|
|
|
OkyDooky
|
2023-01-04 07:31:38
|
<@455316071323402242>\:
On some forums, I've included the suggestion to encourage the developers of Brave and Vivaldi to maintain their own implementations of JXL support, since that shouldn't be much more than what they're already doing to modify Chromium. Pale Moon is, so far, the only browser to have support fully enabled by default, but another Firefox browser could maybe do so, as well.
Honestly, encouraging Brave/Vivaldi devs is good and should be done, but encouraging Mozilla to fully support JXL will be the best move, since it'll either put Firefox out ahead or will force Google's hand in this.
(And I definitely agree with you on having LibRedirect's functionality prepackaged with a modern browser. Haven't checked out HTMX, though.)
|
|
|
|
Sam
|
|
<@455316071323402242>\:
On some forums, I've included the suggestion to encourage the developers of Brave and Vivaldi to maintain their own implementations of JXL support, since that shouldn't be much more than what they're already doing to modify Chromium. Pale Moon is, so far, the only browser to have support fully enabled by default, but another Firefox browser could maybe do so, as well.
Honestly, encouraging Brave/Vivaldi devs is good and should be done, but encouraging Mozilla to fully support JXL will be the best move, since it'll either put Firefox out ahead or will force Google's hand in this.
(And I definitely agree with you on having LibRedirect's functionality prepackaged with a modern browser. Haven't checked out HTMX, though.)
|
|
2023-01-04 07:34:54
|
This is awesome to hear, thanks so much! One other browser community to encourage adoption — Orion Browser. It's Mac specific (built on WebKit) and it seems to be coming along and implementing / building new features very quickly.
|
|
|
fab
|
2023-01-04 07:42:46
|
Librewolf
|
|
2023-01-04 07:43:18
|
But I doubt someone uses Midori browser of chromium
|
|
2023-01-04 07:43:25
|
And librewolf of Firefox
|
|
2023-01-04 07:45:23
|
I use bromite to view av1 videos on xiaomi mi 11 lite 5g
|
|
2023-01-04 07:45:34
|
I watch 1080p resolution
|
|
2023-01-04 07:45:53
|
No 60 fps or at least I think yes
|
|
2023-01-04 07:46:09
|
But video in 8k no av1 for all resolutions
|
|
|
|
OkyDooky
|
2023-01-04 08:51:25
|
I don't know how much the LibreWolf team will be able to do for this, since I have no idea how complicated it would be, given Mozilla's codebase and that the LibreWolf team is very small.
|
|
2023-01-04 09:00:14
|
As for Bromite, they are very dependent on other patches, so I think they would require someone else independently maintain an Android-compatible Chromium patch for JXL support in order for Bromite to have that past Google's date of destiny.
|
|
2023-01-04 09:02:29
|
Oh, and it if anyone is interested in promoting the format, some brilliant anon made this\:
|
|
2023-01-04 09:03:01
|
JXL\_industry\_support.jpg
|
|
|
|
Traneptora
|
2023-01-04 10:01:33
|
just trying -e10 now
|
|
2023-01-04 10:01:35
|
*wow* it's slow
|
|
2023-01-04 10:01:38
|
<:kek:857018203640561677>
|
|
2023-01-04 10:28:20
|
looks like cjxl -e 10 is only using one thread
|
|
2023-01-04 10:28:26
|
|
|
2023-01-04 10:28:31
|
should be using four
|
|
2023-01-04 10:29:09
|
something about it isn't parallelizing well
|
|
|
|
veluca
|
2023-01-05 12:03:17
|
not much of lossless parallelizes well really
|
|
2023-01-05 12:03:55
|
I probably ought to parallelize over the parameter sweep instead
|
|
2023-01-05 12:04:17
|
let me try that
|
|
2023-01-05 12:16:39
|
https://github.com/libjxl/libjxl/pull/2029
|
|
2023-01-05 12:16:47
|
that's way more parallel
|
|
|
|
Traneptora
|
2023-01-05 12:40:39
|
considering something that took -e9 7 seconds, I was expecting a lot but not 2 hours a lot
|
|
2023-01-05 12:40:59
|
however, I am impressed with the results, 0.423bpp vs 0.791 bpp
|
|
|
|
veluca
|
|
Traneptora
considering something that took -e9 7 seconds, I was expecting a lot but not 2 hours a lot
|
|
2023-01-05 01:43:59
|
honestly it could be done with a lot less cpu time
|
|
2023-01-05 01:44:15
|
if we tried being smart at picking what options to try
|
|
2023-01-05 01:44:39
|
but I didn't want to try to be smart xD
|
|
|
|
sklwmp
|
|
veluca
https://github.com/libjxl/libjxl/pull/2029
|
|
2023-01-05 03:01:40
|
yep, that's definitely a lot better
|
|
2023-01-05 04:51:32
|
nearly 15 minutes for e10 vs. around 5 seconds for e9
|
|
|
|
zamfofex
|
2023-01-05 05:43:45
|
But is the resulting image actually smaller?
|
|
|
|
sklwmp
|
2023-01-05 05:51:52
|
by a little bit, yes
|
|
2023-01-05 05:53:03
|
e10 0.454 bpp (40KB)
e9 0.592 bpp (52KB)
oxipng optimized PNG 0.691 bpp (60KB)
|
|
|
|
yurume
|
2023-01-05 05:58:31
|
e10 doesn't seem to correctly describe the slowness, it should be renamed to e99 😉
|
|
2023-01-05 05:58:39
|
(and the future e11 would be e999...)
|
|
|
|
_wb_
|
2023-01-05 06:09:46
|
It's very much nonlinear anyway. e1 is way more than twice as fast as e2
|
|
|
|
yurume
|
2023-01-05 06:30:52
|
it's true, but e9-e10 gap seems far larger than anything else...
|
|
|
|
zamfofex
|
2023-01-05 06:31:05
|
Should the effort values be given descriptive names of what they do instead? Suppose someone wants to add a value that is faster than e10 but slower than e9.
|
|
2023-01-05 06:32:11
|
It doesn’t feel like using numbers where the numbers’ values aren’t representative of what they do (except for their ordering) is generally good.
|
|
|
|
yurume
|
2023-01-05 06:36:58
|
I think the effort level should be some function of the expected time, not necessarily linear but something akin to that
|
|
2023-01-05 06:37:43
|
with a possible exception of lowest and highest levels
|
|
2023-01-05 06:38:05
|
(which I want to name +/-infinity personally 😉
|
|
|
|
_wb_
|
2023-01-05 06:52:40
|
Perhaps we should have used an effort scale from 0 to 100, that makes it easier to add intermediate efforts
|
|
2023-01-05 06:58:57
|
Then again it's hard to imagine someone would care about using effort 57 vs 56
|
|
|
joppuyo
|
|
<@455316071323402242>\:
On some forums, I've included the suggestion to encourage the developers of Brave and Vivaldi to maintain their own implementations of JXL support, since that shouldn't be much more than what they're already doing to modify Chromium. Pale Moon is, so far, the only browser to have support fully enabled by default, but another Firefox browser could maybe do so, as well.
Honestly, encouraging Brave/Vivaldi devs is good and should be done, but encouraging Mozilla to fully support JXL will be the best move, since it'll either put Firefox out ahead or will force Google's hand in this.
(And I definitely agree with you on having LibRedirect's functionality prepackaged with a modern browser. Haven't checked out HTMX, though.)
|
|
2023-01-05 09:07:24
|
Whether they want to support JXL is probably a question of priorities. Both Brave and Vivaldi have pledged to support manifest v2 which Chrome is deprecating so it’s not unheard of that Chrome forks continue to support features removed from Chrome. But removing this API makes it much more difficult to implement ad blockers so there are millions of users who would benefit from this. With JXL we are talking about just some developers who are eager to implement a new image format
|
|
2023-01-05 09:08:33
|
Well, of course millions of people would benefit from faster loading and better quality images, but it’s harder to justify a potential benefit compared to taking existing features away that extensions are using
|
|
|
|
Demiurge
|
2023-01-05 02:43:02
|
Does JXL add anything over FUIF?
|
|
|
|
veluca
|
2023-01-05 03:11:11
|
see https://discord.com/channels/794206087879852103/804324493420920833/1060572747215949945
|
|
2023-01-05 03:17:59
|
but tldr: much better lossy, and better/faster lossless 🙂
|
|
|
|
afed
|
2023-01-05 05:42:02
|
mpv
> `--screenshot-webp-lossless=<yes|no>`
> Write lossless WebP files. --screenshot-webp-quality is ignored if this is set. The default is no.
> `--screenshot-webp-quality=<0-100>`
> Set the WebP quality level. Higher means better quality. The default is 75.
> `--screenshot-webp-compression=<0-6>`
> Set the WebP compression level. Higher means better compression, but takes more CPU time. Note that this also affects the screenshot quality when used with lossy WebP files. The default is 4.
> `--screenshot-jxl-distance=<0-15>`
> Set the JPEG XL Butteraugli distance. Lower means better quality. Lossless is 0.0, and 1.0 is approximately equivalent to JPEG quality 90 for photographic content. Use 0.1 for "visually lossless" screenshots. The default is 1.0.
> `--screenshot-jxl-effort=<1-9>`
> Set the JPEG XL compression effort. Higher effort (usually) means better compression, but takes more CPU time. The default is 3.
I wonder what lossless mode would be optimal for screenshots for mostly 4:2:0 lossy videos, -e 3 is good for photos, that is close to frames from movies, but maybe for some animation it is not quite the same
-e 3 for lossless has good compression and is relatively fast, but still not instantly fast (or even slow on weaker cpu)
but -e 3 for lossy seems too low when most of the jxl features are not used and something like -e 4/5 would be better and still fast
it seems that for something which needs very fast encoding speed, like making screenshots, it would be good to have different default efforts for lossy and lossless <:Thonk:805904896879493180>
|
|
|
|
Traneptora
|
|
afed
mpv
> `--screenshot-webp-lossless=<yes|no>`
> Write lossless WebP files. --screenshot-webp-quality is ignored if this is set. The default is no.
> `--screenshot-webp-quality=<0-100>`
> Set the WebP quality level. Higher means better quality. The default is 75.
> `--screenshot-webp-compression=<0-6>`
> Set the WebP compression level. Higher means better compression, but takes more CPU time. Note that this also affects the screenshot quality when used with lossy WebP files. The default is 4.
> `--screenshot-jxl-distance=<0-15>`
> Set the JPEG XL Butteraugli distance. Lower means better quality. Lossless is 0.0, and 1.0 is approximately equivalent to JPEG quality 90 for photographic content. Use 0.1 for "visually lossless" screenshots. The default is 1.0.
> `--screenshot-jxl-effort=<1-9>`
> Set the JPEG XL compression effort. Higher effort (usually) means better compression, but takes more CPU time. The default is 3.
I wonder what lossless mode would be optimal for screenshots for mostly 4:2:0 lossy videos, -e 3 is good for photos, that is close to frames from movies, but maybe for some animation it is not quite the same
-e 3 for lossless has good compression and is relatively fast, but still not instantly fast (or even slow on weaker cpu)
but -e 3 for lossy seems too low when most of the jxl features are not used and something like -e 4/5 would be better and still fast
it seems that for something which needs very fast encoding speed, like making screenshots, it would be good to have different default efforts for lossy and lossless <:Thonk:805904896879493180>
|
|
2023-01-05 05:53:16
|
I chose e3 as the default because I consider fast screenshots a high priority
|
|
2023-01-05 05:53:32
|
I didn't really poll users first tho
|
|
|
|
_wb_
|
2023-01-05 05:53:58
|
I think e1 would be good for screenshots
|
|
|
|
Traneptora
|
2023-01-05 05:54:03
|
For lossless I recommend e1, which is extremely fast
|
|
2023-01-05 05:54:22
|
Screenshots here are frame extractions from video, btw
|
|
|
|
_wb_
|
2023-01-05 05:54:24
|
Probably even does better than e3 for screen content
|
|
|
|
Traneptora
|
2023-01-05 05:54:37
|
mpv is a media player
|
|
|
|
_wb_
|
2023-01-05 05:54:48
|
For video screenshots specifically, oh, hm
|
|
2023-01-05 05:55:11
|
We should get an api to do lossless yuv420 encoding
|
|
2023-01-05 05:55:33
|
It can be done, we just don't have a way to pass a yuv420 buffer
|
|
2023-01-05 05:56:28
|
If the content is actual yuv420, it will compress better keeping it that way than compressing it as rgb
|
|
|
|
Traneptora
|
|
afed
mpv
> `--screenshot-webp-lossless=<yes|no>`
> Write lossless WebP files. --screenshot-webp-quality is ignored if this is set. The default is no.
> `--screenshot-webp-quality=<0-100>`
> Set the WebP quality level. Higher means better quality. The default is 75.
> `--screenshot-webp-compression=<0-6>`
> Set the WebP compression level. Higher means better compression, but takes more CPU time. Note that this also affects the screenshot quality when used with lossy WebP files. The default is 4.
> `--screenshot-jxl-distance=<0-15>`
> Set the JPEG XL Butteraugli distance. Lower means better quality. Lossless is 0.0, and 1.0 is approximately equivalent to JPEG quality 90 for photographic content. Use 0.1 for "visually lossless" screenshots. The default is 1.0.
> `--screenshot-jxl-effort=<1-9>`
> Set the JPEG XL compression effort. Higher effort (usually) means better compression, but takes more CPU time. The default is 3.
I wonder what lossless mode would be optimal for screenshots for mostly 4:2:0 lossy videos, -e 3 is good for photos, that is close to frames from movies, but maybe for some animation it is not quite the same
-e 3 for lossless has good compression and is relatively fast, but still not instantly fast (or even slow on weaker cpu)
but -e 3 for lossy seems too low when most of the jxl features are not used and something like -e 4/5 would be better and still fast
it seems that for something which needs very fast encoding speed, like making screenshots, it would be good to have different default efforts for lossy and lossless <:Thonk:805904896879493180>
|
|
2023-01-05 05:57:30
|
also default distance is 1, I'd expect a user who fiddles with that to also fiddle with speed
|
|
2023-01-05 05:58:12
|
separate defaults based on the value of other options is kind of hard to do based on mpv current option parser
|
|
|
|
afed
|
|
_wb_
We should get an api to do lossless yuv420 encoding
|
|
2023-01-05 06:00:31
|
yeah, that would be great
|
|
|
Traneptora
separate defaults based on the value of other options is kind of hard to do based on mpv current option parser
|
|
2023-01-05 06:00:37
|
yeah, I haven't think of a good solution either
|
|
|
|
Traneptora
|
|
_wb_
We should get an api to do lossless yuv420 encoding
|
|
2023-01-05 06:01:21
|
what about just accepting yuv420 buffer in general? or for lossy would that be iffy
|
|
2023-01-05 06:01:32
|
since it couldn't do XYB without converting to RGB first
|
|
2023-01-05 06:02:14
|
actually tbh yuv420 lossy doesn't seem like you'd gain anything compared to just turning it into RGB first
|
|
2023-01-05 06:02:21
|
since XYB is 4:4:4 only
|
|
|
|
afed
|
2023-01-05 06:03:52
|
also screenshots are always saved at more than 8bpc when screenshot-high-bit-depth=yes, even if it is not hdr and not 10 bit video, but I guess it depends on filters and output settings
|
|
|
|
Traneptora
|
2023-01-05 06:04:45
|
`screenshot-high-bit-depth=yes` saving screenshots tagged as 16-bit for 8-bit video depends a bit on the other options
|
|
2023-01-05 06:04:51
|
like `screenshot-sw`
|
|
|
|
_wb_
|
|
Traneptora
actually tbh yuv420 lossy doesn't seem like you'd gain anything compared to just turning it into RGB first
|
|
2023-01-05 06:37:54
|
Yeah, unless maybe if it's really high quality, then doing actual yuv420 lossy without xyb (which implies using only dct8x8) might be more effective... Say recompressing a q98 jpeg
|
|
|
|
afed
|
|
Traneptora
For lossless I recommend e1, which is extremely fast
|
|
2023-01-05 07:59:40
|
it would be cool to use fpnge for fast png encoding mode, but sadly mpv uses mostly what ffmpeg has (and it's not worth adding new dependencies for a separate encoder just for fast mode) <:PepeSad:815718285877444619>
for webp lossless the most optimal is 0, it's very fast, and higher modes don't give much compression increase, with significant speed loss
jxl -e 1-3 better than png 4 (which is optimal for compression) and even -e 3 is faster for me, with high-bitepth the difference is even bigger
|
|
|
|
Traneptora
|
|
afed
it would be cool to use fpnge for fast png encoding mode, but sadly mpv uses mostly what ffmpeg has (and it's not worth adding new dependencies for a separate encoder just for fast mode) <:PepeSad:815718285877444619>
for webp lossless the most optimal is 0, it's very fast, and higher modes don't give much compression increase, with significant speed loss
jxl -e 1-3 better than png 4 (which is optimal for compression) and even -e 3 is faster for me, with high-bitepth the difference is even bigger
|
|
2023-01-05 08:24:27
|
you could write a lua script to output screenshots with lossless jxl -e1 or lossless png -e0 and then asynchronously recompress them
|
|
2023-01-05 08:24:35
|
I did that at one point but ended up deciding it wasn't worth
|
|
|
|
afed
|
2023-01-05 09:16:55
|
yeah, recompression is also possible, but I'm more about instant or almost instant screenshots saving, with still good compression and without extra work (especially not everyone will want to use it, but lags after making screenshots or bloated sizes are not satisfied)
|
|
2023-01-05 11:54:19
|
also if jxl will accept yuv420, mpv needs to be able transfer it without conversion <:FeelsReadingMan:808827102278451241>
```Screenshots are currently always RGB. Subject to change, but needs to be
communicated clearly if changed. This commit is not a functional change,
it's merely for code clarity.```
https://github.com/mpv-player/mpv/pull/10998/commits/4aefd26303e839e068bfc55bd7717e5b09ef9b2b
|
|
|
|
Traneptora
|
|
afed
also if jxl will accept yuv420, mpv needs to be able transfer it without conversion <:FeelsReadingMan:808827102278451241>
```Screenshots are currently always RGB. Subject to change, but needs to be
communicated clearly if changed. This commit is not a functional change,
it's merely for code clarity.```
https://github.com/mpv-player/mpv/pull/10998/commits/4aefd26303e839e068bfc55bd7717e5b09ef9b2b
|
|
2023-01-06 12:20:49
|
yea if it gets the ability to receive 4:2:0 buffers I'd have to update the ffmpeg code to do that
|
|
2023-01-06 12:21:02
|
mpv might get it automatically if the libjxl avcodec wrapper declares 4:2:0 as supported
|
|
2023-01-06 12:21:10
|
I'm not sure, it would need testing
|
|
2023-01-06 12:21:42
|
I think it's also an issue with lossy webp which currently only supports VP8's 4:2:0 but libwebp accepts RGB
|
|
|
|
BlueSwordM
|
|
afed
mpv
> `--screenshot-webp-lossless=<yes|no>`
> Write lossless WebP files. --screenshot-webp-quality is ignored if this is set. The default is no.
> `--screenshot-webp-quality=<0-100>`
> Set the WebP quality level. Higher means better quality. The default is 75.
> `--screenshot-webp-compression=<0-6>`
> Set the WebP compression level. Higher means better compression, but takes more CPU time. Note that this also affects the screenshot quality when used with lossy WebP files. The default is 4.
> `--screenshot-jxl-distance=<0-15>`
> Set the JPEG XL Butteraugli distance. Lower means better quality. Lossless is 0.0, and 1.0 is approximately equivalent to JPEG quality 90 for photographic content. Use 0.1 for "visually lossless" screenshots. The default is 1.0.
> `--screenshot-jxl-effort=<1-9>`
> Set the JPEG XL compression effort. Higher effort (usually) means better compression, but takes more CPU time. The default is 3.
I wonder what lossless mode would be optimal for screenshots for mostly 4:2:0 lossy videos, -e 3 is good for photos, that is close to frames from movies, but maybe for some animation it is not quite the same
-e 3 for lossless has good compression and is relatively fast, but still not instantly fast (or even slow on weaker cpu)
but -e 3 for lossy seems too low when most of the jxl features are not used and something like -e 4/5 would be better and still fast
it seems that for something which needs very fast encoding speed, like making screenshots, it would be good to have different default efforts for lossy and lossless <:Thonk:805904896879493180>
|
|
2023-01-06 03:11:00
|
Personally, I just do lossy E3 d 0.1 for screenshots.
|
|
|
|
afed
|
2023-01-06 03:18:50
|
lossless is always true lossless, especially if needed for pixel-hunting comparisons, even d 0.1 is not always artifact-free with zoom
|
|
|
|
sklwmp
|
2023-01-06 01:23:46
|
From <#809126648816336917>: <@274048677851430913> <@1051137277813870592>
https://twitter.com/Kampidh/status/1596107738433859584
|
|
|
Kampidh
|
2023-01-06 01:34:21
|
Oh, yes that one~
|
|
|
|
Jyrki Alakuijala
|
2023-01-06 05:22:59
|
I'd love to learn about opinions related to image quality and robustness of compression after https://github.com/libjxl/libjxl/pull/2024
|
|
2023-01-06 05:23:28
|
if someone has new (or more) worst case images to try, I'd be interested
|
|
2023-01-06 05:23:49
|
also I'm quite interested if the DZgas image is now acceptable at d1 or if more improvement is needed
|
|
|
DZgas Ж
|
2023-01-07 01:00:58
|
<:BlobYay:806132268186861619>
|
|
|
|
fab
|
2023-01-07 01:55:07
|
Increase dct 8x8 at d 1.8
|
|
2023-01-07 01:55:47
|
Too few for my taste
|
|
2023-01-07 01:56:16
|
A little bit even 1%
|
|
|
|
Traneptora
|
2023-01-07 04:02:49
|
I noticed libjxl is spitting out errors on e10 d0 in a debug build
|
|
2023-01-07 04:03:00
|
a lot of stuff like this:
|
|
2023-01-07 04:03:02
|
```
./lib/jxl/fields.cc:730: JXL_FAILURE: No feasible selector for 263325
./lib/jxl/fields.cc:488: JXL_RETURN_IF_ERROR code=1: ok_
./lib/jxl/fields.cc:633: JXL_RETURN_IF_ERROR code=1: CanEncode(fields, &extension_bits, &total_bits)
./lib/jxl/enc_modular.cc:1198: JXL_RETURN_IF_ERROR code=1: Bundle::Write(stream_headers_[stream_id], writer, layer, aux_out)
```
|
|
2023-01-07 04:03:09
|
are these things I should worry about?
|
|
|
|
fab
|
2023-01-07 04:11:08
|
Is e10 also vardct?
|
|
|
|
Traneptora
|
2023-01-07 04:24:56
|
this is -d0 so no
|
|
2023-01-07 05:25:15
|
tho it's a 2 MP image and so far it's used almost 5 hours of cpu time so I'm hoping it actually works <:KEKW:643601031040729099>
|
|
|
yoochan
|
2023-01-07 05:56:48
|
5 hours 😆 almost on par with AV1
|
|
|
|
Traneptora
|
2023-01-07 06:22:59
|
```
./lib/jxl/encode.cc:568: Failed to encode frame
JxlEncoderProcessOutput failed.
EncodeImageJXL() failed.```
failed after 7 hours of CPU time
|
|
2023-01-07 06:23:12
|
I feel like it should fail earlier, or not at all
|
|
|
|
_wb_
|
2023-01-07 06:50:02
|
No feasible selector, that means it's somehow trying to set header fields with values that cannot be represented. That's an encoder bug.
|
|
2023-01-07 06:50:45
|
Does it work at e9?
|
|
|
|
Traneptora
|
|
_wb_
Does it work at e9?
|
|
2023-01-07 06:56:04
|
e9 works fine, yea
|
|
2023-01-07 06:56:22
|
this is just a bug in the e10 exhaustive search
|
|
2023-01-07 06:56:43
|
it would be nice if it either failed fast or if it ignored invalid parameters and just skipped them
|
|
|
|
afed
|
2023-01-07 07:13:47
|
it's an error even without multithreading?
|
|
|
|
_wb_
|
2023-01-07 07:16:12
|
I wonder which combo of e9 parameters is causing the trouble...
|
|
|
|
Traneptora
|
2023-01-07 07:17:25
|
I can't provide the sample as it's a piece of art I did not make
|
|
2023-01-07 07:17:42
|
a friend in a group I'm in made it and I didn't ask permission to post
|
|
2023-01-07 07:17:56
|
but I can try to see if I can find another sample
|
|
|
|
_wb_
|
2023-01-07 07:26:15
|
If you instrument the e10 code to print settings used and the return status for each encode, then maybe you can find a combination of e9 settings that causes trouble?
|
|
2023-01-07 07:27:57
|
It shouldn't happen that some settings cause header field issues, that's a bug.
|
|
|
|
afed
|
2023-01-07 07:34:27
|
knowing what settings are used for encoding would be useful for many other purposes, and having something like this at least in benchmark_xl would be helpful
|
|
|
|
veluca
|
|
Traneptora
```
./lib/jxl/fields.cc:730: JXL_FAILURE: No feasible selector for 263325
./lib/jxl/fields.cc:488: JXL_RETURN_IF_ERROR code=1: ok_
./lib/jxl/fields.cc:633: JXL_RETURN_IF_ERROR code=1: CanEncode(fields, &extension_bits, &total_bits)
./lib/jxl/enc_modular.cc:1198: JXL_RETURN_IF_ERROR code=1: Bundle::Write(stream_headers_[stream_id], writer, layer, aux_out)
```
|
|
2023-01-07 11:22:43
|
it's a bug
|
|
2023-01-07 11:22:51
|
I actually have a patch for it
|
|
2023-01-07 11:24:59
|
https://github.com/libjxl/libjxl/pull/2038
|
|
|
|
Traneptora
|
2023-01-07 11:30:43
|
noice
|
|
|
|
Demiurge
|
2023-01-08 12:41:02
|
I think it would be a bad idea to include -e10 in a release version if it still takes days for it to encode 1 image. Lots of people use -e9 simply because "it's the highest." I think that settings that should almost NEVER be used in practice (like the -m flag), should be placed behind some sort of idiot-warning. Like maybe behind a flag that unlocks idiot functionality like --i-am-an-idiot
|
|
2023-01-08 12:41:36
|
Oh and I guess this should go in <#804324493420920833>
|
|
2023-01-08 12:51:27
|
Otherwise cjxl could have an informative message like "Some settings are provided for purely academic and experimental reasons and should not be used in practice. Use --i-am-an-idiot to override. Try not to hurt yourself."
|
|
2023-01-08 12:52:24
|
Otherwise I worry that people will be writing headlines about how "JXL takes 4 days to encode 1 image."
|
|
2023-01-08 12:53:11
|
If they weren't properly warned, there will be resentment.
|
|
2023-01-08 12:55:07
|
So hopefully by the time 0.8 rolls around it won't be handing out footguns so easy for people to access...
|
|
2023-01-08 12:58:24
|
Not just for CJXL, but I think API usage as well should be considered because a lot of developers are not any wiser either. Like for example I think lossy modular mode, in its current state, should either fail or it should use vardct instead, unless the developer enables idiot mode via a new API.
|
|
2023-01-08 12:59:25
|
Just because I noticed Irfanview has a modular checkbox when saving a JXL
|
|
2023-01-08 12:59:58
|
And I think it can harm the reputation of the format if people are easily handed footguns like that
|
|
2023-01-08 01:00:29
|
without any warning or context of what it is
|
|
2023-01-08 01:02:29
|
People normally assume that if you hand them something they should try it since it must be there for some practical reason and if there are huge caveats there would have been a warning
|
|
2023-01-08 01:02:53
|
People get angry if you hand them a footgun.
|
|
2023-01-08 01:03:44
|
Even if it may be obvious to the people who developed the footgun, it's not obvious to most of the people who get handed one.
|
|
2023-01-08 01:04:05
|
They just feel betrayed.
|
|
2023-01-08 01:05:51
|
Probably won't be much of an issue once lossy modular gets some serious tuning and once e10 gets some very basic speed shortcuts.
|
|
2023-01-08 01:11:06
|
Basically, settings that sabotage or sacrifice the performance of the format (compression, quality or speed) beyond what is sensible or practical, should be blocked at the API level without a new "idiot mode" enabled.
|
|
2023-01-08 01:14:10
|
I think idiot mode could be a great feature and maybe prevent a lot of misunderstandings but maybe I am worrying too much about a hypothetical issue that isn't as serious as I think it is.
|
|
2023-01-08 01:16:16
|
Maybe people are smart enough to realize some things are not the default for a reason.
|
|
2023-01-08 01:17:03
|
But I try not to assume that anyone is smart.
|
|
2023-01-08 01:17:37
|
Because they never are. And yes that includes ourselves too.
|
|
|
Cool Doggo
|
|
Demiurge
Otherwise cjxl could have an informative message like "Some settings are provided for purely academic and experimental reasons and should not be used in practice. Use --i-am-an-idiot to override. Try not to hurt yourself."
|
|
2023-01-08 01:19:05
|
it would make a lot more sense to just have it as a flag instead of an effort in this case, theres no point in having -e 10 if you also need to add another flag to actually use it in the first place
|
|
|
|
Demiurge
|
2023-01-08 01:20:22
|
No, because the idiot mode will be a more general option to enable footgun settings
|
|
2023-01-08 01:22:25
|
It could be used to prevent people from using footguns without explicitly acknowledging that they are being handed a footgun.
|
|
2023-01-08 01:22:58
|
Including the people who currently use settings like -m
|
|
2023-01-08 01:25:10
|
Basically anyone who wants to use a ridiculous setting has to acknowledge, yes, I know I'm shooting myself in the foot and yes that's what I actually want to do.
|
|
2023-01-08 01:25:54
|
Seriously someone needs to stop those crazy people who keep using -m
|
|
2023-01-08 01:27:02
|
idiot mode is the killer feature we need to kill the usage of settings like -m
|
|
2023-01-08 02:01:20
|
JXL_ENC_FRAME_SETTING_I_AM_IDIOT
|
|
2023-01-08 02:01:46
|
That should get programmers' attention :)
|
|
2023-01-08 02:05:22
|
The ironic thing about it is, you have to call yourself an idiot to prove that you aren't an idiot.
|
|
2023-01-08 02:10:31
|
If it still sounds too mean and not silly enough then I guess something appropriately humorless like `JXL_ENC_FRAME_SETTING_DANGER` or `JXL_ENC_FRAME_SETTING_EXPERIMENTAL` will do.
|
|
|
|
afed
|
2023-01-08 03:27:45
|
maybe it would be better to use some combination to fully working -e 10, but i don't think slowness is such a problem
when the first av1 encoders came out they were incredibly slow, even on the faster presets and libaom was the slowest (it got faster later and now there are slower encoders)
but when it was integrated into ffmpeg, the slowest libaom mode was enabled **by default** <:FeelsAmazingMan:808826295768449054>
even now it's very slow, but at that time encoding could take forever, especially for not very fast cpus
many encoders have many parameters that can make efficiency, speed, quality worse if used without understanding what they do, even if they are somehow hidden, people find them on the internet as some magic or best settings
or feel they're getting better and use them as a placebo effect or overthinking without making proper comparisons
|
|
|
|
yurume
|
2023-01-08 04:55:07
|
`--enable-experiments` looks okay. Rust compiler had options like `-Z unstable-options`, which is excluded from the CLI stability guarantee, and given experimental nature of those options, cjxl can do the same
|
|
|
|
BlueSwordM
|
|
afed
maybe it would be better to use some combination to fully working -e 10, but i don't think slowness is such a problem
when the first av1 encoders came out they were incredibly slow, even on the faster presets and libaom was the slowest (it got faster later and now there are slower encoders)
but when it was integrated into ffmpeg, the slowest libaom mode was enabled **by default** <:FeelsAmazingMan:808826295768449054>
even now it's very slow, but at that time encoding could take forever, especially for not very fast cpus
many encoders have many parameters that can make efficiency, speed, quality worse if used without understanding what they do, even if they are somehow hidden, people find them on the internet as some magic or best settings
or feel they're getting better and use them as a placebo effect or overthinking without making proper comparisons
|
|
2023-01-08 05:53:52
|
Not true.
CPU-1 is default in libaom-av1, not CPU-0.
Still decently slow though.
|
|
|
|
afed
|
2023-01-08 06:02:16
|
yeah, it doesn't change the general point, especially for earlier libaom versions
it's faster now, there are also faster cpu's, but still, even now encoding on these presets is very slow
|
|
|
|
Demiurge
|
2023-01-08 06:03:42
|
That's why it's good to remind people that if they are using those settings they are most likely an idiot unless they actually know why they are using them.
|
|
2023-01-08 06:06:07
|
It's easy for people to get some wrong impression and shoot themselves in the foot by using settings that give a really bad tradeoff. It's a nice thing to make sure people know when they are using the encoder the way it was tuned for and when they are using it in an extremely inefficient way that is known to yield extremely bad tradeoffs that could make JXL look bad
|
|
2023-01-08 06:08:34
|
Left to their own devices people for some reason tend to form extremely weird preferences for extremely bad settings if they are given a whole bunch of dangerous knobs they don't understand.
|
|
|
|
afed
|
2023-01-08 06:09:05
|
I think something like svt-av1
> --preset Encoder preset, **presets < 0 are for debugging**. Higher presets means faster encodes, but with a quality tradeoff, default is 10 [-2-13]
would be enough, no need to hide options or make using cli or api more complicated for advanced users
|
|
|
|
Demiurge
|
2023-01-08 06:10:43
|
Basically I just think that certain settings that are absolutely known to be "never recommended in any situation" such as -m (or "this command will take 3 years to complete" would be in a similar boat), should be guarded behind some idiot shield.
|
|
2023-01-08 06:11:46
|
In a way that makes it very clear to whoever is trying to use it, that they are making a very bad choice
|
|
2023-01-08 06:12:45
|
So that you can't proceed without acknowledging that you're doing something you should not normally do
|
|
2023-01-08 06:13:49
|
No no one ever gets the mistaken impression that it's ever a good idea to use -m for example
|
|
|
|
_wb_
|
2023-01-08 07:41:26
|
The problem with "idiot shield" flags is that it can be interpreted as "any other combination of settings will be a good idea".
|
|
2023-01-08 07:43:25
|
I think we should aim to have defaults that make sense. I don't think we should aim to prevent people from using custom settings that may not make much sense.
|
|
2023-01-08 07:54:57
|
Whether or not e10 makes sense depends on a lot of things, most of which libjxl/cjxl cannot know. It's unlikely that e10 will be what you want to do in a typical use case, but I don't think it is necessary to add explicit warnings. If people make the explicit choice to use the slowest available setting and then complain that it's slow, I think it's easy enough for them to understand how to make it faster 🙂
|
|
|
|
yurume
|
|
_wb_
I think we should aim to have defaults that make sense. I don't think we should aim to prevent people from using custom settings that may not make much sense.
|
|
2023-01-08 08:08:42
|
I generally agree, but `-e` is probably one of the most accessible options and something that virtually everyone will try to tweak in addition to `-q`, so some exceptions can be made.
|
|
|
|
_wb_
|
2023-01-08 08:22:05
|
From a ux pov: if I try e10 and it takes ages, then I press ctrl-C and try e9 or e8. It's clear enough that higher effort will be slower. We could add a 'recommended range' in the --help, but I don't really know what would be a good recommendation...
|
|
2023-01-08 08:25:25
|
I mean, default is a sane choice, but exactly how fast or slow is still 'reasonable' depends entirely on your use case. I think e1 can be a very good choice for temporary files in an authoring workflow, while for shipping a tiny 32x32 icon in an app, you might want to use e10.
|
|
|
username
|
2023-01-08 08:36:05
|
having a note somewhere saying something along the lines of that e10 is magnitudes slower then e9 would probably be good although where and how to display that note I don't exactly know
|
|
|
|
DZgas Ж
|
2023-01-08 11:09:40
|
Where and when is brotli used in Jpeg XL
|
|
2023-01-08 11:15:55
|
Is its use caused by the time when the format was created JXL or what? it's just that at the moment I can't understand why ZSTD was not chosen, which at the same speed can compress better
|
|
|
|
sklwmp
|
|
username
having a note somewhere saying something along the lines of that e10 is magnitudes slower then e9 would probably be good although where and how to display that note I don't exactly know
|
|
2023-01-08 11:18:22
|
<:This:805404376658739230> , it's all well and good to expose the options for people who really want to use them, but some people (me included at times) don't really know what the tradeoffs are between settings and don't have the time (or intelligence) to figure it out ourselves
for example, a hypothetical user would not know exactly what they are missing if they just cancel the e10 encode. it might be 2x slower for a 2x file size decrease, for all they know.
so a simple explanation of some of the more common options like -e and their tradeoffs are probably necessary
|
|
2023-01-08 11:18:42
|
although i do agree that options like -e 10 don't *really* need to be put behind an experimental flag, but i'm not opposed to it either way
|
|
|
|
DZgas Ж
|
|
DZgas Ж
Is its use caused by the time when the format was created JXL or what? it's just that at the moment I can't understand why ZSTD was not chosen, which at the same speed can compress better
|
|
2023-01-08 11:28:37
|
the maintainer of one application wrote this as an argument against adding support for JXL (decoder).
|
|
2023-01-08 11:37:12
|
*The container allows the above metadata to be stored either uncompressed (e.g. plaintext XML in the case of XMP) or by Brotli-compression. In the latter case, the box type is brob (Brotli-compressed Box) and the first four bytes of the box contents define the actual box type (e.g. xml ) it represents.*
|
|
2023-01-08 11:41:11
|
is that all?
|
|
|
|
_wb_
|
2023-01-08 11:50:11
|
Yes, it's quite simple
|
|
2023-01-08 11:51:37
|
Did anyone do benchmarks showing zstd performs better than brotli for XMP metadata, which is typically a short snippet of xml?
|
|
|
|
DZgas Ж
|
2023-01-08 12:22:59
|
<:Thonk:805904896879493180>
|
|
2023-01-08 12:25:46
|
index.html 2028 b
index zstd-22.7z 1364 b
index brotl-11.7z 1159 b
|
|
2023-01-08 12:25:49
|
oh well...
|
|
|
|
yurume
|
|
DZgas Ж
Is its use caused by the time when the format was created JXL or what? it's just that at the moment I can't understand why ZSTD was not chosen, which at the same speed can compress better
|
|
2023-01-08 01:33:08
|
to my knowledge, Brotli excels at web-native data (especially smaller ones) while zstandard excels at more general data. as Brotli in JPEG XL is used for mostly XML data and likes, it looks like that the use of Brotli can be justified.
|
|
|
|
Demiurge
|
|
_wb_
The problem with "idiot shield" flags is that it can be interpreted as "any other combination of settings will be a good idea".
|
|
2023-01-08 01:33:25
|
That's true. But there aren't a whole bunch of knobs, and we know what the recommended ranges are, and the recommended range for effort depends on whether you're using lossless or lossy mode. In current libjxl some settings are never a good idea to enable such as -m and I don't think there is anything wrong with requiring an extra step as a way of warning the user and making the user acknowledge that they have been informed. I believe it's a nice way to give the user fair warning and prevent people from shooting themselves accidentally, which is way too easy and common. I also think that -e10 in its current state, until it's further optimized, takes such an exceptionally long time that it basically amounts to a potential DOS and it's probably a poor tradeoff of time.
|
|
|
|
yurume
|
2023-01-08 01:33:30
|
(also JPEG XL itself shares a fair bit of algorithm with Brotli, but nonetheless)
|
|
|
|
Demiurge
|
|
_wb_
From a ux pov: if I try e10 and it takes ages, then I press ctrl-C and try e9 or e8. It's clear enough that higher effort will be slower. We could add a 'recommended range' in the --help, but I don't really know what would be a good recommendation...
|
|
2023-01-08 01:36:55
|
You can't press ctrl C if the encoder is being used by another frontend program that doesn't give you a progress bar or a way to cancel
|
|
|
username
having a note somewhere saying something along the lines of that e10 is magnitudes slower then e9 would probably be good although where and how to display that note I don't exactly know
|
|
2023-01-08 01:40:03
|
My suggestion is to have an extra setting be introduced that basically says "I acknowledge I know what I'm doing is almost always a bad idea"
|
|
2023-01-08 01:41:56
|
And that setting must be enabled (both for API users and cjxl users) to use some of the most useless settings that everyone can all agree should almost never justifiably be used in close to 100% of situations.
|
|
2023-01-08 01:44:16
|
And it can change depending on the state of current libjxl for example if some settings get improved to the point where they become actually realistically practical to use conceivably, then they can be unshielded
|
|
|
DZgas Ж
index.html 2028 b
index zstd-22.7z 1364 b
index brotl-11.7z 1159 b
|
|
2023-01-08 01:47:34
|
lol, so someone is arguing against adding JXL decode support because they have a problem with the format that they are wrong about and don't understand...
|
|
|
|
DZgas Ж
|
2023-01-08 01:48:20
|
stupid people exist
|
|
2023-01-08 01:48:34
|
😱
|
|
|
|
Demiurge
|
2023-01-08 01:48:42
|
This is the bad UX I am worried about that might potentially generate bad press or general resentment.
|
|
2023-01-08 01:50:56
|
I think that the most practical and useful and most pareto-ideal tools should be easy to access. And tools that are known to be not useful at all, or could potentially create a very bad user experience, should be kept separate, in a place where people will not accidentally reach for them without knowing first what they are reaching for!
|
|
2023-01-08 01:56:22
|
You don't throw stumbling blocks in front of a blind man and you organize your tools and you keep the well sharpened tools separate from the rusty knives, and you don't leave dynamite lying around.
|
|
2023-01-08 01:57:25
|
You put dynamite in a safe place with a warning on the box saying DANGER: EXPLOSIVES!
|
|
2023-01-08 01:58:35
|
It's the same principle of keeping your tools organized and separated
|
|
|
|
zamfofex
|
2023-01-08 02:15:43
|
I still feel like the effort settings should be given meaningful names rather than just numeric values. `-e 10` could then be named e.g. `--brute-force` or `--effort=brute-force`
|
|
2023-01-08 02:16:44
|
I don’t know what characterizes each other effort level, so I can’t suggest names for them, but I feel like it should be something descriptive.
|
|
|
|
_wb_
|
2023-01-08 02:26:23
|
For a gimp plugin, probably it's not a good idea to expose settings that are likely to be 'dangerous', like e10.
|
|
2023-01-08 02:26:54
|
Also we should really make it handle OOM better, because currently it can probably crash the whole thing
|
|
|
|
username
|
|
I still feel like the effort settings should be given meaningful names rather than just numeric values. `-e 10` could then be named e.g. `--brute-force` or `--effort=brute-force`
|
|
2023-01-08 02:27:20
|
they interally have names however I think going from 1 to 10 is a lot easier to understand and eaiser to type out
|
|
|
|
_wb_
|
2023-01-08 02:28:39
|
Or just have only e1 ("fast") and e7
|
|
|
|
zamfofex
|
|
username
they interally have names however I think going from 1 to 10 is a lot easier to understand and eaiser to type out
|
|
2023-01-08 02:29:22
|
It feels a bit strange to me that the numbers don’t actually represent anything about how each of them work.
|
|
|
|
JendaLinda
|
2023-01-08 02:30:12
|
-e 10 doesn't seem to be practical, however -e 9 was failing on some types of content and was beaten by webp. I think I would select a few custom settings and use those. It just needs some wrapper that would do the trials automatically and choose the best one.
|
|
|
|
veluca
|
|
JendaLinda
-e 10 doesn't seem to be practical, however -e 9 was failing on some types of content and was beaten by webp. I think I would select a few custom settings and use those. It just needs some wrapper that would do the trials automatically and choose the best one.
|
|
2023-01-08 02:32:51
|
(that's e10)
|
|
2023-01-08 02:33:07
|
(possibly with too many settings :P)
|
|
|
|
JendaLinda
|
2023-01-08 02:33:38
|
I know, I would try just a few that are most likely to be successful.
|
|
2023-01-08 02:37:27
|
Anyway, if -e 9 was guaranteed to be always better than webp, I wouldn't bother with trials.
|
|
|
|
Traneptora
|
|
_wb_
From a ux pov: if I try e10 and it takes ages, then I press ctrl-C and try e9 or e8. It's clear enough that higher effort will be slower. We could add a 'recommended range' in the --help, but I don't really know what would be a good recommendation...
|
|
2023-01-08 02:41:32
|
IMO it should say in --help that `e10` is not recommended as it's orders of magnitude slower than e9.
|
|
2023-01-08 02:42:04
|
since with many tools, like gzip, you just use -9 for everything as it's actually reasonable
|
|
2023-01-08 02:42:12
|
which is also true about -e9 in cjxl, but not -e10
|
|
|
|
Demiurge
|
|
_wb_
For a gimp plugin, probably it's not a good idea to expose settings that are likely to be 'dangerous', like e10.
|
|
2023-01-08 03:45:19
|
You give too much credit to plugin developers...
|
|
2023-01-08 03:46:45
|
Also I am sure e10 will be improved in the near future. Probably 95% of the compression performance can be achieved in less than 5% the amount of time it takes.
|
|
2023-01-08 03:47:32
|
In its current state no attempt was made at all to make it efficient
|
|
2023-01-08 03:47:58
|
That's sure to change as the code matures
|
|
2023-01-08 03:51:03
|
Anyways I'm probably worrying too much about something that isn't really a big deal, but in this early stage of JXL being introduced to the world, I am worried about bad user experience leaving a bad first impression
|
|
2023-01-08 03:51:34
|
And the first impression is what counts
|
|
2023-01-08 03:53:03
|
If there's an easy way to protect people from accidentally or unknowingly using settings they probably don't want to use without acknowledging that they know what they are getting themselves into, I see that only as a plus.
|
|
2023-01-08 03:57:56
|
I think if someone tries to use settings that are known to shoot the user in the foot in some way, it should throw some sort of error condition and either not produce any output, or produce output as if the bad settings are ignored while still being as close as possible to the intent of the rest of the settings.
|
|
2023-01-08 04:02:35
|
So for example if -m is specified then it would produce output as if -m was not specified, while still probably producing an error condition such as JXL_ENC_ERR_NOT_SUPPORTED
|
|
2023-01-08 04:05:00
|
Or if an effort value is outside of the range of currently recommended values (depending on if lossless or lossy mode is being used), it will use the closest recommended value.
|
|
|
|
Cool Doggo
|
|
It feels a bit strange to me that the numbers don’t actually represent anything about how each of them work.
|
|
2023-01-08 05:36:50
|
having them use a name that explains how they work internally would only make it more confusing for no real benefit imo
|
|
2023-01-08 05:37:08
|
it would also be different depending on if your using vardct or modular
|
|
|
|
_wb_
|
2023-01-08 05:48:47
|
There would be no concise name that describes what eX does, and also: what exactly it does might change between versions, since we still need to figure out the best way to get good speed/density trade-offs...
|
|
|
|
OkyDooky
|
2023-01-08 05:55:28
|
your various responses here were interesting, why not post them to https://github.com/Exiv2/exiv2/issues/2398#issuecomment-1369108374 for the exiv2 project's benefit? If the patent warning is not really legit then maybe they could just remove that
(<@794205442175402004>)
|
|
2023-01-08 05:58:52
|
or maybe someone in your team can provide counsel
|
|
|
|
afed
|
|
Traneptora
For lossless I recommend e1, which is extremely fast
|
|
2023-01-09 04:12:08
|
also, e1 = e3 for lossy, so if the default was e1, it would be e1 for lossless and the same as e3 for lossy
though, e3 in my opinion even for lossless is fast enough, especially if there will be further optimizations (but, might be slow for weak cpus or 4-8k+ resolutions)
and for lossy, e4 is almost as fast as e3 but a bit better in quality, e5 is slower but with multi-threading at high quality not much slower or sometimes faster than mozjpeg (and slower at low quality but I don't think low quality will be often used for screenshots)
|
|
|
|
Traneptora
|
|
afed
also, e1 = e3 for lossy, so if the default was e1, it would be e1 for lossless and the same as e3 for lossy
though, e3 in my opinion even for lossless is fast enough, especially if there will be further optimizations (but, might be slow for weak cpus or 4-8k+ resolutions)
and for lossy, e4 is almost as fast as e3 but a bit better in quality, e5 is slower but with multi-threading at high quality not much slower or sometimes faster than mozjpeg (and slower at low quality but I don't think low quality will be often used for screenshots)
|
|
2023-01-09 04:30:52
|
I suppose
|
|
|
|
Jyrki Alakuijala
|
|
yurume
to my knowledge, Brotli excels at web-native data (especially smaller ones) while zstandard excels at more general data. as Brotli in JPEG XL is used for mostly XML data and likes, it looks like that the use of Brotli can be justified.
|
|
2023-01-09 09:29:10
|
brotli can be more effective for streaming -- data comes in order and very little data is shadowed -- zstd comes in blocks and you need to have the whole block to be able to decode it
today, zstd has better lz77 encoders, if those are ported to brotli, it will encode the same speed -- decoding will be slightly faster with zstd
|
|
|
|
yurume
|
2023-01-09 09:30:44
|
fair point, but metadata is rarely decoded in the streaming fashion I think.
|
|
|
|
Jyrki Alakuijala
|
2023-01-09 09:47:32
|
if there is any expensive processing, it can be done during the transmission
|
|
2023-01-09 09:47:47
|
if it can be done in a streaming manner
|
|
2023-01-09 09:48:02
|
like building an xml tree etc
|
|
|
|
afed
|
2023-01-09 02:30:58
|
is `benchmark_xl --codec=jpeg:libjxl:d1.0` the actual jpegli?
for those who can't load .so libs under windows
|
|
|
|
Jyrki Alakuijala
|
2023-01-09 02:34:03
|
Zoltan knows how to exercise it 🙂
|
|
2023-01-09 02:36:22
|
but yes, that does invoke jpegli
|
|
2023-01-09 02:37:03
|
or at least similar computation -- what jpegli could be after some time
|
|
2023-01-09 02:38:15
|
jpegli's goal is to have libjpeg-turbo compatible ABI/API -- which is not yet quite true, some of the computation may be achieved through short-circuiting the computation (not sure if it is still the case)
|
|
2023-01-09 02:39:44
|
also, it can be that jpeg:libjxl:d1.0 computes into a 16 bit frame buffer, whereas a usual ABI-compatible jpegli might compute into an 8 bit (not sure if this is the case)
|
|
2023-01-09 02:40:36
|
in any case, --codec=jpeg:libjxl:d1.0 should produce really compact nice-looking JPEGs
|
|
2023-01-09 02:41:09
|
you may need to add something like --save_compressed to get the converted images
|
|
|
|
_wb_
|
2023-01-09 02:43:16
|
also most of the time will be spent on computing butteraugli 🙂
|
|
|
|
afed
|
2023-01-09 02:45:21
|
yeah, that's just a part of the command line
example with unsplash jpeg, it looks like jpegli, but I'm more about if this is the current version or some other
|
|
2023-01-09 02:51:09
|
would be nice to have like BPP*SSIMULACRA2 also in benchmark_xl or a specific size encoding, for faster and easier visual/metrics comparison
|
|
|
|
_wb_
|
2023-01-09 02:54:49
|
ssimulacra2 is higher-is-better, so maybe better ssimulacra2/bpp or something (which would then also be higher-is-better).
|
|
|
|
Jyrki Alakuijala
|
2023-01-09 02:55:52
|
BPP*SSIMULACRA2 doesn't make sense, since it doesn't attempt to stay constant over qualities -- the same for BPP/SSIMULACRA2 -- div-by-zero danger there and flip the sign, too
|
|
2023-01-09 02:56:33
|
SSIMULACRA2 attempts to replicate a 'quality' value mapping to original jpeg1 quality setting experience (see below Jon explains how it really works)
|
|
|
|
_wb_
|
2023-01-09 02:56:58
|
we should also add columns for stdev(butteraugli) and stdev(ssimulacra2), when benchmarking a large enough corpus this is a relevant measure of encoder consistency
|
|
|
|
Jyrki Alakuijala
|
2023-01-09 02:56:59
|
the scoring would need to be re-engineered into some 'rawSSIMULACRA2' value for BPP*SSIMULACRA2 to make sense
|
|
2023-01-09 02:58:15
|
stddev punishes good behaviour as well as bad, better to use a high norm for aggregation (for higher-is-worse values)
|
|
|
|
_wb_
|
|
Jyrki Alakuijala
SSIMULACRA2 attempts to replicate a 'quality' value mapping to original jpeg1 quality setting experience (see below Jon explains how it really works)
|
|
2023-01-09 02:58:28
|
it's not original jpeg1 quality setting, it maps directly to MOS scores where 10 = "very low", 30 = "low", 50 = "medium", 70 = "high", 90 = "very high"
|
|
|
|
Jyrki Alakuijala
|
2023-01-09 02:58:46
|
thanks
|
|
|
|
afed
|
2023-01-09 02:59:32
|
yeah, I mean something to quick metric quality estimate taking into account the encoding size
|
|
|
|
Jyrki Alakuijala
|
2023-01-09 03:00:02
|
you could normalize it before the MOS-mapping inside SSIMULACRA2
|
|
|
|
_wb_
|
2023-01-09 03:00:15
|
the average MOS score of the original image itself (compared to itself but presented as a 'distorted' image) was 88.9, so you can take that as the score needed to be 'visually lossless'
|
|
|
Fox Wizard
|
|
Jyrki Alakuijala
you may need to add something like --save_compressed to get the converted images
|
|
2023-01-09 03:00:43
|
For some reason this hasn't been working for me, no matter what I used as input. Like when I use this it just doesn't output anything: ``benchmark_xl --input=C:\Users\Damien\Downloads\LosslesslOriginal.png --codec=jpeg:libjxl:d1.0 --save_compressed --output_dir=C:\Users\Damien\Downloads``
|
|
|
|
Jyrki Alakuijala
|
2023-01-09 03:01:11
|
perhaps try more --save options at the same time
|
|
|
|
veluca
|
2023-01-09 03:01:19
|
tbf we never used it on windows so it might be utterly broken
|
|
|
|
afed
|
2023-01-09 03:01:42
|
`--output_dir=.` works for me
|
|
|
|
_wb_
|
2023-01-09 03:01:42
|
the average MOS score of mozjpeg q30 was 36, that of mozjpeg q50 was 55, mozjpeg q70 was 70 and mozjpeg q90 was 87
|
|
|
|
Jyrki Alakuijala
|
2023-01-09 03:02:23
|
perhaps try with --output_dir=C:\Users\Damien\Downloads\
|
|
|
|
Fox Wizard
|
2023-01-09 03:02:58
|
Weird thing, it can create folders if I for example do ``--output_dir=C:\Users\Damien\Downloads\result``
|
|
2023-01-09 03:03:05
|
But then the folder is just empty
|
|
|
|
_wb_
|
2023-01-09 03:05:55
|
so ssimulacra2 does kind of more or less match the _average_ quality of corresponding jpeg qualities, but the stdev of MOS scores mozjpeg produces is around 7, so e.g. mozjpeg q70 does have a MOS of 70 on average, but about 1 image in 6 will have a MOS below 63 and another 1 in 6 images will have a MOS above 77, and only two thirds of the images are within the 70+-7 interval, more or less
|
|
|
|
veluca
|
|
Demiurge
If there's an easy way to protect people from accidentally or unknowingly using settings they probably don't want to use without acknowledging that they know what they are getting themselves into, I see that only as a plus.
|
|
2023-01-09 04:37:13
|
how about https://github.com/libjxl/libjxl/pull/2041
|
|
|
|
Demiurge
|
2023-01-09 04:48:11
|
Huh, wow, I'm pretty surprised you went that far! Looking it over, I think it looks really good, too.
|
|
2023-01-09 04:49:31
|
I'm honored you think the idea is good enough to write this good implementation.
|
|
|
|
veluca
|
2023-01-09 04:53:17
|
you did raise a very valid point 🙂
|
|
|
|
afed
|
2023-01-09 05:10:22
|
in the future I would prefer something like `--faster_decoding=0|1|2|3|4` but for encoding and also for slower encoding, like for each effort to use more choices and slower (or faster) methods that are in between efforts and so -e 10 would work with full brute force or maybe not much slower than -e 9 by default (so there would be no need to hide it), also for others, like -e 1 could use some effort values from fjxl
|
|
2023-01-09 05:32:18
|
suppose `--faster_encoding=-1|0|1` or `--faster_encoding=-3|-2|-1|0|1|2|3` and `-e 10 --faster_encoding=-3` would be the current `-e 10` (or something even slower), but `-e 10` would be slower than `-e 9` by default but acceptably slower
`-e 1 --faster_encoding=3` would be fjxl 0 (or something faster), etc.
this would add much more versatility and --faster_encoding is also a kind of protection against unintended use
and will give more choices for effort values, but will not change anything for non-advanced users or api use, instead of making something like effort 1-100
|
|
2023-01-09 06:36:50
|
having a special flag only for -e 10 and also having a different maximum effort just brings more confusion, in my opinion
|
|
|
|
Demiurge
|
2023-01-09 06:39:43
|
Future patches might change the behavior of some other settings as well, though.
|
|
2023-01-09 06:40:32
|
Basically it's only meant to prevent people from accidentally using settings that aren't "ready" yet.
|
|
2023-01-09 06:41:36
|
Like after -e 10 gets a few shortcuts to increase its speed where it's a more realistic setting that isn't going to scare people new to JXL, it no longer needs protecting then
|
|
2023-01-09 06:43:18
|
Other experimental settings that are good for testing but shouldn't be used by people who don't know what they're dipping their toes into, can also be protected in the future with this new feature.
|
|
2023-01-09 06:46:24
|
For example the behavior of the JXL_ENC_FRAME_SETTING_MODULAR setting could be protected with that flag
|
|
2023-01-09 06:47:34
|
since at the moment, there is no reason to ever force modular mode aside from testing
|
|
2023-01-09 06:50:32
|
It's a way for encoder developers to put safety glass on certain knobs so that newcomers testing out JXL for the first time have easy access to the most useful and practical settings while keeping the rusty knives and dynamite in their own separate cabinet with warning tape on them.
|
|
|
|
_wb_
|
2023-01-09 07:08:22
|
We should probably put most of the 'advanced' cjxl settings (the ones not visible in the default --help) behind that flag
|
|
2023-01-09 07:12:56
|
The end goal is to have a "set it and forget it" encoder where default settings make sense and the only things to configure are:
- fidelity target (quality)
- encode effort (speed)
And possibly extra progressiveness or faster decoding. Any other settings shouldn't be something a typical end-user should know/care about.
|
|
|
|
afed
|
2023-01-09 07:27:42
|
i dont think this is really needed, some video encoders have dozens and hundreds of settings without special flags, maybe just warnings for some useful only for developers
but, effort is one of the main options and different values create some confusion, I agree that currently -e 10 is not intended for normal use, but it would be better to use combination of more universal flag to enable full bruteforce (like in my example), and by default choose only from some of the most impactful settings
|
|
|
|
_wb_
|
2023-01-09 07:30:54
|
tbh I don't think e9 vardct is useful in normal use either - it's not really better than e8, just a lot slower
|
|
|
|
Demiurge
|
2023-01-09 07:38:13
|
yeah, but a lot of people will automatically reach for it because it's there and might as well use it if it's there and there's nowhere other than here where you can find advice to the contrary
|
|
|
_wb_
The end goal is to have a "set it and forget it" encoder where default settings make sense and the only things to configure are:
- fidelity target (quality)
- encode effort (speed)
And possibly extra progressiveness or faster decoding. Any other settings shouldn't be something a typical end-user should know/care about.
|
|
2023-01-09 07:41:56
|
I like and agree with this idea a lot and I think that's the best way for people learning about and trying out JXL for the first time
|
|
2023-01-09 07:47:43
|
All the tools people expect in an easy to reach place, and not mixed in with the rusty knives and other unexpected things.
|
|
|
|
afed
|
2023-01-09 07:52:21
|
modular for lossy as a basic in some gui is not a good idea
but for cli, using most of the settings only through the flag is also an unnecessary complication, just showing the most basic options in the starting help is enough
|
|
|
|
Demiurge
|
2023-01-09 07:54:41
|
It would be pretty easy to make the cjxl command line tool always have the advanced options enabled, but there is very little value in that isn't there?
|
|
2023-01-09 07:55:21
|
They would be settings that would almost never be appropriate to ever use.
|
|
2023-01-09 07:57:55
|
The whole reason of protecting them behind the flag in the first place is because you almost certainly do NOT want to use them in their current form
|
|
2023-01-09 07:59:58
|
-e 10 is really borderline because -e 10 actually is conceivably useful.
|
|
2023-01-09 08:01:01
|
Especially once -e 10 gets faster
|
|
2023-01-09 08:03:06
|
It would be more useful for options that are not useful in any situation and just pointlessly reduce quality for no reason.
|
|
|
|
zamfofex
|
2023-01-09 08:52:28
|
I’ve been experimenting with the various effort settings for a pixel art sequence I’m working on. Mostly for fun! Most images are 128×128 pixels in size, except for the cover which is 64×128 pixels. It seems `-e10` is really effective for it. It also seems like `-e3` is worse than `-e2` in terms of file size for some reason (i.e. it produces larger files).
|
|
2023-01-09 08:54:10
|
These are the resulting file sizes. The PNGs were produced with `zopflipng`. All of them are losslessly encoded.
|
|
|
|
veluca
|
2023-01-09 08:58:08
|
-e3 being worse then -e2 for pixel art doesn't surprise me at all
|
|
2023-01-09 08:58:27
|
e3 is very good for photo, but that's about it
|
|
|
|
Demiurge
|
2023-01-09 09:20:14
|
Maybe it would be nice if the encoder has a "tune for photo" and "tune for graphic" setting... or better yet, if it could tell how to tune/adapt itself, including for images that have a mix of both in different regions of the image.
|
|
2023-01-09 09:21:23
|
Or maybe just a note somewhere about -e 3 being a poor choice for graphics
|
|
|
veluca
e3 is very good for photo, but that's about it
|
|
2023-01-09 09:23:38
|
that's nice to know
|
|
2023-01-09 09:23:57
|
We're talking about lossless right?
|
|
|
|
_wb_
|
|
|
afed
|
2023-01-09 09:36:28
|
perhaps identifying palette as in fjxl and using a different mode for these images would be better
|
|
|
|
zamfofex
|
2023-01-10 03:26:30
|
Is there any reading material on JPEG XL as a format? I’m contemplating investigating tackling creating an implementation, but I haven’t found anything explaining how the format works at all, except for the standard itself.
|
|
|
|
_wb_
|
2023-01-10 07:11:56
|
there's a white paper but that's very high level, not useful for your purpose: https://ds.jpeg.org/whitepapers/jpeg-xl-whitepaper.pdf
|
|
|
|
Traneptora
|
|
Is there any reading material on JPEG XL as a format? I’m contemplating investigating tackling creating an implementation, but I haven’t found anything explaining how the format works at all, except for the standard itself.
|
|
2023-01-10 01:52:53
|
https://github.com/libjxl/libjxl/blob/main/doc/xl_overview.md
|
|
2023-01-10 02:00:43
|
not super thorough but it describes some stuff
|
|
|
|
Demiurge
|
2023-01-10 02:33:10
|
https://github.com/libjxl/libjxl/blob/main/doc/format_overview.md
|
|
2023-01-10 02:33:16
|
Don't you mean this?
|
|
|
|
joppuyo
|
|
Demiurge
Seriously someone needs to stop those crazy people who keep using -m
|
|
2023-01-10 02:55:12
|
What’s the problem with modular? 😮
|
|
|
|
Demiurge
|
2023-01-10 02:57:55
|
Nothing is wrong with modular, but current libjxl modular mode is only really optimized for doing lossless. If you force libjxl to use lossy modular instead of vardct you are just shooting yourself in the foot for no reason
|
|
2023-01-10 02:58:16
|
There's never any situation where you should force the encoder to use modular when it otherwise would not have
|
|
2023-01-10 02:59:44
|
libjxl will automatically decide when to use modular and when to use vardct unless you override it with -m which is never a good idea
|
|
|
|
joppuyo
|
2023-01-10 03:00:18
|
Isn’t lossless just modular with 100% quality?
|
|
|
|
Demiurge
|
2023-01-10 03:00:56
|
sorta, but there are a lot of different coding tools in modular mode and some are not used in lossless mode.
|
|
2023-01-10 03:01:04
|
like the squeeze transform
|
|
|
|
_wb_
|
2023-01-10 03:01:14
|
modular can do many things, but some things only make sense for lossless and some only make sense for lossy
|
|
2023-01-10 03:01:56
|
normal palette for example doesn't mix well with lossy, since a small error in a palette index can make the color completely different
|
|
2023-01-10 03:02:35
|
squeeze works quite well for lossy — not as good as DCT, but it's not super bad either
|
|
2023-01-10 03:03:26
|
squeeze can also be used for lossless (which can be nice for progressive lossless) but it generally comes at a cost in compression density
|
|
|
|
joppuyo
|
2023-01-10 03:04:04
|
If I want to encode a lossless image, I’d use cjxl with -m and -q 100, I don’t get how this is ”wrong” and should be prevented
|
|
|
|
Demiurge
|
2023-01-10 03:04:23
|
Basically the current state of "lossy modular" in libjxl does not have as much tuning as vardct and you should never force it to be used.
|
|
2023-01-10 03:04:43
|
If you use -q 100 then the -m is redundant
|
|
2023-01-10 03:05:04
|
It will automatically tell you're asking for lossless and will use modular mode for lossless
|
|
2023-01-10 03:05:27
|
At least cjxl will
|
|
|
|
_wb_
|
2023-01-10 03:05:32
|
there is no way to do lossless without modular, so -q 100 / -d 0 indeed forces modular
|
|
|
|
joppuyo
|
2023-01-10 03:06:29
|
Hmmm ok
|
|
2023-01-10 03:07:25
|
I remember in some cases cjxl will automatically do lossless jpeg recompression which was suprising to me, if the input is JPEG
|
|
|
|
Demiurge
|
2023-01-10 03:09:03
|
Yes, there is a bug in 0.7 in fact where it will ignore your settings and always do lossless jpeg recompression unless you do --lossless-jpeg=0
|
|
|
|
joppuyo
|
2023-01-10 03:13:49
|
Anyway, I don’t really like the idea of somehow preventing the user from selecting options that are non-optimal at the library level. It would be much better to educate the users and people who implement the library
|
|
2023-01-10 03:14:44
|
Sure, some projects are lazy and just port the API directly but usually different projects have different priorities and choose to implement things in different ways
|
|
|
|
Demiurge
|
2023-01-10 03:50:08
|
You can't educate people. But you can make it harder for the uneducated to shoot themselves in the foot, by adding 1 extra step to the library before handing someone a footgun.
|
|
2023-01-10 03:50:32
|
If anything that's the best way to try to educate people. By forcing them to stop in their tracks and read.
|
|
|
|
_wb_
|
2023-01-10 04:03:26
|
In the tools (cjxl/djxl), I think it's a good idea to keep things simple, do the probably-right thing by default, and hide 'advanced' settings behind flags and/or only mention them in the verbose help
In the API (libjxl), I think it's mostly important to have good documentation and good examples of how to use it and explanations of which settings make the most sense to expose to end-users. We currently basically only have the documentation in the comments of the .h files; probably we should add something like a tutorial document that only covers the 'basic' API functions and encode settings. Also I think we need to just keep an eye on the (main) libjxl integrations and just open issues there when we notice they're handing footguns to end-users...
|
|
|
|
Demiurge
|
2023-01-10 04:22:02
|
Whatever works...
|
|
2023-01-10 04:22:21
|
So long as it gets the job done.
|
|
2023-01-10 04:26:09
|
Communication like that is important too because even if updated libjxl starts to block harmful options by default, they can just keep shipping old libjxl.
|
|
2023-01-10 04:27:03
|
So yeah, whatever method will best inform people and prevent them from shooting themselves, I'm all for.
|
|
|
Rally
|
2023-01-10 11:49:43
|
A question, is it possible to embed a jxl decoder in glsl/shadertoy/interactive shader format shaders?
I have an usecase in mind for live visuals as JXL's generative art capabilities are amazing, but I'm not sure if its even possible
|
|
|
|
Demiurge
|
2023-01-11 12:28:24
|
That's a good idea...
|
|
|
|
veluca
|
|
Rally
A question, is it possible to embed a jxl decoder in glsl/shadertoy/interactive shader format shaders?
I have an usecase in mind for live visuals as JXL's generative art capabilities are amazing, but I'm not sure if its even possible
|
|
2023-01-11 02:14:22
|
I'm going to bet it would be a massive effort, but not too bad if you just want to put jxl art in there
|
|
2023-01-11 02:14:30
|
well, a subset of it
|
|
|
|
yurume
|
2023-01-11 02:44:38
|
isn't MA tree decoding entirely serial? it would be very hard to benefit from shaders for this reason, in general.
|
|
|
|
Demiurge
|
2023-01-11 02:46:11
|
No idea how that prediction tree stuff works. Aside from this: https://jxl-art.surma.technology/wtf.html
|
|
2023-01-11 02:46:19
|
But it should be extremely fast, either way.
|
|
2023-01-11 02:46:57
|
It was designed to be fast to decode after all.
|
|
|
|
yurume
|
2023-01-11 02:47:27
|
a very high level summary: for each pixel (in the scanline order), walk through the decision tree, calculate the predicted pixel value, and move on. (if it weren't a jxl art, this predicted value will be added to the residue.)
|
|
2023-01-11 02:48:16
|
MA tree is _not_ designed to be generally fast; libjxl has multiple specialized routines to handle known MA trees. I learned this hard way.
|
|
|
|
_wb_
|
2023-01-11 05:55:39
|
Walking a tree is inherently a branchy thing, and decision nodes that depend on West make things inherently sequential in terms of data dependencies (if decision nodes only depend on row-above, you could in principle compute contexts in parallel).
|
|
2023-01-11 05:58:42
|
Branching can be reduced by combining nodes (going from binary to quadtree) and doing as much as possible with cmov instead of actual conditional jumps.
|
|
2023-01-11 06:05:42
|
Data dependencies are worst for Weighed predictor and its decision node property (which is inherently sequential). Other predictors/properties still allow some amount of diagonal wavefront parallelism in principle.
|
|
|
|
veluca
|
|
yurume
isn't MA tree decoding entirely serial? it would be very hard to benefit from shaders for this reason, in general.
|
|
2023-01-11 10:28:27
|
depends, if your tree doesn't use the top-right pixel or WP, you can parallelize by diagonals
|
|
2023-01-11 10:29:24
|
you can also go next level and do trees that have linear functions as decision nodes, and then do something a 15x16 matrix-vector multiply to do 3 layers at once
|
|
|
|
_wb_
|
2023-01-11 10:34:34
|
It's basically an expressive context model syntax that is expensive in the general case but has many interesting subsets that can be specialized to become quite cheap (like the current e1, e2 and e3), and I think it's likely that we will identify additional interesting subsets that are worth specializing for but that will still decode with a generic decoder. I consider this to be one of the 'future proof' aspects of jxl.
|
|
|
|
Demiurge
|
2023-01-11 05:25:59
|
Well there are still limits to keep it reasonably fast and simple, like it only applies to a 1024x1024 chunk
|
|
|
|
username
|
|
pandakekok9
I used this PR btw: https://github.com/WaterfoxCo/Waterfox/pull/2938
|
|
2023-01-12 11:49:59
|
does changing ```OrientedIntSize size(mInfo.xsize, mInfo.ysize);``` to ```gfx::IntSize size(mInfo.xsize, mInfo.ysize);``` fix the tab crashing? I am unable to test this myself and the friend I get to test for me is unavailable and the last I heard from them is they couldn't even get the most recent version of unmodified waterfox compiling anymore
|
|
2023-01-12 11:50:38
|
oh yeah I should probably mention it's on line 263 in nsJXLDecoder.cpp
|
|
2023-01-12 11:59:46
|
it would have been on line 262 but I accidentally included a extra newline
|
|
|
|
DZgas Ж
|
2023-01-13 04:22:23
|
Invalid flag value for --effort: Valid range is {1, 2, ..., 9}.
|
|
2023-01-13 04:22:31
|
huh? how use e10
|
|
|
|
pandakekok9
|
|
username
does changing ```OrientedIntSize size(mInfo.xsize, mInfo.ysize);``` to ```gfx::IntSize size(mInfo.xsize, mInfo.ysize);``` fix the tab crashing? I am unable to test this myself and the friend I get to test for me is unavailable and the last I heard from them is they couldn't even get the most recent version of unmodified waterfox compiling anymore
|
|
2023-01-13 04:23:30
|
Whoops just saw this right now, I don't see why that part would cause the crash, but I will try changing it as you said. Thankfully I haven't deleted my obj directory yet, I really don't want to build anything mozilla-central from the beginning ever again
|
|
|
|
DZgas Ж
|
2023-01-13 04:26:20
|
oh --allow_expert_options
|
|
2023-01-13 04:26:42
|
I missed the moment of adding this command
|
|
|
|
pandakekok9
|
|
pandakekok9
Whoops just saw this right now, I don't see why that part would cause the crash, but I will try changing it as you said. Thankfully I haven't deleted my obj directory yet, I really don't want to build anything mozilla-central from the beginning ever again
|
|
2023-01-13 04:26:57
|
Oh you gotta be fucking kidding me, I think the Mozilla build system just decided a clobber is necessary
|
|
2023-01-13 04:27:50
|
I'm literally compling from accessible/* again, despite literally not touching anything else other than the JXL decoder code
|
|
2023-01-13 04:28:26
|
Does anyone know how to properly disable Rust LTO
|
|
|
|
DZgas Ж
|
2023-01-13 04:38:24
|
Encoding [Modular, lossless, effort: 9],
Compressed to 1121 bytes (0.208 bpp).
Encoding [Modular, lossless, effort: 10],
Compressed to 1121 bytes (0.208 bpp).
🥹
|
|
|
|
username
|
|
pandakekok9
Whoops just saw this right now, I don't see why that part would cause the crash, but I will try changing it as you said. Thankfully I haven't deleted my obj directory yet, I really don't want to build anything mozilla-central from the beginning ever again
|
|
2023-01-13 05:43:28
|
my rationale for thinking that change might make a difference is I was looking around and found that at some point mozilla did a refactor with bug 1732115 and in part 3 they changed all instances of a few different things in all the image decoders to be different for example changing "gfx::IntSize" to "OrientedIntSize" and when comparing the pull request for waterfox with the patches from <@288069412857315328> the only thing different besides a stray newline was that <@288069412857315328>'s patch didn't seem to take into account bug 1732115 and when I tested the firefox nightly build they compiled I didn't get a tab crash on https://saklistudio.com/jxltests/ but while testing against a older build of waterfox with the pull request I did get a tab crash
|
|
2023-01-13 05:44:15
|
I don't know much about the refactor in bug 1732115 but I thought it was worth a shot to see if changing that line would make a difference
|
|
|
|
fab
|
2023-01-14 10:58:05
|
i'm trying vardct effort 10
|
|
2023-01-14 10:58:15
|
With a 2k image
|
|
2023-01-14 10:58:23
|
D1
|
|
2023-01-14 10:58:45
|
It doesn't use too much memory
|
|
2023-01-14 10:59:05
|
Is it slower than avm 489.1s
|
|
2023-01-14 10:59:13
|
?
|
|
2023-01-14 11:00:05
|
1 hour is ok?
|
|