|
Traneptora
|
2022-06-18 08:37:46
|
other than, perhaps creating a level4 that uses a significantly smaller max distance for hardware implentations
|
|
|
_wb_
|
2022-06-18 09:32:29
|
I think a hw profile can probably get away with not allowing lz77 at all
|
|
2022-06-18 09:32:44
|
It's not like it is being used for vardct
|
|
|
improver
|
2022-06-19 03:31:17
|
im redoing my previous golang implementation attempt in rust, no promises though
|
|
|
|
veluca
|
2022-06-19 04:02:25
|
if you feel like continuing https://github.com/libjxl/jxl-rs ...
|
|
|
|
improver
|
2022-06-19 04:15:16
|
im not sure if i like its headers derive stuff, as it's kinda overcomplicated, but I'll probably grab entropy coding from there
|
|
2022-06-19 04:16:45
|
i much prefer procedural approach for reading codestream, as this way it's very clear what logic is used for unpacking fields
|
|
|
yurume
|
|
veluca
if you feel like continuing https://github.com/libjxl/jxl-rs ...
|
|
2022-06-19 04:57:17
|
I actually want to port jxsml to Rust when it's done
|
|
|
|
improver
|
2022-06-19 04:58:19
|
rust's error handling ways are really good for implementing this tbh
|
|
|
|
yurume
|
2022-06-19 04:58:37
|
well, actually anything would be better than C (lol)
|
|
|
|
improver
|
2022-06-19 04:59:33
|
golang wasn't non-annoying, but in rust it's just a matter of adding ? at the end of reading routines, and probably even less work if you use declarative approach
|
|
2022-06-19 04:59:47
|
(i don't)
|
|
2022-06-19 05:00:51
|
is your code open source?
|
|
2022-06-19 05:00:56
|
& published
|
|
|
|
yurume
|
2022-06-19 05:01:52
|
technically yes (CC0 and available in the gist), but yet to be actually published
|
|
2022-06-19 05:02:17
|
I haven't updated the gist for a week because of a large refactoring
|
|
|
|
improver
|
2022-06-19 05:02:53
|
if you published it as git repo i guess that'd b better
|
|
2022-06-19 05:03:17
|
i could really use procedural C code as reference
|
|
2022-06-19 05:03:36
|
& you may not even need to do rewrite in rust in the end if you like what i end up with
|
|
|
|
yurume
|
2022-06-19 05:04:03
|
for the reference the latest code is https://gist.github.com/lifthrasiir/137a3bf550f5a8408d4d9450d026101f
|
|
2022-06-19 05:05:32
|
I intentionally avoided many abstractions for the simplicity (because this was supposed to be a jxl equivalent of stb_image) which I really want to reintroduce in the Rust port
|
|
|
|
improver
|
2022-06-19 05:06:12
|
i have the same mindset
|
|
|
|
yurume
|
2022-06-19 05:08:51
|
jxl-rs in comparison feels like a more or less direct translation of libjxl, which is first and foremost a combined decoder & encoder and a platform for format experimentation (which we hardly need now)
|
|
|
|
improver
|
2022-06-19 05:09:40
|
i only wanna do decoder, and i wanna make it easily readable
|
|
2022-06-19 05:10:54
|
& possibly have some important features like memory efficient downscaled / cropped decoding
|
|
|
|
yurume
|
2022-06-19 05:14:40
|
frankly I was originally interested in jxl because its precursor FLIF was (technically) able to be decoded even when truncated
|
|
2022-06-19 05:15:35
|
so I guessed JPEG XL will hopefully have the same charateristics, which is by the way true but not directly available in the browsers
|
|
2022-06-19 05:16:14
|
I looked at the spec (at that time, briefly) and found that you can read TOC to determine where to manually truncate
|
|
2022-06-19 05:16:37
|
that was my initial motivation to reimplement a decoder
|
|
|
|
improver
|
2022-06-19 05:17:09
|
i don't think downscaled mode works in all the cases when truncated
|
|
2022-06-19 05:17:42
|
as in, i think you could still memory efficiently decode whatever sized stuff
|
|
2022-06-19 05:17:53
|
however, non-progressive modular comes in blocks
|
|
|
|
yurume
|
2022-06-19 05:18:03
|
I of course thought of progressive ones
|
|
2022-06-19 05:18:37
|
but as you've mentioned, even non-progressive images can still have a meaningful thumbnail
|
|
|
|
improver
|
2022-06-19 05:18:42
|
so to save memory you'd read block by block and downscale before putting pixels in
|
|
2022-06-19 05:19:03
|
im not yet completely sure whether it'd be hard to implement but i wanna try
|
|
|
|
yurume
|
2022-06-19 05:19:23
|
when I tested however browsers just read all the image even at the lowest scale
|
|
|
|
improver
|
2022-06-19 05:19:43
|
that's because current implementation doesn't have this feature
|
|
|
|
yurume
|
2022-06-19 05:20:05
|
libjxl, or its browser adapter(s)?
|
|
|
|
improver
|
2022-06-19 05:21:08
|
i think libjxl should actually be able to give progression steps, and browsers could just stop decoding when they get something like size 2x of intended viewport
|
|
2022-06-19 05:21:44
|
i think progressive stuff is implemented in libjxl
|
|
|
|
yurume
|
2022-06-19 05:21:44
|
yeah, it's definitely supported by the spec but I guess the implementation is not leveraging that then
|
|
|
|
veluca
|
2022-06-19 08:19:27
|
Yeah wiring it up in browsers is not there yet
|
|
|
|
yurume
|
2022-06-19 09:14:58
|
I'm currently decoding HF coefficients, which makes use of `non_zeros` and that should be less than the number of possible coefficients (`63 * num_blocks`) in the current varblock (the spec has no check for this, by the way). as such I believe the maximum value of `NonZeros` is 63 because `(non_zeros + num_blocks - 1) Idiv num_blocks <= 63`, and in `NonZerosContext` `predicted` will never exceed 64 and the final set of coefficients out of 37 sets will be unused. is my understanding correct?
|
|
2022-06-19 09:16:05
|
I realize my question is quite long to comprehend directly, so let me chop up a bit:
|
|
2022-06-19 09:16:43
|
each varblock consists of `W * H` coefficients where both `W` and `H` are multiples of 8.
|
|
2022-06-19 09:17:42
|
first `num_blocks = (W/8) * (H/8)` coefficients are LLF coefficients and separately encoded, so there are `W * H - num_blocks = 63/64 * (W * H) = 63 * num_blocks` HF coefficients
|
|
2022-06-19 09:18:42
|
`non_zeros` is the number of non-zero HF coefficients to read, and itself is decoded with a context derived from past number of non-zero coefficients `NonZeros(x, y)`.
|
|
2022-06-19 09:19:29
|
since each varblock has different number of coefficients, they are normalized as `ceil(non_zeros / num_blocks)` and stored to `NonZeros`.
|
|
2022-06-19 09:20:03
|
now we have `non_zeros <= 63 * num_blocks`, so the maximum value of the normalized `non_zeros` is 63.
|
|
2022-06-19 09:21:29
|
when the decoder reads `non_zeros` it makes use of multiple contexts including the "predicted" `non_zeros` after the normalization, which again is at most 63. this prediction is mapped from [0, 8) to [0, 8) and from [8, 64) to [8, 36).
|
|
2022-06-19 09:22:22
|
but both the spec (`NonZerosContext` function) and libjxl code suggests that there is a possibility for a mapping from [64, inf) to 36, since there are 37 sets of contexts there
|
|
2022-06-19 09:23:51
|
my question is whether this is due to my misunderstanding, an oversight or just a vestige
|
|
|
|
veluca
|
2022-06-19 10:03:06
|
on one hand, I'd say you're right
|
|
2022-06-19 10:03:22
|
on the other hand, I kinda explicitly remember needing to handle 64 nonzeros
|
|
2022-06-19 10:03:39
|
well, 64 after that ceil()
|
|
|
|
yurume
|
2022-06-19 10:03:57
|
I'm yet to reach the point that I can actually decode coefficients from real images and check for any such cases, so my logic might be off
|
|
|
|
Hello71
|
2022-06-19 06:31:55
|
i feel like wuffs might be ~~better~~more _interesting_ than rust
|
|
|
|
Traneptora
|
2022-06-19 06:45:31
|
wuffs?
|
|
|
OkyDooky
|
2022-06-19 06:58:03
|
Didn't realize there is a bridged Matrix room. Hi I'm an active Krita contributor (not the one who implemented JPEG XL support though.)
|
|
|
|
yurume
|
2022-06-19 07:07:41
|
I agree wuffs is a very interesting approach, but it's not exactly a pleasant language to start with ๐
|
|
|
Traneptora
wuffs?
|
|
2022-06-19 07:10:08
|
https://github.com/google/wuffs basically a parser-generator-cum-language that compiles to C
|
|
2022-06-19 07:13:07
|
IIRC wuffs code can't even dynamically allocate memory (which would simplify many things though)
|
|
2022-06-19 07:16:22
|
a better way to put wuffs to perspective would be that this is actually a single massive library and an overblown DSL to write that library
|
|
|
|
Traneptora
|
2022-06-19 07:21:52
|
what's the "bridged matrix room"
|
|
|
|
yurume
|
2022-06-19 07:22:37
|
matrix as in https://matrix.org/
|
|
|
|
veluca
|
|
yurume
https://github.com/google/wuffs basically a parser-generator-cum-language that compiles to C
|
|
2022-06-19 08:42:34
|
IIRC they didn't manage to write a *JPEG* decoder in wuffs yet
|
|
2022-06-19 08:42:53
|
so I wouldn't hold my breath on something like JPEG XL being doable any time soon...
|
|
|
|
yurume
|
2022-06-19 08:59:20
|
well it might be possible to pierce multiple parts of the decoder which by themselves don't allocate memory
|
|
2022-06-19 08:59:50
|
JPEG XL is far larger but I guess JPEG is within a reach
|
|
|
|
Traneptora
|
2022-06-19 09:11:50
|
I see Matrix has a spambot issue
|
|
2022-06-19 09:12:23
|
and we can't block or ban them here, that's sort of annoying
|
|
|
190n
|
2022-06-19 09:26:03
|
<@&807636211489177661> delete pls
|
|
|
fab
|
2022-06-20 11:49:58
|
test encoding cjxlng
|
|
2022-06-20 11:50:39
|
|
|
2022-06-20 11:56:23
|
epf 0 doesn't change esult in cjxl
|
|
2022-06-20 11:57:08
|
|
|
2022-06-20 11:57:29
|
seems like all the two are the same
|
|
2022-06-20 11:57:50
|
cjxl hasn't received improvements
|
|
2022-06-20 11:58:02
|
it's all placebo
|
|
2022-06-20 11:58:29
|
plus is exactly cjxl at s3
|
|
2022-06-20 11:59:08
|
4 month and 20 days without quality improvements
|
|
2022-06-20 12:01:05
|
|
|
2022-06-20 12:01:06
|
quality is great
|
|
2022-06-20 12:04:40
|
i tried with this image
|
|
2022-06-20 12:04:51
|
a was flagged as explicit
|
|
2022-06-20 12:05:05
|
i can't send here sorry
|
|
2022-06-20 06:17:25
|
a chat between me and wb
|
|
2022-06-20 06:17:36
|
indeciphrable
|
|
2022-06-20 06:27:00
|
|
|
2022-06-20 06:28:05
|
1.329 files, 25 folders
|
|
2022-06-20 06:32:12
|
also cringe command
|
|
2022-06-20 06:32:17
|
|
|
2022-06-20 06:33:18
|
this are old documents i hAVE
|
|
|
|
Traneptora
|
2022-06-21 02:15:45
|
what is this log
|
|
|
|
fab
|
2022-06-21 03:30:15
|
|
|
2022-06-21 03:30:37
|
|
|
2022-06-21 03:31:11
|
simplified
|
|
2022-06-25 11:55:06
|
phton noise 252 q 86.621 s 9 epf 2 fd1 gaborish 0 patches 0
|
|
2022-06-25 11:55:15
|
Is this command possoble
|
|
2022-06-25 11:55:34
|
Does photon noise supports faster decoding 1 in speed 9
|
|
2022-06-25 11:56:54
|
|
|
2022-06-25 11:58:20
|
I'm optimizing the usb drives
|
|
2022-06-25 12:12:09
|
A folder with mixed jpg and png weights 1,8gb
|
|
2022-06-25 12:12:26
|
No problem they are 2400 files
|
|
|
|
Traneptora
|
2022-06-25 03:17:27
|
woot, my libjxl ffmpeg wrapper fix got merged finally
|
|
|
|
_wb_
|
|
|
Traneptora
|
2022-06-25 04:59:31
|
now when you decode a jxl file with libavcodec, it'll tag the colorspace as tagged in the file
|
|
|
|
JendaLinda
|
2022-07-01 10:26:14
|
Can anybody explain what exactly does chroma-from-luma do in lossless JPEG transcodingt? The decoded images slightly differ if CFL is turned on or off.
|
|
|
|
_wb_
|
2022-07-01 12:07:44
|
CFL subtracts some weight times luma coeffs from chroma coeffs. If they are strongly correlated, this can help for compression as it can make chroma coeffs closer to zero.
|
|
2022-07-01 12:08:17
|
CFL works on dequantized coeffs though, not on quantized ones.
|
|
2022-07-01 12:10:57
|
So for jpeg recompression, we do it in such a way that it is lossless w.r.t. requantization, but the image does change slightly since luma and chroma have different quantization constants so after dequant and cfl chroma will get values that are not equal to midpoints of quantization buckets (as you would get without cfl)
|
|
2022-07-01 12:12:49
|
The max error will be smaller than half the min of the luma and chroma quantization bucket size, which is ok for jpeg conformance, but it does introduce some error
|
|
2022-07-01 12:13:57
|
(of course it could be that this error actually makes it get closer to the original image, since if chroma is indeed correlated to luma, cfl might reduce the effective quantization error in chroma)
|
|
2022-07-01 12:40:46
|
So e.g. you could have
Luma quantization: 5
Chroma quantization: 8
Luma quantized coeffs: 1 2 3 4 5
(Unquantized: 5 10 15 20 25)
Chroma quantized coeffs: 1 1 4 2 3
(Unquantized: 8 8 32 16 24)
CFL could then e.g. subtract 1.0 times luma from chroma. The encoded quantized chroma coeffs would then be:
0 0 2 0 0
which would reconstruct to
5 10 31 20 25
which would requantize to
1 1 4 2 3
so the jpeg can be reconstructed, but of course the decoded pixels would be a bit different.
|
|
2022-07-01 12:44:02
|
Of course we don't know what the original values were before quantization, if they were something like
6 9 30 19 25
then the cfl result would be better than the no-cfl result, but if they were something like
9 7 33 15 22
then the no-cfl result would be better than the no-cfl result.
|
|
2022-07-01 12:45:13
|
It would be interesting to test on some large corpus if the effect of cfl tends to be good or bad on average. It's a bit hard to say.
|
|
2022-07-01 12:49:37
|
In any case, you can disable it for jpeg recompression, and it only applies to 4:4:4 jpegs (when chroma is subsampled, cfl cannot be used anyway)
|
|
2022-07-01 12:59:42
|
CfL weights are signalled in 64x64 blocks, and if there is little correlation between luma and chroma, the weight will be zero. If there is enough correlation to make it use a nonzero weight, then my hypothesis is that this might actually be slightly beneficial.
|
|
|
|
JendaLinda
|
2022-07-01 01:00:26
|
Thank you for explanation. Yes, in both cases the exact JPEG data can be restored. The curious fact was that decoding to pixels gives slightly different result. The difference is negligible though, so it doesn't bother me. I've noticed that CFL makes the file slightly smaller.
|
|
|
Yari-nyan
|
|
Jim
|
2022-07-02 03:02:46
|
<:DogWhat:806133035786829875> <:FeelsReadingMan:808827102278451241> The lie detector determined that your life is misinformation <:kekw:808717074305122316>
|
|
|
BlueSwordM
|
|
Yari-nyan
huh?
|
|
2022-07-02 03:37:42
|
Where is this from? <:kekw:808717074305122316>
|
|
2022-07-02 03:39:14
|
`Thank you and sorry for reopening, but I found out that AVIF is more effective than JPEG XL since JPEG XL requires R'G'B' convertion which is lossy, dither is hard to compress, but with it turned off rounding of YCbCr is even harder to compress apparently, does not support subsampling like in old JPEG, no support for superblack/superwhite in limited range and out-of-gamut like sYCC, no support for BT.709 matrix like in JPEG 2000, which is just .JPEG XL is useless for me`
|
|
2022-07-02 03:40:19
|
https://trac.ffmpeg.org/ticket/7621
|
|
2022-07-02 03:40:59
|
https://trac.ffmpeg.org/ticket/7621#comment:17
|
|
2022-07-02 03:44:11
|
This has got to be a low information comment.
1. JXL requires RGB conversion <:kekw:808717074305122316>
Nope, it converts stuff to XYB in lossy mode at very high bit depth internally, which is basically the best thing psycho-visually speaking, especially since no modern lossy video codecs use RGB(for a good reason).
2. Dither is hard to compress.
With its current featureset, libjxl with its encoder cjxl is the absolute best encoder I've found to encode any kind of dither efficiently.
3. Does not support subsampling.
Not a bad thing honestly. We have to move with the times.
4. No support for BT709 matrix.
Is that even really needed these days since you can just attach the relevant metadata or use something like BT1886?
|
|
|
|
Yari-nyan
|
2022-07-02 04:20:20
|
wow
|
|
2022-07-02 04:21:35
|
ty for clarifying
|
|
|
|
Traneptora
|
|
BlueSwordM
This has got to be a low information comment.
1. JXL requires RGB conversion <:kekw:808717074305122316>
Nope, it converts stuff to XYB in lossy mode at very high bit depth internally, which is basically the best thing psycho-visually speaking, especially since no modern lossy video codecs use RGB(for a good reason).
2. Dither is hard to compress.
With its current featureset, libjxl with its encoder cjxl is the absolute best encoder I've found to encode any kind of dither efficiently.
3. Does not support subsampling.
Not a bad thing honestly. We have to move with the times.
4. No support for BT709 matrix.
Is that even really needed these days since you can just attach the relevant metadata or use something like BT1886?
|
|
2022-07-02 04:55:38
|
so clarification, XYB conversion is lossy and that's why it's disabled in lossless mode
|
|
2022-07-02 04:55:45
|
but it's lossy in the sense that it's not bitexact
|
|
2022-07-02 04:55:52
|
if you're doing lossy compression, who cares
|
|
|
|
BlueSwordM
|
|
Traneptora
so clarification, XYB conversion is lossy and that's why it's disabled in lossless mode
|
|
2022-07-02 04:56:35
|
Yeah, I should have clarified that the error is so small that it is super negligible in lossy mode.
Lossless is where you obviously don't want lossy stuff, so no XYB.
|
|
|
spider-mario
|
2022-07-02 06:35:57
|
unless I am misunderstanding โno support for [โฆ] out-of-gamut like sYCCโ, that seems wrong as well
|
|
2022-07-02 06:36:37
|
and yes, what would you need the BT.709 matrix for if itโs going to be compressed in XYB??
|
|
|
|
Traneptora
|
2022-07-03 04:01:12
|
except that requires dealing with TRAC which I hate
|
|
|
|
Yari-nyan
|
2022-07-03 05:54:42
|
i wish android added support for JPEG XL
|
|
2022-07-03 05:55:27
|
and that Open Camera added support for AVIF (which is supported in android 12) and ideally JPEG XL at that point
|
|
2022-07-03 05:56:39
|
i guess libjxl isn't stable enough yet?
|
|
2022-07-03 06:00:40
|
android seems to struggle loading some AVIF images
|
|
2022-07-03 06:02:22
|
like the ones made with cavif-rs from inside termux (why'd they add that instead of the usual tools?)
|
|
2022-07-03 06:02:53
|
but i guess this is off topic now
|
|
2022-07-03 06:20:07
|
maybe android 14 will have it, and then my phone will be unsupported
|
|
2022-07-03 06:20:20
|
: D
|
|
|
BillyLโBobโจ
|
2022-07-03 06:33:58
|
how dose jpeg xl work what extra features for coding patterns DCT, etc..?
|
|
|
Fraetor
|
|
BillyLโBobโจ
how dose jpeg xl work what extra features for coding patterns DCT, etc..?
|
|
2022-07-03 07:53:15
|
This gives a decent overview:
https://docs.google.com/presentation/d/1LlmUR0Uoh4dgT3DjanLjhlXrk_5W2nJBDqDAMbhe8v8/edit#slide=id.gae1d3c10a0_0_17
|
|
|
|
BlueSwordM
|
2022-07-04 02:08:31
|
Yeah, it's fine.
|
|
|
The_Decryptor
|
2022-07-04 08:01:18
|
Sounds like their source data is YUV
|
|
|
|
BlueSwordM
|
|
The_Decryptor
Sounds like their source data is YUV
|
|
2022-07-04 02:08:21
|
>No dither needed in AVIF.
lmaooo good one.
|
|
2022-07-04 02:11:05
|
The guy is a rock head, isn't he?
Did he even try to convert to JXL losslessly, and then check if the output is lossless using a metric of his choice?
|
|
2022-07-04 02:11:49
|
Like, there is a reason XYB is not used in lossless, but instead YCoCgR?....
|
|
|
|
Traneptora
|
|
BlueSwordM
Like, there is a reason XYB is not used in lossless, but instead YCoCgR?....
|
|
2022-07-05 03:57:23
|
YCoCg.... or any of the 42 RCTs instead <:PauseChamp:755852977574510632>
|
|
2022-07-05 07:15:04
|
Ugh, I think https://github.com/libjxl/libjxl/commit/acc405da4471afd7a66c0d8fcd545dd4bfadee7c broke the FFmpeg wrapper
|
|
2022-07-05 07:25:09
|
opened an issue about it
|
|
2022-07-05 07:25:15
|
https://github.com/libjxl/libjxl/issues/1578
|
|
|
|
fab
|
2022-07-06 01:40:28
|
|
|
2022-07-06 01:41:05
|
new djxl isn't faste
|
|
2022-07-06 01:41:13
|
i comp one june vs five july
|
|
2022-07-06 01:41:24
|
july is slowe
|
|
2022-07-06 01:42:12
|
i got now 3,44 mpx/s
|
|
2022-07-06 01:44:27
|
no change fo the encoding quality
|
|
2022-07-06 01:44:31
|
same size
|
|
2022-07-06 01:50:27
|
1,78mb the decoding of 0.6.1
|
|
2022-07-06 01:52:40
|
encoding on a selfie got faster 2,08 mpxs vs 1,99mpx of old month
|
|
2022-07-06 01:52:51
|
nine month passed
|
|
2022-07-06 01:53:58
|
with0.6.1 selfie encoding 1,51 mpx
|
|
2022-07-06 01:54:09
|
for %i in (C:\Users\Use\Pictures\a\*.png) do cjxl "%i" "%i.jxl"
|
|
2022-07-06 01:55:05
|
ram consumption, exif, quality, if size has decreased i have not tested that
|
|
2022-07-06 01:56:02
|
rotation doesn't work with 0.6.1
|
|
2022-07-06 01:56:26
|
even with png
|
|
2022-07-06 01:56:52
|
hope you didn't encoded anime with it
|
|
2022-07-06 04:05:19
|
I'm thinking to convert the jpg i had fear of losing quality to q 97.17 s 4
|
|
2022-07-06 04:06:28
|
I will use jamaika Doom9 compiled cjxl
|
|
2022-07-06 04:06:40
|
It will do less damage
|
|
2022-07-06 04:18:10
|
now
|
|
2022-07-06 04:18:12
|
for %i in (C:\Users\Use\Downloads\a\*.jpg) do cjxl -d 0.787 -s 7 -I 0.744 --epf=3 --intensity_target=245 "%i" "%i.jxl"
|
|
2022-07-06 04:18:34
|
serious comparison with the filter at max
|
|
2022-07-06 04:24:17
|
|
|
2022-07-06 04:26:04
|
all have considerable artifacts
|
|
2022-07-06 04:27:42
|
Jxl typically After the 0.710 distance range is unusable
|
|
2022-07-06 04:28:20
|
Still experimental i won't touch this
|
|
|
|
_wb_
|
2022-07-06 05:44:14
|
I think d0.5 to d3 is the only range you need in practice, and for that range I think libjxl works quite fine
|
|
2022-07-06 05:45:07
|
using d0.787 in combination with --epf=3 doesn't really make sense, unless you're trying to abuse jxl to do some kind of denoising
|
|
|
|
fab
|
2022-07-06 05:53:16
|
I'm trying with default if sizes are different
|
|
|
_wb_
I think d0.5 to d3 is the only range you need in practice, and for that range I think libjxl works quite fine
|
|
2022-07-06 06:06:18
|
why i get diff file sizes evy time i encode
|
|
2022-07-06 06:06:25
|
i'm using
|
|
2022-07-06 06:06:30
|
for %i in (C:\Users\Use\Downloads\ad\*.jpg) do cjxl -d 1 -s 7 "%i" "%i.jxl"
|
|
2022-07-06 06:06:51
|
with png it don't happen
|
|
|
|
_wb_
|
2022-07-06 06:07:30
|
that's weird โ same encoder binary, different results with same command line?
|
|
|
|
fab
|
2022-07-06 06:10:22
|
the jpg is this
|
|
2022-07-06 06:10:39
|
|
|
2022-07-06 06:11:56
|
|
|
2022-07-06 06:12:45
|
C:\Users\Use>cjxl -d 1 -s 7 "C:\Users\Use\Downloads\t\minion.jpg" "C:\Users\Use\
Downloads\t\minion.jpg.jxl"
JPEG XL encoder v0.7.0 711382f [SSSE3,Emu128]
Read 630x354 image, 7.7 MP/s
Encoding [VarDCT, d1.000, squirrel], 4 threads.
Compressed to 55576 bytes (1.994 bpp).
630 x 354, 0.95 MP/s [0.95, 0.95], 1 reps, 4 threads.
C:\Users\Use>for %i in (C:\Users\Use\Downloads\t\*.jpg) do cjxl -d 1 -s 7 "%i" "
%i.jxl"
C:\Users\Use>cjxl -d 1 -s 7 "C:\Users\Use\Downloads\t\minion.jpg" "C:\Users\Use\
Downloads\t\minion.jpg.jxl"
JPEG XL encoder v0.7.0 711382f [SSSE3,Emu128]
Read 630x354 image, 13.6 MP/s
Encoding [VarDCT, d1.000, squirrel], 4 threads.
Compressed to 30529 bytes (1.095 bpp).
630 x 354, 1.07 MP/s [1.07, 1.07], 1 reps, 4 threads.
|
|
2022-07-06 06:14:42
|
|
|
2022-07-06 06:14:47
|
|
|
2022-07-06 06:15:02
|
wait is an i3 330m
|
|
|
|
_wb_
|
2022-07-06 06:31:31
|
yuck, that looks like a nasty bug
|
|
2022-07-06 06:32:42
|
the encoder is supposed to be deterministic (modulo float arithmetic possibly giving slightly different results depending on architecture/compiler, which can lead to slightly different results)
|
|
2022-07-06 06:33:12
|
this is a huge difference in filesize though, I wonder what's up with that
|
|
2022-07-06 06:38:12
|
can you run it with -v please? and does it persist if you do --num_threads=0 ?
|
|
|
|
fab
|
2022-07-06 06:40:34
|
with v it doesn't
|
|
2022-07-06 06:40:40
|
same size
|
|
2022-07-06 06:41:11
|
--num_threads=0 it does pessit
|
|
2022-07-06 06:41:46
|
-v + --num_threads=0 it does pessit
|
|
2022-07-06 06:43:12
|
yes it does pessit
|
|
2022-07-06 06:43:18
|
even with v
|
|
2022-07-06 06:43:51
|
new files
|
|
2022-07-06 06:43:58
|
|
|
2022-07-06 06:45:01
|
is a cmd error o cpu old
|
|
|
|
_wb_
|
2022-07-06 07:14:47
|
i mean could you share the output of -v
|
|
|
|
fab
|
2022-07-06 07:15:32
|
sorry im noob
|
|
2022-07-06 07:15:36
|
ok
|
|
|
|
Deleted User
|
2022-07-07 12:03:42
|
So currently the roadmap is to release libjxl 0.7 in the coming months, and then *after* everything is stabilized in libjxl 1.0, bugs/feature requests in Chromium and Firefox will be prioritized? The bugs that are dependencies for enabling JXL by default.
|
|
|
Jyrki Alakuijala
|
|
So currently the roadmap is to release libjxl 0.7 in the coming months, and then *after* everything is stabilized in libjxl 1.0, bugs/feature requests in Chromium and Firefox will be prioritized? The bugs that are dependencies for enabling JXL by default.
|
|
2022-07-07 11:17:11
|
I believe we are making relatively fast progress on that -- some stabilization is needed for 0.7, but we have a working internal proto of the Chrome integration (with progression and HDR) already
|
|
2022-07-07 11:17:26
|
I wouldn't be surprised if we are able to make it within a month
|
|
|
|
Jim
|
2022-07-07 12:29:06
|
<:FeelsAmazingMan:808826295768449054> <:JXL:805850130203934781>
|
|
|
|
Deleted User
|
2022-07-07 09:19:47
|
Oh great so browser integration-related work is already a priority thank you
|
|
2022-07-07 09:24:28
|
Hmm that makes sense though to focus on progressive and HDR image support before browser integration because they're both key features of the format for web developers
|
|
|
fab
sorry im noob
|
|
2022-07-07 10:03:33
|
whaaat... but your roles say you are a champion? <:PeepoDiamondSword:805394101340078092>
|
|
|
Pashi
|
2022-07-08 07:12:39
|
Is there any advantage over jph aka HTJ2K?
|
|
2022-07-08 07:17:36
|
HT seems to have all the features of JXL due to it's wavelet design, progressive and region of interest coding, better speed and even good efficiency compared to JXL?
|
|
|
|
The_Decryptor
|
2022-07-08 07:24:37
|
JXL has better hope of support in apps and browsers
|
|
|
|
Pashi
|
2022-07-08 07:27:53
|
Unlike jxl, every jph is progressive due to wavelet design. Unlike jxl where it needs to be encoded in progressive mode and for some reason it is off by default it looks like in libjxl.
|
|
2022-07-08 07:28:42
|
No progressive decoding and responsive design if the person who encoded the jxl didn't pass a flag to the encoder
|
|
|
The_Decryptor
JXL has better hope of support in apps and browsers
|
|
2022-07-08 07:29:38
|
What do you mean?
|
|
|
|
_wb_
|
|
Pashi
Is there any advantage over jph aka HTJ2K?
|
|
2022-07-08 07:32:04
|
compression ๐
|
|
2022-07-08 07:32:46
|
htj2k is not compressing better than regular j2k, i.e. just marginally better than old jpeg
|
|
|
|
fab
|
2022-07-08 08:55:36
|
fun fact image quality is bette in
|
|
2022-07-08 08:55:38
|
-s 7 -d 0.622 --patches=0 --faster_decoding=1
|
|
2022-07-08 08:56:05
|
and default works unoptimaly
|
|
2022-07-08 09:03:44
|
it looks like avif
|
|
|
diskorduser
|
2022-07-08 10:59:21
|
Still no android gallery apps with jxl support.
|
|
|
Moritz Firsching
|
|
Pashi
No progressive decoding and responsive design if the person who encoded the jxl didn't pass a flag to the encoder
|
|
2022-07-08 12:09:06
|
even without special encoding flags, for most images without alpha it should be possible to get at least some form of progressive decoding, namely having the DC-image (8x8 downsampled) first, which then can be rendered with some nice smoothing. The goal is to get at least that functionality going inside chrome...
|
|
|
|
_wb_
|
2022-07-08 12:20:33
|
even with alpha, by default (when doing lossy) the alpha gets Squeezed so alpha data equivalent to 8x8 downsampled alpha is available at the same time as the vardct DC image.
|
|
|
|
BlueSwordM
|
|
Pashi
HT seems to have all the features of JXL due to it's wavelet design, progressive and region of interest coding, better speed and even good efficiency compared to JXL?
|
|
2022-07-08 05:06:40
|
JPEG-XL has much better coding performance, better psycho-visual performance, better lossless, can be obscenely fast at encoder and is just generally better.
Also, base progressive images are always present.
|
|
|
|
Pashi
|
2022-07-08 08:15:57
|
Is there a comparison that compares filesize between the two in lossless mode?
|
|
|
|
_wb_
|
2022-07-08 08:34:55
|
I think j2k is quite ok for photographic lossless, not sure how close htj2k gets to regular j2k in density though
|
|
2022-07-08 08:35:15
|
For non-photo, they're both worse than png
|
|
2022-07-08 08:42:42
|
|
|
2022-07-08 08:44:32
|
Here are some old test results comparing jxl to a bunch of other things, jpeg 2000 is one of them
|
|
2022-07-08 08:45:45
|
No idea where htj2k would be, but I suppose worse than regular j2k since it focuses on speed, not density
|
|
|
|
Pashi
|
2022-07-09 05:54:36
|
https://twitter.com/_michaeldsmith_/status/1478414688325046275?s=20&t=Zd8ToW2ZIQQ4rymoeYFYJQ
|
|
2022-07-09 05:54:53
|
Second image compares compression density
|
|
2022-07-09 05:55:21
|
And seems to show jph has smaller files somehow. No idea how that's possible but that's what this random graph says
|
|
2022-07-09 05:57:04
|
So is xyb-ms-ssim better than butteraugli?
|
|
2022-07-09 05:57:39
|
|
|
2022-07-09 05:59:04
|
The scaling factor of these graphs is poor
|
|
|
|
fab
|
|
Pashi
So is xyb-ms-ssim better than butteraugli?
|
|
2022-07-09 06:21:39
|
that's a lossy metric
|
|
|
|
Pashi
|
2022-07-09 06:22:45
|
I know, it's a separate topic, I was just wondering
|
|
|
|
_wb_
|
|
Pashi
And seems to show jph has smaller files somehow. No idea how that's possible but that's what this random graph says
|
|
2022-07-09 06:22:53
|
Well that graph only includes the fastest jxl encode settings
|
|
2022-07-09 06:25:40
|
Also the speed is measured in a weird way that might be measuring png decode time more than anything else
|
|
|
|
Pashi
|
2022-07-09 06:28:43
|
Someone ought to do a better, proper comparison.
|
|
|
|
BlueSwordM
|
|
Pashi
So is xyb-ms-ssim better than butteraugli?
|
|
2022-07-09 03:03:39
|
No, it's mainly used to be much faster.
|
|
|
|
_wb_
|
2022-07-09 04:06:14
|
well it's a bit hard to tell, if butteraugli would also be tuned to the training data I have, it will probably be better too
|
|
|
Dwedit
|
2022-07-10 04:10:30
|
I have a question. Is there a way to *accurately* estimate the size of a JXL->JPEG converted file (exact size in bytes) without actually running the transcode?
|
|
|
|
_wb_
|
2022-07-10 04:20:29
|
no, but I guess you could make a somewhat informed guess by just looking at the subsampling and quantization factors
|
|
|
|
Dwedit
|
2022-07-10 04:24:14
|
That's too bad, that would have been a really useful feature. I was thinking of making a user-mode filesystem that automatically transcoded JXL files to JPG on demand, but the filesize needs to be known ahead of time for that to work.
|
|
2022-07-10 04:24:55
|
Only way to make that work would be to mass decode everything, write down the sizes, then discard everything. Wasteful.
|
|
2022-07-10 04:26:45
|
Not knowing a good estimate of the JPEG size also messes with the transcoding API, which basically forces you to play a guessing game where you guess the final filesize, and if it's wrong, redo the whole process with a bigger buffer.
|
|
2022-07-10 04:27:50
|
Just 8 lousy bytes could have covered this use case.
|
|
|
|
Hello71
|
2022-07-10 05:01:30
|
8 bytes is a lot
|
|
2022-07-10 05:03:35
|
what is the point of this system anyways? i assume you mean specifically "JPEG bit-exact reconstruction" decoding, but in general transcoding can produce an image of any size
|
|
2022-07-10 05:06:46
|
if you are restricted to a very specific kind of file, then just store the size yourself somewhere when you encode it. otherwise, store it when you process the image for listing, in the image metadata or in some cache somewhere
|
|
2022-07-10 05:07:15
|
or expose a file type which has unknown size (pipe/device), or expose ppm files instead
|
|
2022-07-10 05:07:56
|
in general it seems much easier to modify the application to handle jxl itself. userspace filesystems on current operating systems are extremely slow
|
|
|
|
yurume
|
2022-07-10 05:09:46
|
not to mention that those 8 lousy bytes can't be trusted at all, making them effectively useless for your purpose anyway
|
|
|
|
_wb_
|
2022-07-10 05:56:26
|
Why not store the original size in your filesystem?
|
|
|
|
improver
|
2022-07-10 07:24:44
|
doing file system sounds a bit hard. you'd need to cache potentially arbitrary sized jpegs, for both read (with possibility to seek) and write (with possibility of jpegs that won't encode properly if the file is incomplete (it can be appended to become complete later) and jpegs which won't convert at all (jxl can't encode all of possible jpegs, though for popular usage it works really well))
|
|
2022-07-10 07:26:10
|
either way you will need something like sqlite database to store tracking data for all of that stuff, probably
|
|
2022-07-10 07:28:38
|
though i guess one could sort of get away without it, but yeah you can't know precise size of original jpeg from jxl, and even if you could, parsing jxls to get original size would likely be a lot slower than simple database lookup
|
|
|
|
_wb_
|
2022-07-10 07:30:39
|
You could add a custom isobmf box right after ftyp that has size of reconstructed jpeg, if you want, but probably you'll need filesystem metadata anyway so probably better to store it there
|
|
|
|
Traneptora
|
2022-07-10 09:48:52
|
Is libjxl's thread-safety documented anywhere?
|
|
2022-07-10 09:49:16
|
i.e. suppose I call JxlDecoderCreate from two different threads with two different JxlMemoryManagers, etc.
|
|
2022-07-10 09:49:28
|
is the thread-safety of operations like this one documented?
|
|
2022-07-10 09:49:47
|
I assume this shouldn't be an issue, but I want to prove it via documentation
|
|
|
|
veluca
|
2022-07-11 06:52:23
|
I'm not sure it's documented, but it is safe (I think by default we assume that nothing has the `strtok` level of thread unsafety)
|
|
|
|
Traneptora
|
|
veluca
I'm not sure it's documented, but it is safe (I think by default we assume that nothing has the `strtok` level of thread unsafety)
|
|
2022-07-11 05:30:15
|
I ask because I'm trying to mark the libjxl wrapper in FFmpeg as init-threadsafe, which requires the library calls to be re-entrant
|
|
2022-07-11 05:30:34
|
if this were documented it would be easier to merge the commit in so I don't have to prove it by inspecting the code
|
|
|
vertexgamer
|
2022-07-11 06:11:21
|
for some reason when i try to encode images with this name, the encode fail
|
|
2022-07-11 06:11:32
|
> `โlv2-2_jack_idle_a_0001`
|
|
2022-07-11 07:21:04
|
ok, it only happens when while the path gets written without ""
|
|
2022-07-11 07:22:39
|
for example: `C:\Users\user\Desktop\folder\Sprite\โlv2-2_jack_idle_a_0001.png` will crash, but `"C:\Users\user\Desktop\folder\Sprite\โlv2-2_jack_idle_a_0001.png"` works
|
|
|
w
|
2022-07-11 07:29:09
|
if user isnt your user is there a space in your user
|
|
|
|
Traneptora
|
|
vertexgamer
for example: `C:\Users\user\Desktop\folder\Sprite\โlv2-2_jack_idle_a_0001.png` will crash, but `"C:\Users\user\Desktop\folder\Sprite\โlv2-2_jack_idle_a_0001.png"` works
|
|
2022-07-11 08:22:54
|
`user` is presumably your username. does it have a space?
|
|
2022-07-11 08:23:13
|
same with `folder`, is there a space?
|
|
|
|
vertexgamer
|
|
Traneptora
`user` is presumably your username. does it have a space?
|
|
2022-07-11 08:23:25
|
nope, thers no space
|
|
|
|
Traneptora
|
2022-07-11 08:23:48
|
how does it *crash* exactly, what does it say?
|
|
|
|
vertexgamer
|
2022-07-11 08:27:15
|
the 2021 version of jxl from "releases" says "failed to read image"
|
|
2022-07-11 08:28:46
|
the Commit 13f8170 just says nothing
|
|
|
|
Traneptora
|
2022-07-11 08:33:13
|
It sounds to me like there's some sort of special character in `user` or `folder` that needs to be quoted for command prompt to interpret it properly
|
|
2022-07-11 08:33:29
|
since quoting it happens on the command-prompt level
|
|
2022-07-11 08:34:00
|
cjxl never sees the quotes
|
|
|
|
vertexgamer
|
2022-07-11 08:34:23
|
strange, don't have any sort of special character in my path
|
|
2022-07-11 08:34:34
|
let me check again to be sure
|
|
|
|
Traneptora
|
2022-07-11 08:34:37
|
if you pass `cjxl` a file that doesn't exist, it will just print its banner and exist. This should probably be fixed though
|
|
2022-07-11 08:34:56
|
so it looks to me like somehow it's not reading the file you're passing it and isntead reading another file path, which is invalid
|
|
|
|
vertexgamer
|
2022-07-11 08:35:51
|
nope everything is fine. So i guess it's a microsoft windowsโข fuck up, thx for your help!
|
|
2022-07-11 08:36:45
|
i guess i should switch to linux ๐
|
|
|
|
Traneptora
|
2022-07-11 08:36:51
|
or just quote
|
|
2022-07-11 08:36:53
|
ยฏ\_(ใ)_/ยฏ
|
|
|
vertexgamer
the Commit 13f8170 just says nothing
|
|
2022-07-11 08:41:09
|
https://github.com/libjxl/libjxl/issues/1592
|
|
|
|
fab
|
2022-07-12 03:55:15
|
-d 0.567 -s 8 -I 0.8 --gaborish=0 --epf=2 --faster_decoding=2 --target_intensity=300 --dots=0
|
|
2022-07-12 03:58:27
|
for %i in (C:\Users\Use\Documents\dd\*.png) do cjxl -d 0.567 -s 8 -I 0.8 --gaborish=0 --epf=2 --faster_decoding=2 --intensity_target=300 --dots=0 "%i" "%i.jxl"
|
|
2022-07-12 03:59:03
|
this is what i recommend
|
|
2022-07-12 07:02:24
|
|
|
2022-07-12 07:02:27
|
watch this
|
|
2022-07-13 12:07:54
|
djxl works
|
|
|
|
Traneptora
|
2022-07-14 02:53:25
|
How does JPEG XL handle reconstructed 4:2:0 JPEGs?
|
|
2022-07-14 02:53:33
|
I was under the impression JPEG XL doesn't support 4:2:0
|
|
|
|
The_Decryptor
|
2022-07-14 03:07:12
|
It does support them, I know the format supports "resampled" channels and I'm pretty sure it uses them to store the chroma planes
|
|
|
|
_wb_
|
2022-07-14 06:12:58
|
Well it does not support 4:2:0 XYB, only in case of YCbCr does the subsampling get signaled
|
|
2022-07-14 06:14:12
|
It does support subsampling _all_ XYB channels at the same time, but not e.g. only X and B but not Y.
|
|
2022-07-14 06:15:43
|
Extra channels can also be subsampled, e.g. depth maps are often only at 1:8 resolution and jxl can represent them like that.
|
|
|
|
Traneptora
|
|
_wb_
Well it does not support 4:2:0 XYB, only in case of YCbCr does the subsampling get signaled
|
|
2022-07-14 03:26:54
|
when they are subsampled, what upscale algorithm is used for the other planes, and what's the chroma location?
|
|
|
|
_wb_
|
2022-07-14 03:58:14
|
For YCbCr upsampling, the same algorithm and siting is used as in jpeg: siting is centered, one sample X becomes two samples 0.75 * X + 0.25 * (nearest neighbor that is not X), iirc
|
|
2022-07-14 03:59:13
|
For the other upsamplings, the fancy non-seperable upsampling method is used that we also use for DC upsampling in progressive previews.
|
|
|
|
Pashi
|
2022-07-15 12:41:27
|
Does JXL format support CMYK including reconstructing CMYK JPEGS? Is it a format limitation or an encoder limitation that it isn't supported?
|
|
|
|
improver
|
2022-07-15 02:00:36
|
iirc encoder for CYMK, and for jpeg CYMK spec+encoder
|
|
2022-07-15 02:03:13
|
i don't think you gonna see jxl handling of cymk jpegs anytime soon if ever, but for cymk coding it should b quite possible just idk if presently implemented
|
|
2022-07-15 02:04:00
|
kinda useless answers its sleep time
|
|
|
|
JendaLinda
|
2022-07-15 03:53:21
|
JXL is able to handle CMYK, although no encoder with CMYK support seems to be available at the moment.
CMYK JPEG is a non standard extension and lossless transcoding to JXL is not supported. I suppose the way how CMYK is implemented in JPEG is incompatible with JXL.
|
|
|
|
_wb_
|
2022-07-15 06:15:28
|
Jxl only supports 3 DCT channels
|
|
2022-07-15 06:16:01
|
So it could transcode CMY losslessly but the K would need to be decoded to pixels and encoded that way
|
|
|
|
JendaLinda
|
2022-07-15 07:18:13
|
I was experimenting with CMYK images a little and I found out that viewing CMYK images is not that easy. I supposed programs would use some sensible default, like sRGB is used for untagged RGB images. Some programs are indeed trying to emulate real ink colors, however every program uses slightly different ink colors. Other programs just used inverted RGB colors which look completely wrong. So I'm wondering how would JXL encoder handle CMYK input without color profile and what color profile would be used for decoded image.
|
|
|
|
_wb_
|
2022-07-15 08:31:09
|
It's impossible to encode a cmyk jxl without color profile
|
|
2022-07-15 08:32:23
|
The only way to have a valid cmyk jxl, is to have a cmyk icc profile and a kBlack channel.
|
|
|
|
JendaLinda
|
2022-07-15 10:20:48
|
So what I did I've created a four channel raw bitmap file and converted it using raw2tiff program to TIFF with the photometric interpretation set to CMYK. So it's a CMYK TIFF file, it has C, M, Y, K channels, but it doesn't have any color profile attached to it. So if I understand correctly, this is not enough and I would have to add a suitable color profile to be able to encode it using JXL.
|
|
|
|
The_Decryptor
|
2022-07-15 10:24:27
|
There's no standard CMYK profile since it's intended for printing, and different regions use different inks and such
|
|
2022-07-15 10:26:39
|
So there might be profiles for specific printers, and different regions might agree on supporting a specific profile, because otherwise there's no way to know e.g. which specific cyan is needed for "100% cyan" in a file
|
|
2022-07-15 10:27:37
|
I think the term is "output referred", it varies on a specific output device
|
|
|
|
JendaLinda
|
2022-07-15 10:40:48
|
That makes sense. That's what I've observed, displaying my CMYk file on a computer screen is total chaos.
Anyway, I was just fooling around in unknown territory. It's fun to experiment and learn something new.
|
|
|
|
_wb_
|
2022-07-15 11:03:14
|
Yes, cmyk without profile doesn't make much sense since there is not really any 'reasonable default' like sRGB for rgb images.
|
|
2022-07-15 11:08:04
|
You could assume US Web Coated SWOP v2 or something, but this stuff is more based on regional standards and evolves with new printing technologies. We decided not to make an enum for cmyk but just require an icc profile to be used for cmyk.
|
|
|
|
Traneptora
|
2022-07-15 05:42:24
|
Is there a known workaround for https://github.com/google/highway/issues/836
|
|
|
_wb_
For YCbCr upsampling, the same algorithm and siting is used as in jpeg: siting is centered, one sample X becomes two samples 0.75 * X + 0.25 * (nearest neighbor that is not X), iirc
|
|
2022-07-15 05:43:10
|
JPEG doesn't specify and algorithm for upsampling iirc
|
|
2022-07-15 05:43:16
|
it just outputs 4:2:0 YCbCr
|
|
2022-07-15 05:43:30
|
at least some decoders do, like lavc's
|
|
|
|
_wb_
|
|
Traneptora
JPEG doesn't specify and algorithm for upsampling iirc
|
|
2022-07-15 05:59:42
|
The original jpeg spec doesn't, but after jfif and libjpeg became a thing, things eventually converged de facto to something that was then standardized later in jpeg xt
|
|
2022-07-15 06:00:36
|
Point is: jxl upsamples ycbcr exactly like all common jpeg decoders do it. Other things it upsamples with fancier upsampling methods.
|
|
|
|
Traneptora
|
2022-07-15 06:01:20
|
anyway, gcc+aarch64 fails to build master libjxl
|
|
2022-07-15 06:01:22
|
with bundled highway
|
|
2022-07-15 06:01:28
|
is there a simple fix?
|
|
2022-07-15 06:01:52
|
or workaround, really
|
|
2022-07-15 06:01:57
|
without just downgrading the bundled version
|
|
|
|
Pashi
|
2022-07-18 04:42:34
|
Common jpeg decoders have horrible looking chroma upsampling that looks weird and blocky
|
|
2022-07-18 04:42:43
|
Like nearest neighbor
|
|
|
|
Traneptora
|
2022-07-18 07:02:19
|
turbojpeg does at least, yea
|
|
|
|
_wb_
|
2022-07-18 08:07:40
|
Does any jpeg decoder in common use actually do NN chroma upsampling?
|
|
2022-07-18 08:08:56
|
Iirc Apple's heic implementation (used to) do that for heic, not sure if it still does
|
|
|
argo
|
2022-07-18 10:02:22
|
Thoughts on guetzli for JPGs?
|
|
|
|
spider-mario
|
2022-07-18 10:10:29
|
better be patient
|
|
|
Fox Wizard
|
2022-07-19 02:20:02
|
I didn't like anything about it tbh. At high quality it blurs and mozjpeg produced an in my opinion better looking image every time. Also... it's slow af <:KekDog:884736660376535040>
|
|
|
WoofinaS
|
2022-07-19 01:45:48
|
If I remember correctly mozjpeg gave me a image with a smaller `butteraugli` distance then guetzli quite a few times.
|
|
|
|
_wb_
|
2022-07-19 04:54:50
|
guetzli and mozjpeg have very different approaches as to how to encode
|
|
2022-07-19 04:57:58
|
afaik, guetzli basically uses q100 jpegs with 'poor man's adaptive quantization' guided by butteraugli, i.e. effectively doing more quantization selectively on blocks that can take it. (but I wasn't involved in guetzli at all and some people here were, so they can probably explain how it works a lot better than me)
|
|
2022-07-19 05:02:26
|
while afaik mozjpeg is basically a regular jpeg encoder that just applies some extra tricks like deadzone quantization, exaggerating black/white contrast so dct artifacts end up in the clamped range, using better default quant tables, etc. And then it spends some extra effort on entropy coding, trying different progressive scan scripts to see which one compresses best, etc. (but I also wasn't involved in mozjpeg at all and some people here are/were, so they can probably explain it a lot better than me)
|
|
|
daxpedda
|
2022-07-22 01:04:27
|
Hi!
I would like to write a JPEG XL en/decoder in Rust and wanted to start by looking at the specification. Sadly, because of ISO standardization, the spec is now behind a significant paywall.
I found this https://arxiv.org/pdf/1908.03565.pdf.
Is there a newer public committee draft? Preferably I would like to have the actual final committee draft.
|
|
|
|
yurume
|
2022-07-22 01:18:32
|
that draft is too old to be useful today. the bitstream format has been fixed and formalized in ISO/IEC 18181-1, which is unfortunately behind the paywall.
|
|
2022-07-22 01:20:04
|
another problem is that ISO/IEC 18181-1 as it stands today has a lot of bugs compared to libjxl and it's believed that the specification alone can't produce an independent implementation right now.
|
|
2022-07-22 01:21:44
|
to my knowledge there have been four independent reimplementation efforts: jxl-rs (by veluca), jxsml (mine), thebombzen's libavcodec adaptation, and I think there is another one in the planning stage (thebombzen knows more about this, I don't)
|
|
2022-07-22 01:22:45
|
among them jxsml is the most closest, able to handle most Modular images and some VarDCT images, provided that they are not progressive and not animated
|
|
|
|
veluca
|
2022-07-22 01:22:52
|
> jxl-rs (by veluca),
not *quite* indepedent ๐
|
|
|
|
yurume
|
2022-07-22 01:23:11
|
as independent as "indie" game? :p
|
|
|
|
veluca
|
2022-07-22 01:23:59
|
that said, if anybody feels like picking up where jxl-rs left things, I'd be happy to chat with them
|
|
|
|
yurume
|
2022-07-22 01:24:04
|
anyway, so that's what you can expect from the specification right now. jxsml *is* based on the specification but also libjxl due to the spec's bugs.
|
|
2022-07-22 01:24:32
|
yeah, there have been unofficial drafts silently circulating ๐
|
|
2022-07-22 01:25:28
|
if you have a copy of ISO/IEC 18181-1 but not those drafts, another possibility is to check jxsml comments for known issues (I tried to mark all of them)
|
|
2022-07-22 01:26:38
|
by the way I do have an intention of making a Rust version of jxsml, if you are interested in this I can help you
|
|
|
|
veluca
|
2022-07-22 01:27:05
|
If you want to continue from jxl-rs (or take over the repo) feel free to ping me
|
|
|
|
Jim
|
2022-07-22 01:29:34
|
<:Agree:805404376658608138> Consider contributing to the existing rust implementations. There seem to be a few of them floating around and will be easier working with jxl devs who are more familiar with libjxl
|
|
|
|
yurume
|
2022-07-22 01:31:05
|
are there any such implementations past the image header?
|
|
2022-07-22 01:31:51
|
to my knowledge most current Rust implementations focused on extracting the basic information (like dimensions)
|
|
|
|
veluca
|
2022-07-22 01:32:04
|
Jxl-rs has some entropy coding too iirc
|
|
|
|
yurume
|
2022-07-22 01:32:16
|
except for jxl-rs, yeah
|
|
|
|
daxpedda
|
|
yurume
another problem is that ISO/IEC 18181-1 as it stands today has a lot of bugs compared to libjxl and it's believed that the specification alone can't produce an independent implementation right now.
|
|
2022-07-22 01:34:13
|
uhm, I don't know much about the standardization process, but shouldn't that be amended?
|
|
|
yurume
to my knowledge there have been four independent reimplementation efforts: jxl-rs (by veluca), jxsml (mine), thebombzen's libavcodec adaptation, and I think there is another one in the planning stage (thebombzen knows more about this, I don't)
|
|
2022-07-22 01:34:34
|
I'm unable to find anything on jxsml
|
|
|
|
yurume
|
2022-07-22 01:34:52
|
yes, they should be fixed in the future as I'm giving feedbacks to the spec editor (wb here)
|
|
2022-07-22 01:35:50
|
thank you for the link. I intend to make an actual repo when it is able to handle most VarDCT images, so please bear with that right now...
|
|
2022-07-22 01:36:20
|
(I frequently post about jxsml on <#794206087879852106> too)
|
|
|
|
daxpedda
|
|
yurume
by the way I do have an intention of making a Rust version of jxsml, if you are interested in this I can help you
|
|
2022-07-22 01:36:31
|
you mean you intend to rewrite it into Rust?
|
|
|
|
yurume
|
2022-07-22 01:36:42
|
I intend to have two versions of jxsml, one in C and one in Rust
|
|
|
|
daxpedda
|
2022-07-22 01:37:10
|
is the intention to have a decoder only, or also an encoder?
|
|
|
|
yurume
|
2022-07-22 01:37:31
|
the encoder is a much harder business and best left to libjxl
|
|
2022-07-22 01:37:37
|
so jxsml is strictly a decoder only
|
|
2022-07-22 01:38:37
|
one of the best decisions made by libjxl is to make libjxl *the* library for encoding jxl files, not just a reference implementation
|
|
2022-07-22 01:38:51
|
and I think I can't simply compete with that ๐
|
|
2022-07-22 01:39:38
|
(to be fair one can make a low-complexity subset encoder like fjxl, but the full-featured encoder is unlikely)
|
|
|
|
daxpedda
|
2022-07-22 01:39:52
|
fair enough
|
|
|
|
_wb_
|
2022-07-22 01:39:53
|
Next week is jpeg meeting week. I will propose there to delay the CD ballot for the second edition until we've ironed out the bugs yurume and others have found / are finding
|
|
|
|
daxpedda
|
2022-07-22 01:40:18
|
is there any relation between jxsml and jxl-rs?
|
|
|
|
yurume
|
2022-07-22 01:40:27
|
none :p
|
|
|
|
daxpedda
|
2022-07-22 01:40:55
|
do you plan to re-use jxl-rs for the Rust port?
|
|
|
|
yurume
|
2022-07-22 01:41:56
|
I don't think so, because jxl-rs only has a field parser and an entropy decoder I think
|
|
|
|
daxpedda
|
2022-07-22 01:42:40
|
alright, so whats the plan? ๐
when can I start contributing?
|
|
|
|
yurume
|
2022-07-22 01:43:32
|
currently I'm working on a major redesign of the input handling, which is probably the biggest task to make it a proper library
|
|
2022-07-22 01:45:29
|
as per the feature set, jxsml currently lacks: progressive handling (including Squeeze transform), inverse Hornuss transform, YCbCr, upscaling, post filters (epf/gab), extra image features, animation and blending
|
|
2022-07-22 01:45:54
|
and its VarDCT implementation has some bugs I'm yet to fully figure out
|
|
2022-07-22 01:46:12
|
and it lacks testing (haha)
|
|
|
|
_wb_
|
2022-07-22 01:46:42
|
Still, getting close to being able to decode simple images with default libjxl encode settings
|
|
|
|
yurume
|
2022-07-22 01:47:16
|
that's why I plan to release it in public once VarDCT has been sorted out, the remainder will take another month or two and it seems unwise to delay the release by now
|
|
|
|
_wb_
|
2022-07-22 01:47:53
|
YCbCr itself is trivial, with upsampling it might be a bit trickier
|
|
2022-07-22 01:48:10
|
Both are needed to decode recompressed jpegs though
|
|
|
|
daxpedda
|
2022-07-22 01:48:29
|
so how do you do it?
do you just take `libjxl` as the specification? or do you use the specification and then correct it by comparing to the reference implementation?
|
|
|
|
yurume
|
2022-07-22 01:48:39
|
more like the latter
|
|
2022-07-22 01:49:03
|
I implement according to the spec as much as possible and analyze the discrepancy from libjxl
|
|
|
|
_wb_
|
2022-07-22 01:49:03
|
Take the spec and look at libjxl when you get stuck
|
|
2022-07-22 01:49:42
|
Getting stuck means either the spec is unclear or it is wrong
|
|
|
|
daxpedda
|
2022-07-22 01:50:28
|
okay so in any case I can't do much unless I get my hands on the specification somehow
unless I just really reverse-engineer `libjxl`
|
|
|
|
_wb_
|
2022-07-22 01:50:28
|
(or libjxl is wrong)
|
|
2022-07-22 01:51:05
|
I am on holiday now but I will probably put a draft here next week or so
|
|
|
|
daxpedda
|
2022-07-22 01:52:21
|
thanks, that would be much appreciated
|
|
|
|
_wb_
|
2022-07-22 01:52:46
|
Probably if you scroll up enough in one of these channels you'll find an older draft somewhere
|
|
2022-07-22 01:53:51
|
|
|
2022-07-22 01:54:03
|
This is a version I have on my phone
|
|
|
|
daxpedda
|
|
yurume
that's why I plan to release it in public once VarDCT has been sorted out, the remainder will take another month or two and it seems unwise to delay the release by now
|
|
2022-07-22 01:54:15
|
I guess I will wait then, would you mind pinging me when you release? I guess we can talk about the details of porting to Rust then!
does that sound alright to you?
|
|
|
|
yurume
|
2022-07-22 01:54:41
|
you can also take a look at the current jxsml to plan out ๐
|
|
|
|
daxpedda
|
|
_wb_
|
|
2022-07-22 01:55:55
|
oh wow, that's pretty new, thank, that should do the trick!
|
|
|
|
_wb_
|
2022-07-22 01:56:49
|
There are still known bugs in that one, but it's at least better than the version from the May JPEG meeting
|
|
|
|
daxpedda
|
|
yurume
you can also take a look at the current jxsml to plan out ๐
|
|
2022-07-22 01:58:50
|
I mean I can't do much without a repo so you can actually take a look
Would you mind if I open a repo, invite you and start with some basic architecture you can take a look at? (or you hosting the repo is fine too)
|
|
|
|
_wb_
|
|
daxpedda
Hi!
I would like to write a JPEG XL en/decoder in Rust and wanted to start by looking at the specification. Sadly, because of ISO standardization, the spec is now behind a significant paywall.
I found this https://arxiv.org/pdf/1908.03565.pdf.
Is there a newer public committee draft? Preferably I would like to have the actual final committee draft.
|
|
2022-07-22 01:59:06
|
Compared to that very early committee draft, it's very different
|
|
|
|
yurume
|
2022-07-22 01:59:28
|
hmm, maybe I can just make an empty repo for wiki and discussion
|
|
|
|
_wb_
|
2022-07-22 02:00:01
|
The Independent JPEG XL Group
|
|
|
|
daxpedda
|
2022-07-22 02:00:57
|
that's a great idea, but I meant an actual code repo ๐ (even if it starts empty)
|
|
|
|
_wb_
|
2022-07-22 02:01:10
|
IJG made the first jpeg implementation and they implemented only part of the spec, thus creating the de facto jpeg we have now
|
|
|
|
yurume
|
2022-07-22 02:01:26
|
there has been also an early feedback that "jxsml" too looks like a markup language or programming language (SML), so I may change the name haha
|
|
2022-07-22 02:01:39
|
(the current contender is "lwjxl" for lightweight jxl)
|
|
|
|
_wb_
|
2022-07-22 02:01:59
|
libjxl-turbo
|
|
|
|
yurume
|
2022-07-22 02:02:11
|
I can't promise that XD
|
|
|
|
_wb_
|
2022-07-22 02:02:19
|
Not a good name since libjxl is already simdified etc
|
|
|
|
daxpedda
|
2022-07-22 02:02:49
|
`tiny-jxl`
|
|
|
|
_wb_
|
2022-07-22 02:02:52
|
lwjxl sgtm
|
|
|
|
yurume
|
2022-07-22 02:02:58
|
it's very hard to think of the name that has "j", "x" and "l" as a subsequence and something reasonable
|
|
2022-07-22 02:03:14
|
and short enough
|
|
|
|
_wb_
|
2022-07-22 02:03:19
|
jixel
|
|
|
|
yurume
|
2022-07-22 02:03:49
|
(favorably at most 5 letters long, 6 is acceptable)
|
|
2022-07-22 02:04:08
|
that's why I avoided tinyjxl or similar for now
|
|
|
|
_wb_
|
2022-07-22 02:05:11
|
jxlid
|
|
2022-07-22 02:05:20
|
(independent decoder)
|
|
|
|
yurume
|
2022-07-22 02:06:02
|
another contender was "jxled", which can read as "jxl encoder and decoder" and pronounced jay-sled; would be okay if I do have an encoder implementation...
|
|
2022-07-22 02:06:39
|
anyway enough bikeshedding, I'll think more about the public development
|
|
|
|
daxpedda
|
2022-07-22 02:07:42
|
๐
looking forward!
|
|
2022-07-22 02:13:41
|
<@794205442175402004> by any chance would you also have part 2 and 3? the file says part 1-3 but as far as I can see it's only part 1
|
|
|
|
_wb_
|
2022-07-22 02:31:58
|
Oh it's just the third time I saved a file with that name on my phone, lol
|
|
2022-07-22 02:33:39
|
Part 2 is file format (the isobmf boxes thingie), but you don't really need that unless you want to reconstruct jpegs. Otherwise you can just ignore all boxes except the jxlc/jxlp ones (or make a decoder that only decodes naked codestreams, which are also valid files)
|
|
2022-07-22 02:34:44
|
Part 3 is conformance testing, but that's basically just boilerplate explaining the structure of the conformance repo
|
|
2022-07-22 02:35:05
|
And Part 4 is just a zipfile containing a snapshot of libjxl, lol
|
|
2022-07-22 02:35:31
|
Part 1 is all you need to make a decoder
|
|
|
|
daxpedda
|
|
_wb_
Part 2 is file format (the isobmf boxes thingie), but you don't really need that unless you want to reconstruct jpegs. Otherwise you can just ignore all boxes except the jxlc/jxlp ones (or make a decoder that only decodes naked codestreams, which are also valid files)
|
|
2022-07-22 02:42:48
|
I don't know yet what "isobmf boxes thingie" is, but I guess if naked codestreams are valid files then thats good enough for now
just looked at the conformance repo, should be fine
|
|
2022-07-22 02:43:34
|
do most of the tests in the conformance repo use naked codestreams?
|
|
|
|
yurume
|
2022-07-22 02:54:36
|
regarding 18181-2: I think `jxll` can be anywhere in the container, including at the very end of the file, right?
|
|
2022-07-22 02:56:15
|
my interpretation of this is that, the maximum codestream level is only the user's discretion and there is no sort of "negotiation" possible here
|
|
|
|
_wb_
|
2022-07-22 02:56:46
|
We're changing it so that if there's a jxll, it has to be right in the beginning
|
|
|
|
yurume
|
|
|
_wb_
|
2022-07-22 02:57:08
|
And if there is none, then it's implicitly level 5
|
|
|
|
yurume
|
2022-07-22 02:58:13
|
yeah, so in my mind there were two potential uses of `jxll`: to declare the maximum amount (so to say) of resource to decode this file, and to abort the decoding as early as possible when the level is too high
|
|
2022-07-22 02:58:24
|
if `jxll` can happen at the end of the file the former use is gone
|
|
|
|
_wb_
|
2022-07-22 02:58:34
|
So there is no way to signal "no level declared" - but I guess you could signal level 255 if you want to go outside level 10 but still within the spec (I don't really see a use case for that though, level 10 is pretty much unlimited already)
|
|
|
|
Pashi
|
2022-07-22 03:24:33
|
sjxl
|
|
2022-07-22 03:24:42
|
Suckless :)
|
|
2022-07-22 03:25:17
|
Or Simple
|
|
2022-07-22 03:26:03
|
For an easy to use small self contained C lib with no dependencies
|
|
|
|
190n
|
2022-07-22 04:11:43
|
suckless <:monkaMega:809252622900789269>
|
|
|
|
Jim
|
2022-07-22 04:14:53
|
speedy
|
|
|
|
Traneptora
|
2022-07-24 07:34:49
|
ugh
|
|
2022-07-24 07:34:50
|
https://github.com/libjxl/libjxl/commit/524427c4b544f86a4f57629cef9837e9452ca515
|
|
2022-07-24 07:34:53
|
you broke the ABI
|
|
2022-07-24 07:35:47
|
if you passed `-1` as an `int32_t`, the library now treats it as an `int64_t` which on a little-endian system is only guaranteed to have the bottom 32 bits filled with 1s
|
|
2022-07-24 07:35:59
|
so as a result the `int64_t` that the library receives will not be
|
|
2022-07-24 07:36:25
|
will not be equal to -1
|
|
2022-07-24 07:36:29
|
and then it'll error out
|
|
|
|
veluca
|
2022-07-24 07:38:10
|
well yeah, encoder ABI is not stable between 0.6 and 0.7
|
|
|
|
Traneptora
|
2022-07-24 07:38:57
|
well the library has been versioned at 0.7 for a long time
|
|
2022-07-24 07:39:08
|
so I figured the ABI would match the 0.7 release that came out 3 days ago
|
|
2022-07-24 07:40:05
|
or is 0.7 not released yet?
|
|
2022-07-24 07:41:09
|
what is our timeline on that? people want a stable ABI
|
|
|
|
veluca
|
2022-07-24 08:02:57
|
0.7 is in RC IIRC
|
|
|
|
Traneptora
|
2022-07-24 08:03:04
|
ah okay
|
|
|
|
veluca
|
2022-07-24 08:03:09
|
but I can't see a stabilization branch for it
|
|
2022-07-24 08:03:30
|
so not sure if something went wrong
|
|
|
|
_wb_
|
2022-07-25 08:19:01
|
current draft of 18181-1 Ed.2
|
|
|
|
Traneptora
|
2022-07-26 08:09:09
|
does this include all of the bugfixes found be me and/or yurume so far?
|
|
|
|
_wb_
|
2022-07-26 01:08:44
|
not all, but quite a few
|
|
2022-07-26 01:08:55
|
i still have a backlog of bug reports to go through ๐
|
|
2022-07-26 01:09:41
|
current proposal is to finalize the CD by the next JPEG meeting (October) and only submit it to ballot then
|
|
|
|
fab
|
2022-07-27 03:32:31
|
can you send brotlidec.dll
|
|
2022-07-27 03:32:36
|
cjxl fails
|
|
2022-07-27 03:34:17
|
please
|
|
|
|
JendaLinda
|
2022-07-27 03:45:49
|
I have no idea where are we supposed to get brotli dlls but I'm using those bundled with Gimp and cjxl is not complaining.
|
|
|
|
fab
|
2022-07-27 03:46:14
|
send hee
|
|
2022-07-27 03:46:40
|
please send now
|
|
2022-07-27 03:47:09
|
where to copy in path programs or system32
|
|
|
|
JendaLinda
|
2022-07-27 03:48:55
|
Just put them in the same folder as the program.
|
|
|
|
fab
|
2022-07-27 03:52:31
|
it woks
|
|
2022-07-27 04:27:18
|
|
|
|
|
diskorduser
|
2022-07-27 04:47:35
|
what is this? h266 encoder?
|
|
|
|
Traneptora
|
2022-07-27 08:10:28
|
does anyone know what this means when compiling libjxl?
|
|
2022-07-27 08:10:31
|
`WARNING: CPU supports 1a00 but software requires 2000000000000000`
|
|
|
32 Bit Link
|
|
diskorduser
what is this? h266 encoder?
|
|
2022-07-28 02:34:27
|
Yeah
uvg266 is UltraVideo's (Kvazarr) VVC encoder
|
|
|
|
190n
|
|
Traneptora
`WARNING: CPU supports 1a00 but software requires 2000000000000000`
|
|
2022-07-28 07:13:55
|
i get that warning when compiling and even running but it still seems to work fine
|
|
|
|
Jyrki Alakuijala
|
2022-07-29 11:31:22
|
do we want to have a separate channel for #jxsml ?
|
|