JPEG XL

Info

rules 57
github 35276
reddit 647

JPEG XL

tools 4225
website 1655
adoption 20712
image-compression-forum 0

General chat

welcome 3810
introduce-yourself 291
color 1414
photography 3435
other-codecs 23765
on-topic 24923
off-topic 22701

Voice Channels

General 2147

Archived

bot-spam 4380

jxl

Anything JPEG XL related

DZgas Ж
2023-03-20 08:15:09
oh its you <@206628065147748352>
Traneptora try this
2023-03-20 08:19:22
bad
2023-03-20 08:23:13
but. in general, this is the only noticeable error (without a thorough check) otherwise everything is fine... but these dots are very noticeable.
Traneptora
DZgas Ж bad
2023-03-20 08:23:21
I have noticed this, and haven't quite figured out a solution yet
DZgas Ж
Traneptora I have noticed this, and haven't quite figured out a solution yet
2023-03-20 08:24:06
I'll be waiting. and make a build in Releases.
Traneptora
2023-03-20 08:24:42
ill put one up soon
fab
2023-03-20 08:30:37
2023-03-20 08:30:55
Dzgas be like
Traneptora
DZgas Ж I'll be waiting. and make a build in Releases.
2023-03-20 10:38:44
https://github.com/thebombzen/hydrium/releases/tag/0.1.0
DZgas Ж
Traneptora https://github.com/thebombzen/hydrium/releases/tag/0.1.0
2023-03-21 12:26:32
👍 👍
jonnyawsom3
2023-03-22 09:10:11
Well damn... I had a folder of about 500 screenshots from a VR game and just thought "Oh, that'll be perfect for JXL!" Turns out I had converted everything into jpegs using ffmpeg of all things... Presumably because I already had a bat file to convert folders to different types and didn't consider standalone encoders would be any different, or that JXL existed... Thankfully I at least set it to the maximum quality if I do that now, but still annoying I screwed myself over
username
2023-03-22 09:10:59
atleast you can do JXL transcoding
jonnyawsom3
2023-03-22 09:11:07
Can still save 20~% at least, but there's quite a lot of blocking in them now compared to the PNGs they were
2023-03-22 09:11:09
Yeah
username
2023-03-22 09:12:30
also speaking of the JPEG transcoding feature you can actually get very slightly more space savings if you put the jpegs through something like jpegoptim first and also set the effort to like 8 or 9 in libjxl
2023-03-22 09:13:05
binaries here: https://github.com/XhmikosR/jpegoptim-windows
jonnyawsom3
2023-03-22 09:13:34
Yeah, I did notice the effort setting *does* actually effect transcoding, makes me wonder if defaulting to 9 in that case might be good, since it seems near instant no matter what I use
username
2023-03-22 09:15:35
me and a friend tested it on a 7/6 MB jpeg that is 10408 x 14693 and it was definitely not instant
2023-03-22 09:15:57
although it did get it down to like 2MB
2023-03-22 09:16:14
which is a amazing amount of space savings
jonnyawsom3
2023-03-22 09:16:36
A tad more than 25%
username
2023-03-22 09:17:16
jpegoptim/jpegtran only got it down from 7.9MB to 6.2MB
2023-03-22 09:18:36
also the image itself was digital art so lossless webp got it down to 2MB
2023-03-22 09:19:06
if you are curious the image is this https://d3gz42uwgl1r1y.cloudfront.net/in/indecentfellow/submission/2018/11/d5ca1d7d1c947166392044279eb5013f/2500x1500.jpg
2023-03-22 09:19:50
not sure why it's a jpeg at max quality and not something lossless but atleast it's a interesting test case
jonnyawsom3
2023-03-22 09:19:51
Oh god I found a jpeg normal map I made years ago out of naivety
2023-03-22 09:21:47
Huh... Out of all the images I certainly wasn't expecting to see that character turn up
2023-03-22 09:22:40
And yeah, I've seen a few cases where the image is extremely synthetic and shared as a high resolution jpeg instead
Oh god I found a jpeg normal map I made years ago out of naivety
2023-03-22 09:24:35
Oh, huh... I found the source files and apparently a model got bundled with jpegs only, odd
username
2023-03-22 09:25:51
oh yeah some websites and places where you can download models/textures and stuff only ship jpeg files
Traneptora
2023-03-22 09:28:57
turns out right shifting negative numbers is bad idea
2023-03-22 09:29:02
who knew
2023-03-22 09:29:21
caused me a few bugs
2023-03-22 09:33:13
What's an ideal ssimulacra2 score for cameras?
2023-03-22 09:37:43
currently libhydrium hits around 90, dunno if that's ideal
_wb_
2023-03-22 09:48:38
around 90 should be visually lossless
Traneptora
2023-03-22 09:49:37
so that's a good target
2023-03-22 09:51:12
I'm also wondering if there's value in using 16x16 varblocks instead of 8x8 varblocks
2023-03-22 09:51:25
for memory it won't affect much since I discard the OG values after I run the forward DCT
jonnyawsom3
username if you are curious the image is this https://d3gz42uwgl1r1y.cloudfront.net/in/indecentfellow/submission/2018/11/d5ca1d7d1c947166392044279eb5013f/2500x1500.jpg
2023-03-22 09:59:54
I ran it through myself and yeah, takes about half a minute, although when I tried to do it lossless JXL it immediately went to 11GB RAM usage so I had to stop it. So transcoding is definitely pretty efficient
Demez
username if you are curious the image is this https://d3gz42uwgl1r1y.cloudfront.net/in/indecentfellow/submission/2018/11/d5ca1d7d1c947166392044279eb5013f/2500x1500.jpg
2023-03-22 10:01:32
this as a lossless jxl is 1.83 MB, and 2.05 MB as lossless webp
jonnyawsom3
2023-03-22 10:11:55
Using the original image and not the resized one I assume?
username
2023-03-22 10:29:17
oh whoops I sent the wrong image
username if you are curious the image is this https://d3gz42uwgl1r1y.cloudfront.net/in/indecentfellow/submission/2018/11/d5ca1d7d1c947166392044279eb5013f/2500x1500.jpg
2023-03-22 10:29:41
this is the actual full size image https://d3gz42uwgl1r1y.cloudfront.net/in/indecentfellow/submission/2018/11/d5ca1d7d1c947166392044279eb5013f.jpg
2023-03-22 10:29:59
you can tell because discord fails to embed it
Using the original image and not the resized one I assume?
2023-03-22 10:30:07
yes the original full size image
jonnyawsom3
2023-03-22 10:30:56
Ahh, I did check if it had a rezize portion of the URL but apparently missed it, I used a reverse image search to get the full size anyway
Traneptora
2023-03-22 11:34:25
<@179701849576833024> for some reason, no matter how I try to cluster the HF coefficients, it seems like using one grand cluster outperforms everything else
2023-03-22 11:34:36
this sounds weird...
2023-03-22 11:35:47
I find this odd
2023-03-22 11:36:11
like, the best cluster map I've found is literally just `const uint8_t map[7425] = { 0 };`
Jyrki Alakuijala
Traneptora then you tag the image as YCbCr
2023-03-23 01:09:28
I thought this would be possible -- I tried finding it myself manually, then used an optimization system to do it, couldn't get it right -- I concluded it was too difficult and possibly impossible (several other people tried doing it or otherwise considered it not worth trying or impossible)
Traneptora
2023-03-23 01:09:55
ah, that's unfortunate
Jyrki Alakuijala
DZgas Ж bad
2023-03-23 01:13:03
this is rather ugly and new to me -- I have a theory what it could be but need to check
DZgas Ж Can you run tests with this image? it really work on any old JPEG XL version, even by 0.6.0, on any quality q90 q80 q70 q60 q50 - it looks as it should. but on the last build it starts to break even starting with the quality of q95
2023-03-23 01:18:00
will do
DZgas Ж
2023-03-23 01:18:31
👍 👍
Demez
username this is the actual full size image https://d3gz42uwgl1r1y.cloudfront.net/in/indecentfellow/submission/2018/11/d5ca1d7d1c947166392044279eb5013f.jpg
2023-03-23 01:18:35
also i think this image can crash cjxl lossy iirc
Jyrki Alakuijala
Traneptora and we've got a working image!
2023-03-23 01:20:00
could it be that there is a minor AC scaling issue since all AC looks a bit overcompressed in range
Traneptora
Jyrki Alakuijala could it be that there is a minor AC scaling issue since all AC looks a bit overcompressed in range
2023-03-23 01:20:41
found a number of bugs that have since been fixed in this
Jyrki Alakuijala this is rather ugly and new to me -- I have a theory what it could be but need to check
2023-03-23 01:21:47
this is caused by a bug in hydrium's DCT where `x >> 18` is not the same thing as `x / (1 << 18)` for negative x
2023-03-23 01:21:52
since been fixed
Jyrki Alakuijala
_wb_ gaborish crosses group boundaries but not frame boundaries, so you might get some seams if you do gaborish while encoding one frame per group
2023-03-23 01:22:13
you can mitigate by turning up adaptive quantization a few steps at boundaries
Traneptora this is caused by a bug in hydrium's DCT where `x >> 18` is not the same thing as `x / (1 << 18)` for negative x
2023-03-23 01:24:55
ah, it was from hydrium -- could we create a separate grouping for hydrium to bring more clarity?
Traneptora
2023-03-23 01:26:09
sure
veluca
Traneptora <@179701849576833024> for some reason, no matter how I try to cluster the HF coefficients, it seems like using one grand cluster outperforms everything else
2023-03-23 06:32:16
Very odd
Traneptora
Jyrki Alakuijala you can mitigate by turning up adaptive quantization a few steps at boundaries
2023-03-23 08:08:04
when you say adaptive quantization are you referring to the HF Multiplier?
veluca
veluca Very odd
2023-03-23 11:19:06
What did you try?
Traneptora
veluca What did you try?
2023-03-23 05:14:30
I tried one large cluster, I tried setting the nonzeroes to cluster=0 and everything else to cluster=1 I tried setting each set of block contexts to be their own clusters, I tried setting each histogram context to be their own cluster
veluca
2023-03-23 05:16:28
The second one should definitely work better
2023-03-23 05:16:41
Check your histogram code? 😛
Traneptora
2023-03-23 05:16:55
the streams are verifying
2023-03-23 05:18:04
for example, this produced a much larger file
2023-03-23 05:18:17
```c uint8_t map[7425]; for (int k = 0; k < 15; k++) { memset(map + 37 * k, k, 37); memset(map + 555 + 458 * k, k + 15, 458); } ```
2023-03-23 05:20:13
I feel like this should work better
2023-03-23 05:21:16
but it's not
veluca
2023-03-23 05:24:09
How are building histograms on the encoder side?
Traneptora
veluca How are building histograms on the encoder side?
2023-03-23 05:25:28
```c for (size_t k = 0; k < stream->alphabet_size; k++) total += frequencies[k]; for (size_t k = 0; k < stream->alphabet_size; k++) { if (!frequencies[k]) continue; frequencies[k] = ((frequencies[k] << 12) / total) & 0xFFFF; if (!frequencies[k]) frequencies[k] = 1; } ```
2023-03-23 05:25:31
this, essentially
2023-03-23 05:26:01
I build them after seeing every symbol
veluca
2023-03-23 05:26:04
Then I guess the question is how do you collect frequencies 😛
Traneptora
2023-03-23 05:26:18
every symbol sent I just stick in a buffer
veluca Then I guess the question is how do you collect frequencies 😛
2023-03-23 05:27:50
```c size_t table_size = stream->num_clusters * stream->alphabet_size; /* populate frequencies */ stream->frequencies = HYD_ALLOCA(stream->allocator, table_size * sizeof(size_t)); memset(stream->frequencies, 0, table_size * sizeof(size_t)); for (size_t pos = 0; pos < stream->symbol_pos; pos++) stream->frequencies[stream->tokens[pos].cluster * stream->alphabet_size + stream->tokens[pos].token]++; ```
2023-03-23 05:27:53
naively
veluca
2023-03-23 05:28:21
Are you sure the cluster is right?
Traneptora
2023-03-23 05:28:43
what do you mean?
2023-03-23 05:30:06
basically I call this function to send a symbol ```c HYDStatusCode hyd_ans_send_symbol(HYDEntropyStream *stream, size_t dist, uint32_t symbol) { HYDAnsToken token; HYDAnsResidue residue; token.cluster = stream->cluster_map[dist]; HYDHybridUintConfig *config = &stream->configs[token.cluster]; int split = 1 << config->split_exponent; if (symbol < split) { token.token = symbol; residue.residue = 0; residue.bits = 0; } else { uint32_t n = hyd_fllog2(symbol) - config->lsb_in_token - config->msb_in_token; uint32_t low = symbol & ~(~UINT32_C(0) << config->lsb_in_token); symbol >>= config->lsb_in_token; residue.residue = symbol & ~(~UINT32_C(0) << n); symbol >>= n; uint32_t high = symbol & ~(~UINT32_C(0) << config->msb_in_token); residue.bits = n; token.token = split + (low | (high << config->lsb_in_token) | ((n - config->split_exponent + config->lsb_in_token + config->msb_in_token) << (config->msb_in_token + config->lsb_in_token))); } stream->tokens[stream->symbol_pos] = token; stream->residues[stream->symbol_pos++] = residue; if (token.token >= stream->alphabet_size) stream->alphabet_size = 1 + token.token; if (stream->symbol_pos > stream->init_symbol_count) return HYD_INTERNAL_ERROR; return HYD_OK; } ```
veluca
2023-03-23 05:31:28
Mh weird
Traneptora
2023-03-23 05:36:02
it seems a bit odd
2023-03-23 05:36:40
like I increase the size of the file by almost 200k
2023-03-23 05:36:57
which is significantly more than the overhead of writing a complex distribution
2023-03-23 05:41:00
wait no
2023-03-23 05:41:01
maybe it's not
veluca
2023-03-23 05:50:23
ah but your frames are all empty right?
2023-03-23 05:50:31
well not all, one group isn't
Traneptora
2023-03-23 05:50:35
I have 56 frames, so 4k overhead is ~200k overhead
2023-03-23 05:50:37
maybe that explains it
veluca
2023-03-23 05:50:56
for those groups you very likely want to use singleton distributions
2023-03-23 05:51:05
"flat" distr with 1 symbol and huffman coding
Traneptora
2023-03-23 05:51:17
well all my frames are a single group
2023-03-23 05:51:43
but I think I need to implement prefix coding
veluca for those groups you very likely want to use singleton distributions
2023-03-23 06:01:20
I'm not using "empty" groups I just set `have_crop` in the frame header
2023-03-23 06:01:29
so the individual frame sizes are only 256x256
2023-03-23 06:02:00
one group per frame means no TOC
2023-03-23 06:02:14
or rather, trivial TOC
veluca
2023-03-23 06:03:13
ah wait, how are you encoding the context map itself?
Traneptora
2023-03-23 06:03:30
apparently with ANS, which is about ~4k bytes overhead. which is definitely not ideal
2023-03-23 06:03:39
that's where the extra 200k is coming from
2023-03-23 06:04:12
I think my solution is to implement prefix coding, which should definitely simplify this matter
veluca
2023-03-23 06:04:33
4k for a context map seems very wrong
Traneptora
2023-03-23 06:04:45
does it?
2023-03-23 06:04:53
the context map is size 7k
veluca
2023-03-23 06:05:00
if it's almost constant, definitely yes
Traneptora
2023-03-23 06:05:15
it's not almost constant, it's got 31 different clusters
veluca
2023-03-23 06:05:20
pretty sure libjxl encodes one of those in \~10s of bytes
Traneptora it's not almost constant, it's got 31 different clusters
2023-03-23 06:05:25
? how so?
Traneptora
Traneptora ```c uint8_t map[7425]; for (int k = 0; k < 15; k++) { memset(map + 37 * k, k, 37); memset(map + 555 + 458 * k, k + 15, 458); } ```
2023-03-23 06:05:39
see here
2023-03-23 06:05:50
I mean, I'm encoding 32 different symbols
veluca
2023-03-23 06:05:50
ah
2023-03-23 06:06:08
don't split block types, it's not very useful 😄
Traneptora
2023-03-23 06:06:18
what do you mean?
veluca
2023-03-23 06:06:41
I'd suggest encoding a block context map that just maps every block type to the same ctx
Traneptora
2023-03-23 06:06:49
note: I'm only using one type of varblock
2023-03-23 06:06:59
8x8 DCT
veluca
2023-03-23 06:07:01
even more so
Traneptora
2023-03-23 06:07:24
so you think I'm getting the best efficiency with one cluster simply because I'm using one type of varblock?
veluca
2023-03-23 06:08:01
https://github.com/veluca93/libjxl/blob/fmjxl/experimental/fmjxl/fmjxl.cc#L577
Traneptora so you think I'm getting the best efficiency with one cluster simply because I'm using one type of varblock?
2023-03-23 06:08:44
no no, just saying that your context map is bigger than it needs to be
Traneptora
2023-03-23 06:09:50
even if I use two clusters: one for the non_zeroes and one for everything else, it's *still* bigger
veluca I'd suggest encoding a block context map that just maps every block type to the same ctx
2023-03-23 06:10:58
pretty sure I'm technically doing that
2023-03-23 06:12:58
```c uint8_t map[7425] = { 0 }; memset(map, 1, 555); ``` even *this* is worse
2023-03-23 06:15:19
this compresses down to 928 bytes
veluca
Traneptora pretty sure I'm technically doing that
2023-03-23 06:15:25
your ctx map shouldn't have 7425 contexts if you did, I think
Traneptora ```c uint8_t map[7425] = { 0 }; memset(map, 1, 555); ``` even *this* is worse
2023-03-23 06:15:32
how big is the encoded ctx map? that should be 4 bytes
2023-03-23 06:15:39
ah no, I misread that 555
2023-03-23 06:15:47
928 bytes is way too much
Traneptora
2023-03-23 06:15:52
7425 is the number of pre-clustered distributions
veluca
2023-03-23 06:16:24
I thought it was half of that
2023-03-23 06:16:28
maybe I misremember 😄
Traneptora
2023-03-23 06:16:42
(458 + 37) * 15
2023-03-23 06:17:11
37 non-zero contexts, 458 hist contexts, and 15 contexts for block types
veluca
2023-03-23 06:17:43
right, you don't need 15
2023-03-23 06:17:47
you can just have 3 😛
2023-03-23 06:17:54
(one per channel)
2023-03-23 06:17:57
possibly even 2
Traneptora
2023-03-23 06:17:57
this is pre-clustered distributions
veluca 928 bytes is way too much
2023-03-23 06:18:41
a context map of size 7425 at 1 bit per context (i.e. 2 clusters) is 928 bytes
2023-03-23 06:19:06
7425/8 = 928
veluca
2023-03-23 06:19:11
https://github.com/veluca93/libjxl/blob/fmjxl/experimental/fmjxl/fmjxl.cc#L701 this is a \~20 byte context map that maps all nnz together, and all AC together, but splits per channel
Traneptora
2023-03-23 06:19:36
how did you get that?
veluca
2023-03-23 06:20:04
with libjxl 😆 but I assume it's using lz77
Traneptora
2023-03-23 06:20:11
oh
2023-03-23 06:20:19
I'm not using lz77
veluca
2023-03-23 06:20:28
that for sure doesn't help
Traneptora
2023-03-23 06:20:38
that's an extra 4 MB of ram, which doubles my existing ram
veluca
2023-03-23 06:20:48
not really?
2023-03-23 06:20:54
you can do rle-only lz77
Traneptora
2023-03-23 06:21:01
ah, right, I suppose
veluca
2023-03-23 06:21:01
good enough for this
2023-03-23 06:21:19
you don't *really* need lz77 for the main content
Traneptora
2023-03-23 06:23:18
yea I didn't think of using lz77-based rle
veluca you can do rle-only lz77
2023-03-23 09:04:06
with rle-only lz77 and MTF I got it down to 200 bytes or so of overhead
2023-03-23 09:04:11
I think I can't beat that without using prefix codes
2023-03-23 09:04:25
based on my current implementation
2023-03-23 09:05:21
er, that's without lz77 haha
veluca
2023-03-23 09:09:12
Ah that makes more sense 😛
Traneptora
2023-03-23 09:09:28
wait, no that is with lz77 <:Thonk:805904896879493180>
2023-03-23 09:11:02
maybe I can't get better than that without prefix codes
2023-03-23 09:11:04
idk
2023-03-23 09:11:08
ANS has a lot of overhead
Jarek
Traneptora ANS has a lot of overhead
2023-03-24 10:37:26
overhead as storing final state can be compensated by storing information in initial ecoder state, overhead as storing probability distribution can be done cheaper than for Huffman: https://arxiv.org/abs/2106.06438
Traneptora
Jarek overhead as storing final state can be compensated by storing information in initial ecoder state, overhead as storing probability distribution can be done cheaper than for Huffman: https://arxiv.org/abs/2106.06438
2023-03-24 11:18:29
I'm referring specifically to the overhead as it exists in JXL's implementation tho
2023-03-24 11:18:44
for very minimal distributions, JXL's prefix codes have lower overhead
2023-03-24 11:19:44
for example, JXL always starts with 0x130000 as its initial state on encode / final on decode, so that's 32 bits "wasted" so to speak
Jarek
2023-03-24 11:21:29
this is free checksum mechanism - alternative is storing a few bits in this initial encoder state
2023-03-24 11:23:24
for storing probability distribution, one can use exactly the same approximation as in canonical Huffman ... but we can also use different - there is larger space of probability quantization to optimize in
2023-03-24 11:27:01
in above arxiv, for 256 size alphabet and 1/1000 penalty in compression ratio, we need approx 100 bytes to encode probability distribution (fig 2)
2023-03-24 11:28:22
canonical Huffman would require more (e.g. 256 x 4 bits) for much higher compression penalty
2023-03-24 11:29:57
and in image compression there are very specific distributions, like Laplace - requiring to write just one parameter to encode disteibution
_wb_
2023-03-24 11:52:19
The hybriduint encoding also helps to keep the distributions small
HLBG007
2023-03-24 02:25:42
This images are in avif better as in jxl https://i.redd.it/59rr49wh1qo31.png https://i.redd.it/l1u04bb15uvx.png https://i.redd.it/rlwol6tgnqd21.png
Fox Wizard
2023-03-24 04:31:52
With what parameters though?
Demiurge
2023-03-24 05:34:45
yes, more detail would be much gooder
HLBG007
2023-03-24 06:39:22
59rr49wh1qo31.png
2023-03-24 06:39:43
l1u04bb15uvx.png
2023-03-24 06:39:53
rlwol6tgnqd21.png
2023-03-24 06:42:37
Details
2023-03-24 06:42:38
s 8 avif e 5 jxl
2023-03-24 06:44:26
lossless e 9 s 0
2023-03-24 06:44:31
diagram
2023-03-24 06:44:32
x=bpp y=ssimulacra2
yoochan
2023-03-24 06:47:40
And you vary the quality for lossy? Why e 5?
2023-03-24 06:51:14
And what are all the options you sent to cjxl?
2023-03-24 06:52:13
Lossless jxl seems to win all rounds
_wb_
2023-03-24 07:47:00
You are doing something wrong if lossy jxl cannot reach a high ssimulacra2 score - maybe colorspaces?
HLBG007
_wb_ You are doing something wrong if lossy jxl cannot reach a high ssimulacra2 score - maybe colorspaces?
2023-03-24 08:02:36
$ ./cjxl source/59rr49wh1qo31.png 59rr49wh1qo31.png.jxl -q 95 -e 5 --quiet $ ./djxl 59rr49wh1qo31.png.jxl 59rr49wh1qo31.jxl.png --quiet $ ./ssimulacra2 source/59rr49wh1qo31.png 59rr49wh1qo31.jxl.png -11.49490490 $ identify source/59rr49wh1qo31.png source/59rr49wh1qo31.png PNG 2028x1346 2028x1346+0+0 8-bit sRGB 4.11953MiB 0.000u 0:00.000 $ identify 59rr49wh1qo31.jxl.png 59rr49wh1qo31.jxl.png PNG 2028x1346 2028x1346+0+0 8-bit sRGB 898345B 0.000u 0:00.000
_wb_
2023-03-24 08:10:25
Hm, looks like a bug
2023-03-24 08:10:43
Is this that bug where djxl produces linear rgb?
2023-03-24 08:11:13
8-bit linear rgb is quite horrible for banding, and ssimulacra2 hates banding
2023-03-24 08:11:45
What happens if you decode to 16-bit png?
HLBG007
2023-03-24 08:25:03
$ ./djxl 59rr49wh1qo31.png.jxl 59rr49wh1qo31.jxl.png --bits_per_sample=16 --quiet $ ./ssimulacra2 source/59rr49wh1qo31.png 59rr49wh1qo31.jxl.png 92.04592574 $ identify 59rr49wh1qo31.jxl.png 59rr49wh1qo31.jxl.png PNG 2028x1346 2028x1346+0+0 16-bit sRGB 6.67921MiB 0.000u 0:00.000
_wb_
2023-03-24 08:29:07
Yep, that's probably that djxl bug
2023-03-24 08:30:22
https://github.com/libjxl/libjxl/issues/2289
2023-03-24 08:32:39
<@795684063032901642> / <@604964375924834314> we need to fix that, it's quite bad that it's currently doing this. Doing linear RGB in 8-bit is basically always a bad idea...
HLBG007
2023-03-24 08:45:35
if I encode the image with -d 8 in avif i get ssimulacra2 : 87.93220993 with -d 10 : 97.60463415 -d 12 : 100.00000000 Filesize 348522 576616 683585
2023-03-24 08:46:27
I should change the bench from 10 to 8 so I can save a lot of filesize
_wb_
2023-03-24 08:50:05
How do the curves look like though? If you match the file sizes, doesn't the 12-bit avif get a better score than the 8-bit one?
HLBG007
_wb_ How do the curves look like though? If you match the file sizes, doesn't the 12-bit avif get a better score than the 8-bit one?
2023-03-24 08:59:04
Yes it does, but it does not make sense because jxl has here already ssimulacra2 92.04592574 with filesize 219369
Demiurge
HLBG007 $ ./djxl 59rr49wh1qo31.png.jxl 59rr49wh1qo31.jxl.png --bits_per_sample=16 --quiet $ ./ssimulacra2 source/59rr49wh1qo31.png 59rr49wh1qo31.jxl.png 92.04592574 $ identify 59rr49wh1qo31.jxl.png 59rr49wh1qo31.jxl.png PNG 2028x1346 2028x1346+0+0 16-bit sRGB 6.67921MiB 0.000u 0:00.000
2023-03-25 01:09:17
that's a huge difference...
HLBG007 Yes it does, but it does not make sense because jxl has here already ssimulacra2 92.04592574 with filesize 219369
2023-03-25 01:11:47
Yeah, that pretty much completely destroys avif then... if not for the djxl bug
yoochan
HLBG007 $ ./djxl 59rr49wh1qo31.png.jxl 59rr49wh1qo31.jxl.png --bits_per_sample=16 --quiet $ ./ssimulacra2 source/59rr49wh1qo31.png 59rr49wh1qo31.jxl.png 92.04592574 $ identify 59rr49wh1qo31.jxl.png 59rr49wh1qo31.jxl.png PNG 2028x1346 2028x1346+0+0 16-bit sRGB 6.67921MiB 0.000u 0:00.000
2023-03-25 06:45:07
Ssimulacra2 don't take jxl as input?
_wb_
2023-03-25 07:02:28
None of the tools do, I suppose we should change that...
Traneptora
_wb_ None of the tools do, I suppose we should change that...
2023-03-25 02:52:44
should I open an issue about that? I feel like anything that accepts PNG input (except perhaps cjxl) should allow jxl input as well
fab
2023-03-25 02:54:26
new version of hydrium
Traneptora
fab new version of hydrium
2023-03-25 02:54:47
I was actually just about to publish one. files should look identical as I haven't tweaked any quant settings but they should be slightly smaller
fab
2023-03-25 02:55:26
0.2.1?
Traneptora
2023-03-25 02:56:02
probably
fab
2023-03-25 02:57:26
you know the newest version of my font
2023-03-25 02:57:44
_wb_
Traneptora should I open an issue about that? I feel like anything that accepts PNG input (except perhaps cjxl) should allow jxl input as well
2023-03-25 02:58:00
there is already one: https://github.com/libjxl/libjxl/issues/2193
Traneptora
2023-03-25 02:58:14
I meant for the other tools, unless you want to keep the issue the same
_wb_
2023-03-25 02:59:48
basically once it is added to extras/dec, it should work for any tool that uses that to read input images
Traneptora
2023-03-25 03:00:19
ah, so it's the same fix
_wb_
2023-03-25 03:00:27
so making it work for cjxl will also make it work for benchmark_xl / butteraugli / ssimulacra etc
Traneptora
fab 0.2.1?
2023-03-25 03:34:13
https://github.com/thebombzen/hydrium/releases/tag/v0.2.1
diskorduser
2023-03-26 02:27:13
Linux build?
Traneptora
diskorduser Linux build?
2023-03-26 02:45:16
included in the thing if you want
2023-03-26 02:45:30
but it's simpler to just `meson setup build/ && ninja build/`
2023-03-26 02:45:53
more bugs in git master fixed anyway
diskorduser
2023-03-26 02:45:57
Good
zamfofex
2023-03-26 03:12:28
Mostly for fun, I tried to build it with tyfkda’s xcc, and it seems to work fine, except that a few macros were missing from its libraries. In particular, the `[U]INT(N)_C()` macros and `LONG_MAX` for lodepng. But adding them in allowed it to compile just fine!
Traneptora
2023-03-26 03:17:22
what's that?
zamfofex
2023-03-26 03:18:42
A C compiler: <https://github.com/tyfkda/xcc>
2023-03-26 03:19:13
It includes its own C library too. It is able to build static executables.
Traneptora
2023-03-26 03:19:55
what is its improvement upon other compilers? or is it just a project from someone
zamfofex
2023-03-26 03:22:34
Well, I generally use it to build Wasm binaries, since it allows me to use standard library functions without needing WASI nor to fiddle around with `--sysroot` in Clang. But other than that, it’s just a fun project someone decided to work on, I think! It does compile in under five seconds (which is several hours less than GCC), and allows the creation of static executables that should run under any Linux operating system.
Traneptora
2023-03-26 03:23:20
I feel like linking to system libc is probably more secure
2023-03-26 03:24:14
unfortunately the wasm compiler is missing goto support, which is pretty helpful for cleanup
2023-03-26 03:26:37
stuff like ```c int func() { int ret = 0; // stuff char *p = NULL; if ((ret = func_that_can_fail()) < 0) goto end; p = malloc(bar); if (!p) goto end; // more stuff end: if (p) free(p); return ret; } ```
zamfofex
2023-03-26 03:27:41
That is true.
DZgas Ж
Traneptora https://github.com/thebombzen/hydrium/releases/tag/v0.2.1
2023-03-26 09:24:16
is there really no choice of quality? right now it's around q98 of cjxl e1
fab
2023-03-26 09:48:18
to me it looks like d 0.456 e4
2023-03-26 09:50:59
anyway better quality than cjxl -d 0.575 -e 6 --epf=2 C:\Users\Use\Desktop\fk\uu.png C:\Users\Use\Desktop\fk\uuq.jxl
2023-03-26 09:53:33
LIKE
2023-03-26 09:53:35
cjxl -d 6.54 -e 2 --epf=3 -I 0.4 C:\Users\Use\Desktop\fk\uu.png C:\Users\Use\Desktop\fk\uuq.jxl
2023-03-26 09:56:07
I read draft 2 of jon sneyers and compare decoding times
fab cjxl -d 6.54 -e 2 --epf=3 -I 0.4 C:\Users\Use\Desktop\fk\uu.png C:\Users\Use\Desktop\fk\uuq.jxl
2023-03-26 09:56:28
this is as slow as hydrium in the decoding time
2023-03-26 09:59:17
the quality is around -q 42.16 -e 8 0.6.1 libjxl
2023-03-26 10:05:07
2023-03-26 10:06:02
2023-03-26 10:11:34
cjxl -d 0.976 -s 7 --epf=2 --gaborish=0 --intensity_target=267 C:\Users\Use\Desktop\sw\uunef.png C:\Users\Use\Desktop\sw\q0sixone.jxl
2023-03-26 10:11:48
with 0.6.1 libjxl is about the same
2023-03-26 10:12:03
2023-03-26 10:27:27
At d 2.002 hydrium is better
Traneptora I feel like linking to system libc is probably more secure
2023-03-26 10:29:14
1.567 s 3
Traneptora
DZgas Ж is there really no choice of quality? right now it's around q98 of cjxl e1
2023-03-26 10:56:28
not at the moment
2023-03-26 10:57:14
still a work in progress
RockPolish
2023-03-26 03:41:01
is it to be expected to sometimes encounter a .png that you just cannot seem to get smaller with lossless jxl?
sklwmp
2023-03-26 03:43:56
probably?
2023-03-26 03:43:59
jxl isn't perfect
2023-03-26 03:44:07
there have been a few instances of that on this server, iirc
RockPolish
2023-03-26 03:50:08
yes I remember seeing one example but I thought with some specific cjxl parameters it could be beaten anyway which I thought was cool
Demez
RockPolish is it to be expected to sometimes encounter a .png that you just cannot seem to get smaller with lossless jxl?
2023-03-26 04:52:26
have you tried lossless webp on any of those?
RockPolish
Demez have you tried lossless webp on any of those?
2023-03-26 05:30:48
I'm trying to right now (with ffmpeg) but it doesn't seem to be lossless (with `--lossless 1` that is), never tried webp before though. also seems like for my image with -e 10 it is much better than png, just not with -e 9
Demez
2023-03-26 05:34:22
i generally prefer using cwebp directly, the arguments i used to use for ffmpeg are ```-lossless 1 -compression_level 6``` and for cwebp i use ```-mt -q 100 -alpha_filter best -lossless -m 6 -progress -v -metadata all -exact```
RockPolish
Demez i generally prefer using cwebp directly, the arguments i used to use for ffmpeg are ```-lossless 1 -compression_level 6``` and for cwebp i use ```-mt -q 100 -alpha_filter best -lossless -m 6 -progress -v -metadata all -exact```
2023-03-26 05:37:59
turns out `-preset default` changed it back to lossy which is why it wasn't working, actually with lossless webp it's 150 KB (jxl -e 10: 165 KB, -e 9: 266 KB, original PNG: 245 KB), so that's interesting
Demez
2023-03-26 05:38:21
what cjxl arguments are you using?
RockPolish
2023-03-26 05:39:59
just `cjxl i.png e10.jxl -d 0 -e 10 --allow_expert_options` and for `-e 9` I also have `-E 3` and tried all 4 values for `-g` (0 to 3)
Demez
Demez i generally prefer using cwebp directly, the arguments i used to use for ffmpeg are ```-lossless 1 -compression_level 6``` and for cwebp i use ```-mt -q 100 -alpha_filter best -lossless -m 6 -progress -v -metadata all -exact```
2023-03-26 05:43:12
also with cwebp i can process images a bit faster (multithreading most likely) and compress slightly more actually
RockPolish
2023-03-26 05:44:37
oh really? well for me it was just out of convenience as I don't have cwebp but I do sometimes use ffmpeg (but not for jxl)
Demez
2023-03-26 05:45:38
oh wait i didn't even use the lossless crunch compressor on cwebp (activated by -q 100), and it was still smaller than ffmpegs output lol
2023-03-26 05:46:59
2023-03-26 05:47:25
ok the crunch compressor did practically nothing in my test image lmao
2023-03-26 05:47:59
also the image im using would be better compressed as lossy anyway, as they are printer scans, but i wanted a big png to test quickly
Traneptora
RockPolish I'm trying to right now (with ffmpeg) but it doesn't seem to be lossless (with `--lossless 1` that is), never tried webp before though. also seems like for my image with -e 10 it is much better than png, just not with -e 9
2023-03-26 07:34:59
you can use -z 9
RockPolish
Traneptora you can use -z 9
2023-03-26 07:41:12
that's for cwebp, right? if I understand the docs correctly, that overrides a few options so probably it'll do the same as `-lossless 1 -compression_level 6 -quality 100` in ffmpeg?
username
2023-03-26 09:42:24
I would like to note that "-z 9" or ''-m 6 -q 100" for lossless in cwebp results in the lossless cruncher being activated which is described as doing this ```"Go over the whole compression step for each of the transforms and pick the best one."```
2023-03-26 09:43:11
I am not experienced with ffmpeg but I think there might be some options it doesn't expose meanwhile using cwebp directly exposes everything
2023-03-26 09:43:56
I would also recommend always using "-mt" with cwebp because it makes the lossless cruncher noticeably faster
Demiurge
2023-03-27 06:00:03
BUT! Does it make it noticeably CRUNCHIER?
gameplayer55055
2023-03-27 08:26:16
Hi there. I've discovered this format years ago, tested but didnt use much. Now i've got hands on my macbook pro with HDR screen I want to test and create HDR pictures, using the jpeg xl(png, jpg, webp cant into HDR), luckily photoshop camera raw can do it. The question is: how to view them on HDR display. JXLook shows the pic in SDR mode, Finder preview is broken, gallery app cant import it, AVIF shows SDR only as well. So, practically only Photoshop Camera Raw allows me to view JXL (and avif) in HDR Here's the amateur HDR jxl reference pic made from my nikon's raw:
2023-03-27 08:26:20
w
2023-03-27 08:43:11
sounds like you answered your own question
Jyrki Alakuijala
2023-03-27 08:47:48
thorium might be able to display HDR jpeg xls?
username
2023-03-27 08:49:01
I was going to mention thorium however I'm not sure if there are MacOS builds for it
2023-03-27 08:49:40
oh there are huh
gameplayer55055 Hi there. I've discovered this format years ago, tested but didnt use much. Now i've got hands on my macbook pro with HDR screen I want to test and create HDR pictures, using the jpeg xl(png, jpg, webp cant into HDR), luckily photoshop camera raw can do it. The question is: how to view them on HDR display. JXLook shows the pic in SDR mode, Finder preview is broken, gallery app cant import it, AVIF shows SDR only as well. So, practically only Photoshop Camera Raw allows me to view JXL (and avif) in HDR Here's the amateur HDR jxl reference pic made from my nikon's raw:
2023-03-27 08:49:58
https://github.com/Alex313031/Thorium-Special/releases/tag/M111.0.5563.111
2023-03-27 08:50:55
this should hopefully work since I know chromium has HDR support along with the JXL support in chromium supporting HDR
gameplayer55055
2023-03-27 08:51:09
i am weak in linuxes
w sounds like you answered your own question
2023-03-27 08:51:33
well, noone uses Photoshop as pic viewer
w
2023-03-27 08:53:12
or jxl as image format <:trollface:834012731710505011>
gameplayer55055
w or jxl as image format <:trollface:834012731710505011>
2023-03-27 08:54:05
or any HDR content <:trolw:1015162529225392128>
username https://github.com/Alex313031/Thorium-Special/releases/tag/M111.0.5563.111
2023-03-27 08:57:05
quick test shows it supports HDR 🥰
2023-03-27 08:57:27
still not a pic viewer but good tho
HLBG007
2023-03-27 11:08:47
https://twitter.com/dominikhlbg/status/1640309416040833030
_wb_
2023-03-27 11:12:59
https://discord.com/channels/794206087879852103/794206170445119489/1088922468133126304
gameplayer55055
HLBG007 https://twitter.com/dominikhlbg/status/1640309416040833030
2023-03-27 11:27:20
avif is video, i want a picture
2023-03-27 11:27:41
<:trol:1005506850369831033>
HLBG007
2023-03-27 02:55:42
If JPEGXL is so good why is then the codec not the new AV1 of AVIF as JXL in better. I missing this in AV1/HEVC videos that I taken. The sences between I Frames are very blurred. JXL could show here a better world. A little bit implementation of P,B,D Frames from Group of Pictures for the best video quality based on JPEGXL developer skills. Every frame looks like an I Frame but its not an I Frame. You would impress the world with this.
gameplayer55055
2023-03-27 02:56:47
progressive decoding?
2023-03-27 02:57:10
and rare gpus support it
2023-03-27 02:57:14
in hw
jonnyawsom3
2023-03-27 03:04:34
If you want less blurry videos then set the bitrate 10x higher, that's all you'd end up doing making everything into a JXL anyway. It's made for excellent lossless, medium and high quality. If you want low sizes then you go for the video compression
Foxtrot
2023-03-27 03:07:15
This seems like the same argument when people say "we need only programing language X for everything". Different products are good at different things. It's good to have choice.
gameplayer55055
Foxtrot This seems like the same argument when people say "we need only programing language X for everything". Different products are good at different things. It's good to have choice.
2023-03-27 03:10:30
yes
jonnyawsom3
_wb_ <@795684063032901642> / <@604964375924834314> we need to fix that, it's quite bad that it's currently doing this. Doing linear RGB in 8-bit is basically always a bad idea...
2023-03-27 03:17:48
Is there a PR open for that yet? Considering how badly it effects the scoring I'm surprised someone didn't try and patch it up immediately, could be loosing a chunk of adoption if people have been testing on their own
HLBG007
If you want less blurry videos then set the bitrate 10x higher, that's all you'd end up doing making everything into a JXL anyway. It's made for excellent lossless, medium and high quality. If you want low sizes then you go for the video compression
2023-03-27 03:45:28
Then it does not need JPEGXL if here is referred to other image and video formats! What probably the AVIF people think when they read your comments here? So completely you do not hold on to JPEGXL. JXL is supposed to be the non plus ultra format that can reduce photographs much better than the video formats. we had a time when minidv motionjpeg was used. every frame was a picture. it was great for editing. and today? where is motion jpeg xl? it would be an enrichment. Don't forget that the cinema films are all shown in Digital Cinema Package JPEG 2000 on the screen. it could become JPEGXL. https://en.wikipedia.org/wiki/Digital_Cinema_Package
gameplayer55055
2023-03-27 03:50:49
personally i want hdr
2023-03-27 03:51:13
so only AVIF and JXL. and prob dng <:trolw:1015162529225392128>
jonnyawsom3
2023-03-27 04:12:47
Oh, well if you had said it was to replace MJPEG then yeah, JXL could do quite well. When someone talks about videos it's usually best to assume they mean H264/H265/AV1
Foxtrot
HLBG007 Then it does not need JPEGXL if here is referred to other image and video formats! What probably the AVIF people think when they read your comments here? So completely you do not hold on to JPEGXL. JXL is supposed to be the non plus ultra format that can reduce photographs much better than the video formats. we had a time when minidv motionjpeg was used. every frame was a picture. it was great for editing. and today? where is motion jpeg xl? it would be an enrichment. Don't forget that the cinema films are all shown in Digital Cinema Package JPEG 2000 on the screen. it could become JPEGXL. https://en.wikipedia.org/wiki/Digital_Cinema_Package
2023-03-27 04:23:59
What they think? I guess they are very grateful that they were picked as the "chosen ones" by Chrome and weren't removed without fair judgment like JPEG XL...
gameplayer55055
Foxtrot What they think? I guess they are very grateful that they were picked as the "chosen ones" by Chrome and weren't removed without fair judgment like JPEG XL...
2023-03-27 04:51:42
i use firefox
2023-03-27 04:51:52
because chrome is spywser
2023-03-27 04:52:15
but... electronjs. its chromium based so no jxl there as well (discord too lol)
Foxtrot
2023-03-27 05:06:15
I just hope JPEG XL will be able to replace TIFF as working format. Even when I use ZIP compression it's just so big. So I hope jxl with layers could be alternative.
_wb_
Is there a PR open for that yet? Considering how badly it effects the scoring I'm surprised someone didn't try and patch it up immediately, could be loosing a chunk of adoption if people have been testing on their own
2023-03-27 05:53:05
It only affects images with an icc profile that cjxl doesn't represent with an Enum. E.g. sRGB images don't trigger this bug. But yes, it should be fixed.
Foxtrot I just hope JPEG XL will be able to replace TIFF as working format. Even when I use ZIP compression it's just so big. So I hope jxl with layers could be alternative.
2023-03-27 05:57:13
Yes, also it would be a standardized and interoperable format for images with layers. TIFF is a mess, was standardized at some point but now Adobe claims it again as their proprietary format so it keeps changing every year. Same with PSD and XCF and all the others that are tied to specific image editors: they change all the time, every new version of the editor brings new stuff in the file format...
Traneptora
2023-03-27 09:12:49
tbf, PSD and XCF have specific goals which are to save all information needed by those editors
2023-03-27 09:13:23
they're not intended to have cross-workflow interoperability so it's hard to fault them for not having it
Foxtrot
2023-03-27 09:34:38
They could at least implement better compression. Why not just internally compress layers with jpeg xl. That could be doable.
Traneptora
2023-03-27 09:35:38
they introduce incompatibilities in each major version so it could be possible to use lossless jxl for that
2023-03-27 09:35:43
but only on a new major version break
Foxtrot
2023-03-27 09:36:45
I just looked and Photoshop added JPEG compression into TIFF. So jxl should also be possible.
Traneptora
2023-03-27 09:48:35
that's true, but third party applications reading TIFF would need to then support JXL. It'd be easier to add it to PSD
Foxtrot
2023-03-27 09:55:48
Yeah... From my point of view as user of photoshop, both is good. I just want smaller working files. 🙂
2023-03-27 09:58:02
I remember how surprised I was in past when I found out that my raws from camera are smaller than compressed tiffs from photoshop 😄
_wb_
Traneptora they're not intended to have cross-workflow interoperability so it's hard to fault them for not having it
2023-03-28 07:13:27
Yes, there will always be a need for editor-specific formats. It's just nice to also have an interoperable format that has layers. Currently there's no good way to do that, the best you can do is using an old version of PSD and hope that it will work everywhere. But that compresses very poorly, and it's interoperable only because Photoshop's market share is big enough to force everyone to reverse-engineer a PSD decoder...
Jyrki Alakuijala
2023-03-28 09:18:26
https://github.com/libjxl/libjxl/pull/2337 improves/changes something -- I think it makes pixel graphics slightly better, some less (mostly just different) border oscillations, doesn't yet solve the red-to-gray-bleaching that DZgas has showed to happen (this is what I am attempting to fix, just didn't succeed yet)
Traneptora tbf, PSD and XCF have specific goals which are to save all information needed by those editors
2023-03-28 09:25:15
I'd love to see JPEG XL to power XCF, it would be a major improvement
Foxtrot
2023-03-28 09:42:17
yeah, and for working files you need lossless compression, so for this application JPEG XL is obviously better than AVIF
HLBG007
Jyrki Alakuijala I'd love to see JPEG XL to power XCF, it would be a major improvement
2023-03-28 09:50:19
That would mean that XCF should only support jpegxl for practical photography compression? At least that's how I interpret it after your last tweet https://twitter.com/jyzg/status/1640312244708712448
improver
2023-03-28 09:54:20
you should interpret it as "AVIF is optimized for video & pixels, JXL is optimized for photos", not more than that
2023-03-28 09:55:09
in particular you shouldn't interpret it as "lossless JXL isn't optimized for pixels"
2023-03-28 09:55:33
because lossless jxl is imo kinda best we currently have out of practical lossless algos
HLBG007
improver because lossless jxl is imo kinda best we currently have out of practical lossless algos
2023-03-28 10:30:53
No there are better lossless compressions than jxl https://encode.su/threads/3942-My-new-lossless-image-compressor?p=76111&viewfull=1#post76111
improver
2023-03-28 10:31:43
I said "practical"
2023-03-28 10:32:20
which means "some real enough software support"
2023-03-28 10:32:44
and somewhat sane encoding/decoding speeds
HLBG007
improver and somewhat sane encoding/decoding speeds
2023-03-28 10:34:35
This would also be fast if the developers had such a strong team as google has. then all intel intrinsics processor optimization would be in there too
improver
2023-03-28 10:35:04
closed source binaries & crazy encode speeds don't count. also "demo" stuff also probably doesn't as it may also change anytime
2023-03-28 10:35:49
FLIF which JXL lossless was (partially) based on wasn't all that "strong team" either
2023-03-28 10:39:20
at this point I'd legit see XCF getting to use JXL as something possibly practical due to somewhat stable standard status & somewhat okay implementation
2023-03-28 10:39:46
wouldn't say the same about some other stuff in that link
HLBG007
improver closed source binaries & crazy encode speeds don't count. also "demo" stuff also probably doesn't as it may also change anytime
2023-03-28 10:41:30
I would also not put my product source code online. It's more of an application from them so that their product is bought by big companies or they are hired
improver
2023-03-28 10:42:05
that's fair, but don't expect any open-source software to pick up to use your format
username
HLBG007 No there are better lossless compressions than jxl https://encode.su/threads/3942-My-new-lossless-image-compressor?p=76111&viewfull=1#post76111
2023-03-28 10:43:17
it seems like in that thread they are accidently using avif in lossy mode and not lossless, also even if they where using avif in lossless mode it still technically wouldn't be lossless as avif can only be in YUV and not RGB
username it seems like in that thread they are accidently using avif in lossy mode and not lossless, also even if they where using avif in lossless mode it still technically wouldn't be lossless as avif can only be in YUV and not RGB
2023-03-28 10:44:14
well sorta depends on how the converting is done between YUV and RGB but still
HLBG007
improver FLIF which JXL lossless was (partially) based on wasn't all that "strong team" either
2023-03-28 10:46:39
That's true but why only FLIF why not a mixture of all? That creates diversity
improver
2023-03-28 10:47:43
because large formats end up with progressively insane standartization/implementation requirements
2023-03-28 10:47:58
I've seen people say bad things about JXL because of that already
2023-03-28 10:48:13
but it's not even that bad
2023-03-28 10:51:24
though I guess XCF could support multiple
2023-03-28 10:51:58
and likely will to have backwards compat
username
username it seems like in that thread they are accidently using avif in lossy mode and not lossless, also even if they where using avif in lossless mode it still technically wouldn't be lossless as avif can only be in YUV and not RGB
2023-03-28 11:01:50
also they don't seem to be testing lossless webp that well as they only change -m and not -q at all and even then it's recommended to use -z to change the losslesss compression effort
2023-03-28 11:02:07
speaking of they don't even test lossless webp at max effort
Jyrki Alakuijala
HLBG007 That would mean that XCF should only support jpegxl for practical photography compression? At least that's how I interpret it after your last tweet https://twitter.com/jyzg/status/1640312244708712448
2023-03-28 12:35:04
photography and lossless, too -- fast lossless with 16 bits per channel is pretty awesome for saving working copies quickly
HLBG007
Jyrki Alakuijala photography and lossless, too -- fast lossless with 16 bits per channel is pretty awesome for saving working copies quickly
2023-03-28 12:35:46
15 bit produce better ssimulacra2
gameplayer55055
2023-03-28 12:35:50
may use jxl instead dng as well
spider-mario
_wb_ <@795684063032901642> / <@604964375924834314> we need to fix that, it's quite bad that it's currently doing this. Doing linear RGB in 8-bit is basically always a bad idea...
2023-03-28 12:43:08
my attempt at a better default is in https://github.com/libjxl/libjxl/pull/2338/files, but while the code is executed, the preferred color space does not seem to be honored
2023-03-28 12:43:29
even though we seem to be in the situation where it should be respected, according to the comment on SetOutputColorSpace
2023-03-28 12:43:39
<@179701849576833024> <@795684063032901642> any idea what could be happening?
_wb_
2023-03-28 12:44:10
iirc SetOutputColorSpace is api-only atm, still needs to be implemented
2023-03-28 12:44:31
it's one of the main things that still need to be done before 1.0 imo
Jyrki Alakuijala
HLBG007 No there are better lossless compressions than jxl https://encode.su/threads/3942-My-new-lossless-image-compressor?p=76111&viewfull=1#post76111
2023-03-28 12:44:50
that is unfortunately a result of incompetent testing, mixing near-lossless, pseudo-lossless and real lossless
HLBG007 15 bit produce better ssimulacra2
2023-03-28 12:45:33
lossless should be the same ssimulacra2
HLBG007
2023-03-28 12:50:55
I do not understand why avif does not need 16bit output for lossy image, but jxl needs it now to get non-negative ssimulacra2 values
Jyrki Alakuijala
2023-03-28 12:52:22
why are you obsessed about negative ssimulacra2 values?
2023-03-28 12:53:10
it is like compressing cjpeg with -q 3 and then comparing with -q 4 and having a philosophical discussion about garbage -- this is just waste of time
2023-03-28 12:53:31
look at ssimulacra2 values above 80, preferrably around 95-100
2023-03-28 12:54:35
distance 3 is about 75 ssimulacra2 -- you shouldn't consider compressing more than that
2023-03-28 12:55:26
if you must save so much, perhaps just take/download less photos or use text-based websites, you'll get a better experience that way than negative quality images
2023-03-28 12:57:55
ascii graphics may also be an option
2023-03-28 12:58:50
let's try to scope the discussion something that is relevant 🙂
HLBG007
2023-03-28 01:00:31
I talk about this https://github.com/libjxl/libjxl/issues/2289#issuecomment-1485662260
Jyrki Alakuijala
2023-03-28 01:01:59
that looks like a bug that is being worked on, let's wait for the fix before discussing it more?
2023-03-28 01:02:17
if you have a great test for it, perhaps contribute into the bug
HLBG007
2023-03-28 01:06:04
Done https://github.com/libjxl/libjxl/issues/2289#issuecomment-1486852447
Jyrki Alakuijala
2023-03-28 01:32:41
I believe we have about 123 significant bugs in libjxl -- we just work together fixing them instead of making each one a huge problem -- there is quite a bit of promise in the tech even with all the problems that we have 🙂
2023-03-28 03:29:11
I found another quality improvement while working on the difficult images, PR https://github.com/libjxl/libjxl/pull/2339 -- this should be good news for further improvement in encoder robustness -- this change is inspired by DZgas reporting the bleaching problem where low intensity colors disappear
BlueSwordM
username it seems like in that thread they are accidently using avif in lossy mode and not lossless, also even if they where using avif in lossless mode it still technically wouldn't be lossless as avif can only be in YUV and not RGB
2023-03-28 05:42:46
It actually is lossless.
2023-03-28 05:43:35
Lossless AV1 can be coded in the RGB domain rather than YCbCr. It does come with an efficiency penalty, but it is lossless. Better not spread misinformation if possible 🙂
username
BlueSwordM Lossless AV1 can be coded in the RGB domain rather than YCbCr. It does come with an efficiency penalty, but it is lossless. Better not spread misinformation if possible 🙂
2023-03-28 05:47:44
ah good to know I was under the impression it could only operate under YUV/YCbCr, I forgot where I heard that from though
afed
2023-03-28 06:06:38
in this thread it's lossy, but lossless is possible for avif, though only for one encoder (libaom), as far as I know
username
2023-03-28 06:08:30
it being used in it's lossy mode for the thread is pretty bad since that thread is supposed to only be comparing lossless modes/codecs
BlueSwordM Lossless AV1 can be coded in the RGB domain rather than YCbCr. It does come with an efficiency penalty, but it is lossless. Better not spread misinformation if possible 🙂
2023-03-28 06:33:33
how do encoders usually handle exposing this? or do most encoders do rgb when told to do lossless?
_wb_
2023-03-28 06:55:18
Besides jxl, most encoders don't have a clue what colorspace they are encoding in. They just know the number of components and treat them as 2D arrays of numbers that need to be preserved (with or without some loss)
2023-03-28 06:58:20
E.g. JPEG just has 1 to 4 components, but what those components mean is not specified. This is "just metadata" from the jpeg spec pov, and considered a task of the file format, not the core codestream.
2023-03-28 07:00:35
In jxl we changed that and promoted colorspace to a first class citizen, making it part of the core spec and integrated deeply, so lossy encoders always 'know what they're doing' because the codec itself operates in an absolute colorspace
BlueSwordM
username how do encoders usually handle exposing this? or do most encoders do rgb when told to do lossless?
2023-03-28 07:47:25
Through colorspace metadata. This is actually a big problem when it comes to compression in general.
_wb_
2023-03-28 07:55:25
I guess originally digital images were just "RGB" and nobody cared about getting colorspaces right because displays were crappy CRTs anyway, often with knobs on them to adjust brightness, contrast and saturation to your own taste.
gameplayer55055
_wb_ I guess originally digital images were just "RGB" and nobody cared about getting colorspaces right because displays were crappy CRTs anyway, often with knobs on them to adjust brightness, contrast and saturation to your own taste.
2023-03-28 07:57:07
CMYK enthusiasts: <:HMMM:792880998836600843>
2023-03-28 07:58:26
and well this is great, now remembering we have TN screens and sRGB is de facto standard
_wb_
2023-03-28 08:00:03
It's crazy that HDR video has better web support than HDR images or css.
gameplayer55055
_wb_ It's crazy that HDR video has better web support than HDR images or css.
2023-03-28 08:00:20
yes
2023-03-28 08:00:40
and i also hate when HDR means messed up colors
2023-03-28 08:00:57
like the one in my phone camera settings
gameplayer55055 and i also hate when HDR means messed up colors
2023-03-28 08:05:01
AVIF, JXL, and TIFF for HDR i liked jxl
Traneptora
2023-03-28 08:57:04
PNG actually supports HDR too
2023-03-28 08:57:18
you just need 16-bit pixel data and some HDR ICC profile (or cICP tag)
gameplayer55055
2023-03-28 09:03:43
can't import well
Traneptora
2023-03-28 09:07:05
yea, not a lot of software supports HDR PNGs but PNG does support HDR
2023-03-28 09:07:09
cjxl notably does
veluca
2023-03-29 06:22:40
Chrome also does ;)
HLBG007
Jyrki Alakuijala I believe we have about 123 significant bugs in libjxl -- we just work together fixing them instead of making each one a huge problem -- there is quite a bit of promise in the tech even with all the problems that we have 🙂
2023-03-29 09:18:27
Your problem with the bugs is that each one is comparable to a big scratch on a CD. You remove/polish it away but then many new bugs/scratches are created. This is libjxl
Jyrki Alakuijala
HLBG007 Your problem with the bugs is that each one is comparable to a big scratch on a CD. You remove/polish it away but then many new bugs/scratches are created. This is libjxl
2023-03-29 09:34:55
it is still better to have than the blurry sound of a tube radio it is still better than pretending that you don't have issues
Demiurge
Jyrki Alakuijala distance 3 is about 75 ssimulacra2 -- you shouldn't consider compressing more than that
2023-03-29 01:12:19
Some images, depending on the type of image, are transparent at d=4 and higher. This is because libjxl is not psychovisually uniform quantization
2023-03-29 01:13:34
libjxl is not yet as close as it should be to recognizing what humans do and don't notice
gameplayer55055
2023-03-29 08:27:04
macos preview is cropped weirdly
Jyrki Alakuijala
2023-03-30 08:52:09
Quality is still improving for difficult images: https://github.com/libjxl/libjxl/pull/2343 -- But more steps are needed. Next I'll look into selection of integral transforms with some more focus on max error including the difficult images.
2023-03-30 09:37:17
last time I run optimization of those heuristics we didn't have subsampling in place and the heuristics needed to make some compromises related to d32 - d128 area (which is just completely useless for any actual utility, but people benchmark there, too) -- now that subsampling takes care of it, I can put no emphasis on it
fab
2023-03-30 06:19:33
D 1.3 is improving
gameplayer55055
2023-03-31 12:12:46
WTF
2023-03-31 12:12:49
2023-03-31 12:12:59
Accept image/avif,image/jxl,image/webp,*/*
2023-03-31 12:14:49
why accept something you cant show
Traneptora
gameplayer55055 Accept image/avif,image/jxl,image/webp,*/*
2023-03-31 12:18:28
do you have `image.jxl.enabled` in about:config?
gameplayer55055
2023-03-31 01:05:08
yes
kmadura
2023-03-31 02:01:22
does this option do anything on regular firefox (not nightly?)
2023-03-31 02:01:47
i have set it as true and still browser doesn't recognize jxl files
Traneptora
2023-03-31 02:04:14
it doesn't do anything except cause it to send `image/jxl` in the `accept:` headers
2023-03-31 02:04:22
it doesn't actually allow it to do anything with it
yoochan
2023-03-31 02:05:48
for firefox the best is the extension by our pal <@171381182640947200> : https://addons.mozilla.org/fr/firefox/addon/jxl/
2023-03-31 02:06:12
(or firefox nightly with the support activated in the about:config)
gameplayer55055
2023-03-31 02:30:39
telecharger?
Traneptora it doesn't do anything except cause it to send `image/jxl` in the `accept:` headers
2023-03-31 02:30:58
:D
yoochan
2023-03-31 02:31:29
télécharger what ?
gameplayer55055
2023-03-31 02:31:37
wht can't linux enthusiasts make something like interface IPictureProvider
2023-03-31 02:31:45
then put all these codecs there
Traneptora
gameplayer55055 telecharger?
2023-03-31 02:31:51
it's french, https://addons.mozilla.org/en-US/firefox/addon/jxl/
yoochan
2023-03-31 02:32:10
lol indeed
gameplayer55055
Traneptora it's french, https://addons.mozilla.org/en-US/firefox/addon/jxl/
2023-03-31 02:32:22
thanks, will try when i open my laptop
yoochan
2023-03-31 02:32:28
who don't understand omelette du fromage !?
gameplayer55055
2023-03-31 02:33:07
return false;
fab
2023-03-31 02:44:52
It doesn't work
2023-03-31 03:19:25
2023-03-31 03:20:48
E6 looks blurred
2023-03-31 03:21:03
Jpeg xl can simplify images very well
gameplayer55055
fab
2023-03-31 03:24:15
how do you do this on windows without wsl2
2023-03-31 03:24:24
windows doesnt even has ls
2023-03-31 03:24:34
only dir 🤮
2023-03-31 03:25:21
and i see that is batch converting
fab
2023-03-31 03:32:45
Use latest build
2023-03-31 03:32:58
Not
2023-03-31 03:33:16
You should use the one by jyrki
BlueSwordM
yoochan (or firefox nightly with the support activated in the about:config)
2023-03-31 03:47:07
No.
2023-03-31 03:47:37
There is a better alternative now: https://github.com/Alex313031/Mercury/releases
yoochan
2023-03-31 04:05:45
indeed 😄 didn't try it yet
gb82
BlueSwordM There is a better alternative now: https://github.com/Alex313031/Mercury/releases
2023-03-31 04:09:49
Can vouch
2023-03-31 05:39:59
Lmao, Firefox Nightly on Android
jonnyawsom3
2023-03-31 05:45:19
Huh, so that's what it's meant to look like
gb82
2023-03-31 06:09:23
Wonder why only the jxl image understands my display is wide gamut
_wb_
2023-03-31 06:52:08
No, it's a case of two wrongs making a right, I think.
2023-03-31 06:57:25
Your display may be wide gamut but firefox/android is probably treating it like sRGB. Then it renders the P3 image in jpeg/png "correctly", i.e. color managed and clipped to display space (sRGB), so you don't see the logo. But for jxl, firefox hasn't merged the patch yet to make it properly do color management, so it simply treats the P3 pixel values as if they are sRGB and directly sends them to the screen. Which may be accidentally the right thing if your display is actually P3.
gameplayer55055
BlueSwordM There is a better alternative now: https://github.com/Alex313031/Mercury/releases
2023-03-31 08:26:44
lol logo. can it be even more oversimplified? also it reminds me ♂️
2023-03-31 08:27:03
why does everyone try to kill yhe damn fox
gb82 Lmao, Firefox Nightly on Android
2023-03-31 08:27:58
i see it on my phone, maybe because it's a screenshot. did the same thing with macbook, P3 sees it, srgb doesnt
_wb_ Your display may be wide gamut but firefox/android is probably treating it like sRGB. Then it renders the P3 image in jpeg/png "correctly", i.e. color managed and clipped to display space (sRGB), so you don't see the logo. But for jxl, firefox hasn't merged the patch yet to make it properly do color management, so it simply treats the P3 pixel values as if they are sRGB and directly sends them to the screen. Which may be accidentally the right thing if your display is actually P3.
2023-03-31 08:28:28
oh
gb82
_wb_ Your display may be wide gamut but firefox/android is probably treating it like sRGB. Then it renders the P3 image in jpeg/png "correctly", i.e. color managed and clipped to display space (sRGB), so you don't see the logo. But for jxl, firefox hasn't merged the patch yet to make it properly do color management, so it simply treats the P3 pixel values as if they are sRGB and directly sends them to the screen. Which may be accidentally the right thing if your display is actually P3.
2023-03-31 10:33:00
ah, that's interesting
username
2023-04-01 05:05:43
it seems that even with the unmerged patches something isn't quite right about the color management with JXL in firefox
2023-04-01 05:07:09
gfx.color_management.mode set to **0**:
2023-04-01 05:07:29
gfx.color_management.mode set to **1**:
2023-04-01 05:07:47
gfx.color_management.mode set to **2**:
2023-04-01 05:08:20
https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/3.5/ICC_color_correction_in_Firefox
2023-04-01 05:08:54
the default for "gfx.color_management.mode" is 2 even in the most recent versions of firefox
2023-04-01 05:13:04
the behavior of "0" and "1" are fine but "2" is where things seems to mess up
2023-04-01 05:13:29
and this is a somewhat big problem since the default value is "2"
spider-mario
gameplayer55055 telecharger?
2023-04-01 10:42:16
“download”
2023-04-01 10:42:34
theoretically, to upload is _téléverser_, but I know absolutely no one who uses that word
2023-04-01 10:42:56
(_verser_ = to pour)
2023-04-01 10:43:36
the wiktionary says it’s more common in Canada
2023-04-01 10:43:40
fair enough, I guess
pshufb
Jyrki Alakuijala last time I run optimization of those heuristics we didn't have subsampling in place and the heuristics needed to make some compromises related to d32 - d128 area (which is just completely useless for any actual utility, but people benchmark there, too) -- now that subsampling takes care of it, I can put no emphasis on it
2023-04-01 03:27:18
how do you "run optimization"? is this a largely manual process, or is this mostly automated? if the latter, could you please elaborate on how it works?
fab
2023-04-02 09:22:25
2023-04-02 09:23:39
<@853026420792360980>
Jyrki Alakuijala
2023-04-03 06:05:30
I don't like format linting -- if it compiles, the code is good https://github.com/libjxl/libjxl/pull/2353
pshufb how do you "run optimization"? is this a largely manual process, or is this mostly automated? if the latter, could you please elaborate on how it works?
2023-04-03 06:33:16
some of it is manual, but often I used the tools/optimizer/simplex_fork.py to run a tweaked Nelder-Mead algorithm to find good parameter values -- it is a universal autotweaker
gameplayer55055
2023-04-03 06:44:22
btw is there any .net thing for jxl, because i hate c++ and linuxes. yes it has dllimport but maybe there's a great wrapper/implementation
2023-04-03 06:45:52
asking because I don't like repositories with 10 commits and 1 star
Traneptora
gameplayer55055 btw is there any .net thing for jxl, because i hate c++ and linuxes. yes it has dllimport but maybe there's a great wrapper/implementation
2023-04-03 07:05:37
the libjxl API is in C, if you want to use C# for something
2023-04-03 07:05:44
since C# is compatible with C
gameplayer55055
Traneptora since C# is compatible with C
2023-04-03 07:07:36
dllimport
2023-04-03 07:08:19
C is linux thing, on windows you have completely different standards, and everything incompatible except int main and switch case
Traneptora
2023-04-03 07:12:49
C is not a "linux thing," it's completely possible on windows
2023-04-03 07:12:54
Microsoft Visual C is a thing
gameplayer55055
2023-04-03 07:14:19
COM and WPF instead POSIX?
Traneptora
2023-04-03 07:14:40
what are you talking about
gameplayer55055
2023-04-03 07:15:51
I've tried using libpng. the attempt catastrophically failed. i want to experiment with jpeg xl, use channels for PBR in opengl for example
Traneptora
2023-04-03 07:16:04
libpng is very weird
2023-04-03 07:16:09
it uses setjmp
2023-04-03 07:16:25
libjxl doesn't do anything usual like that
2023-04-03 07:16:34
here's a SO answer about that issue: https://stackoverflow.com/questions/5010957/call-function-from-dll
gameplayer55055
2023-04-03 07:17:18
looking for wrappers, if there are none then i will have a nice journey
_wb_
2023-04-03 07:18:43
The whole point of programming languages is portability across platforms (both cpu and kernel syscall apis). If you have a platform that cannot properly deal with C, then I think that's more a problem with the platform than with C.
Traneptora
2023-04-03 07:19:14
C# can deal with C
2023-04-03 07:19:46
I think they were looking for someone who already wrote bindings for libjxl with a C#-like interface
2023-04-03 07:19:58
but as far as I'm aware, none exist atm
gameplayer55055
2023-04-03 07:20:03
asm is for the same cpu c and c++ are for the same os everything with vm like java c# or interpreted are true cross platform if there's vm implementation
Traneptora I think they were looking for someone who already wrote bindings for libjxl with a C#-like interface
2023-04-03 07:20:12
exactly
Traneptora
2023-04-03 07:20:19
be the change you want to see in the world also C# doesn't run on a VM, it runs on windows
gameplayer55055
2023-04-03 07:20:28
CLR?
2023-04-03 07:20:58
it's a Microsoft Java, come on