|
gggol
|
2025-11-23 07:07:44
|
This is the one that Eye of Gnome couldn't view. |
|
2025-11-23 07:14:30
|
The image is best compressed losslessly. This is the lossless compression of the original. |
|
|
AccessViolation_
|
2025-11-23 08:28:58
|
thoughts on a minor modification to `jxl_cli` that would name the resulting binary `jxl-cli` (hyphen instead of underscore) when `cargo install`ed? it feels more appropriate. if yes, I can make the change |
|
2025-11-23 08:29:52
|
it'd just be a change in `Cargo.toml` without actually changing the names of any packages |
|
|
jonnyawsom3
|
2025-11-23 08:33:07
|
Could just make it jxl-rs to match the library name, or djxl-rs to match libjxl and prep for future encoding. I'm no expert |
|
|
|
AccessViolation_
|
2025-11-23 08:39:01
|
yeah, sounds good to me too |
|
2025-11-23 08:53:46
|
I guess that'd be a more contentious decision. replacing the underscore probably less so. a hyphen is much more common in cli tools so it'd be nice to match what people probably expect |
|
2025-11-23 08:56:02
|
I also personally don't mind `jxl-cli` instead of `jxl-rs`, especially if it intends to become the reference implementation, superseding libjxl/cjxl |
|
|
|
veluca
|
2025-11-25 10:20:25
|
djxl-rs? |
|
|
monad
|
|
Tirr
|
2025-11-25 11:06:09
|
I'd prefer `jxl -d` or something, not a strong opinion though |
|
2025-11-25 11:07:27
|
(jxl-oxide has `jxl-oxide -d` for decode and `jxl-oxide -I` for info) |
|
|
RaveSteel
|
2025-11-25 07:37:43
|
Tomorrow is libjxl v0.11.1's first anniversary🥳 |
|
|
Meow
|
2025-11-26 02:38:48
|
I don't know if this is a good news |
|
|
|
AccessViolation_
|
2025-11-27 06:31:20
|
is it intended that libjxl dithers output for normal recompressed JPEGs (that are 8 bpc to begin with)? |
|
|
|
jonnyawsom3
|
2025-11-27 06:35:20
|
https://discord.com/channels/794206087879852103/794206170445119489/1442562907488391320 |
|
|
|
AccessViolation_
|
2025-11-27 06:37:26
|
right, 12 bit, but don't JPEG encoders receive 8 bit inputs? |
|
2025-11-27 06:39:39
|
never mind, jpegli |
|
|
spider-mario
|
2025-11-27 06:48:15
|
even then, 8-bit -> (higher-precision processing) -> 8-bit should still, in principle, be redithered after the processing |
|
|
|
AccessViolation_
|
2025-11-27 06:51:53
|
hmm, I get why that probably works in principle, but I think in practice people might wonder why their losslessly transcoded JPEG now looks noisy, where parts that were previously a flat color now have a sort of grain |
|
2025-11-27 06:52:42
|
the effect is probably less obvious if your image viewers blurs pixels when zooming in, I should say I explicitly disabled that in EOG |
|
|
|
spider-mario
|
2025-11-27 06:52:42
|
is it that noticeable? |
|
|
|
AccessViolation_
|
|
|
spider-mario
|
2025-11-27 06:57:58
|
ah, right, the grid pattern kind of gives it away |
|
|
|
AccessViolation_
|
2025-11-27 07:00:59
|
it's very noticeable in that one above, but when "blur images when zoomed in" is enabled they look close enough. of course the dithered one looks a bit better since the blur gradient will be smoother :) |
|
|
lonjil
|
2025-11-27 07:01:24
|
a good image viewer should get the pixels in at least f16 from libjxl and then dither at whatever scale the image is being displayed at, so that the dither doesn't get bigger with zooming |
|
|
|
AccessViolation_
|
2025-11-27 07:02:03
|
yeah I agree |
|
|
|
jonnyawsom3
|
2025-11-27 07:08:07
|
Use modern djxl |
|
|
|
AccessViolation_
|
2025-11-27 07:08:42
|
I'm not sure why it's not doing blue noise dithering, it did before... maybe I downgraded and don't remember |
|
2025-11-27 07:09:03
|
ah no that'd be because it's using the old one in EOG I suppose |
|
|
|
monad
|
2025-11-28 04:19:38
|
EOG sucky |
|
|
Demiurge
|
2025-11-28 11:26:57
|
Yes. 8 bit output should always be redithered. Even if the source is 8 bit. After being converted to JPEG it's no longer an image of 8-bit pixel data. It's in DCT representation and it has to be converted back to actual pixels. And if you're only using 8 bit output, then dither is definitely necessary if you don't want ugly banding artifacts. |
|
|
|
AccessViolation_
|
2025-11-28 04:26:57
|
I was mainly thinking that in terms of appearance, the dithered output is far from lossless with respect to the original JPEG, but you could make the argument that it's wrong to display the original JPEG in 8 bit too, though presumably not according to the JPEG spec |
|
|
|
lonjil
|
2025-11-28 04:59:02
|
the jpeg spec is really lose with what the output should be |
|
|
|
jonnyawsom3
|
2025-11-29 09:36:12
|
Took another look at the 0.12 tracking issue <https://github.com/libjxl/libjxl/issues/4376>
Seems like the only major things are the progressive noise bug https://github.com/libjxl/libjxl/issues/4368 and the JPEGLI PRs |
|
|
A homosapien
|
2025-11-29 09:56:52
|
Hopefully we can get a release in by Christmas |
|
|
Magnap
|
2025-12-01 10:07:11
|
I am very new to color management, is this correct?
- for a non-XYB encoded file (such as a Modular-mode losslessly compressed one), the only way to do colorspace conversion would be to decode to pixels, do the conversion, then reencode
- for an XYB-encoded file, you could do "colorspace conversion" by changing the signaled colorspace (but it would still be XYB under the hood)
if so, is there a tool to do the latter to an XYB-encoded JXL file? without re-encoding? |
|
2025-12-01 10:12:18
|
also, is there a way, either with `cjxl` or just the libjxl encoder API, to give it XYB data and end up with a losslessly compressed file with `metadata.xyb_encoded` being true? |
|
|
|
ignaloidas
|
2025-12-01 10:32:17
|
for XYB-encoded files, the signaled colorspace is mostly irrelevant for displaying it |
|
|
|
Magnap
|
2025-12-01 10:46:08
|
that's why I put "colorspace conversion" in quotes for that case
but whichever thumbnail generator thunar is using on my system doesn't think so 🙃 to be clear, I know it's the part that's the problem here, but it got me wondering if there was a convenient way to swap out the ICC profile |
|
|
Traneptora
|
2025-12-01 12:06:50
|
So XYB and non-XYB encoded files have different semantics when it comes to tagging the color space
- If the file is XYB-encoded, the tagged colorspace is a suggestion. Decoders are free to decode into that space, or just decode into another space. The design of this was to allow decoders to decode directly into the target space, of, say, the monitor.
- If the file is not XYB-encoded, the tagged colorspace is descriptive. It tells you "the pixels are in this space." Decoding the file will always decode into that space. |
|
2025-12-01 12:09:15
|
To answer the question about tools:
- Theoretically someone could write an editor to change color/header metadata values without recompressing the file, but at the moment that software does not exist (yet).
- cjxl/libjxl always accepts RGB data. XYB modular exists, but even with it enabled libjxl only accepts RGB and will do the XYB conversion internally. So this isn't a limitation of the codestream format, but rather, a limitation of the software that exists right now. |
|
|
|
Magnap
|
2025-12-01 12:12:52
|
that's how I understood it from the "the JPEG XL image coding system" paper, I just wasn't sure if my conclusions from that were correct, but it sounds like they were, so that's great 😅 |
|
|
|
Traneptora
|
2025-12-01 12:12:57
|
I was consideirng writing up some kind of JXL editor but then life caught up with me |
|
2025-12-01 12:13:03
|
i.e. time required |
|
|
|
Magnap
|
2025-12-01 12:13:06
|
understandable, it's pretty niche |
|
|
|
lonjil
|
2025-12-01 12:14:14
|
I have also considered making something like that, but ah, school is keeping me busy. |
|
|
Exorcist
|
2025-12-01 12:18:03
|
Today, there is still no JPEG XL bitstream analyzer?<:FeelsAmazingMan:808826295768449054> |
|
|
|
Traneptora
|
2025-12-01 12:21:15
|
bitstream analyzer exists |
|
2025-12-01 12:21:17
|
`jxlinfo -v` |
|
2025-12-01 12:21:25
|
just not one that can edit |
|
|
|
ignaloidas
|
2025-12-01 01:06:39
|
eh, it's really not a lot, jxlatte with --debug --info=trace is way more helpful in that regard |
|
2025-12-01 01:07:49
|
I am working on a more full-featured parser that would keep stuff like positions of fields so that you could analyse the bitstream, but entropy coding is taking me a while to fully grok |
|
|
_wb_
|
2025-12-01 01:07:52
|
We really should make some `jxltran` tool to do all kinds of header-only transformations. Changing the colorspace tag would be nice functionality to have, even for RGB images (where it of course does change the way the image looks, so it would only be useful for accidentally mistagged images). |
|
|
|
Traneptora
|
2025-12-01 01:19:07
|
git master already has one but it's very primitive |
|
|
|
AccessViolation_
|
2025-12-01 01:20:10
|
maybe I can create a byte pattern for hex editors so you can tell which part of the image you're changing when you open it up in a hex editor |
|
2025-12-01 01:28:11
|
though I shouldn't get distracted by that, I'm currently working on lossy format-aware transcoding :3 |
|
|
|
ignaloidas
|
2025-12-01 01:28:34
|
I've found hex editors borderline useless considering basically nothing is byte-aligned |
|
|
|
AccessViolation_
|
2025-12-01 01:29:37
|
ah, I guess only the outer headers are byte aligned |
|
|
|
ignaloidas
|
2025-12-01 01:32:31
|
the signature, and starts of groups are aligned, basically everything else isn't |
|
2025-12-01 01:32:48
|
(essentially, only stuff you might want to skip to is aligned) |
|
|
|
AccessViolation_
|
2025-12-01 01:35:27
|
what if we signal huffman trees where every byte is represented by itself :3
or something that does the equivalent in ANS, if possible |
|
2025-12-01 01:35:52
|
wait can you even do that with huffman trees? I forget how they work exactly |
|
2025-12-01 01:35:57
|
anyway I'm just being silly |
|
|
|
ignaloidas
|
2025-12-01 01:36:27
|
that only solves the issue for entropy coded data, headers would still be a forest of unaligned fields |
|
|
|
Magnap
|
2025-12-01 01:36:36
|
I mean, a fixed-length code is a prefix code, isn't it? |
|
|
|
ignaloidas
|
2025-12-01 01:36:38
|
and yes you can |
|
|
|
jonnyawsom3
|
2025-12-01 02:49:15
|
You can specify an output colorspace with djxl and XYB files will then decode to it |
|
|
|
Magnap
|
2025-12-01 05:29:58
|
what happens if the `modular_16bit_buffers` flag is wrong? like, being false is safe IIUC, but if it's true and 16-bit precision isn't actually enough.
both in terms of what djxl does and also more broadly what a compliant decoder is allowed to do |
|
|
|
veluca
|
2025-12-01 05:31:23
|
a decoder can do whatever |
|
|
|
jonnyawsom3
|
2025-12-01 05:31:28
|
We had that happen a while ago when people tried to do 'lossless VarDCT'. It's undefined behaviour, so the program can either error or just output the lower precision result |
|
|
|
veluca
|
2025-12-01 05:31:31
|
and I believe libjxl ignores it |
|
|
|
Magnap
|
2025-12-01 05:36:37
|
ok cool, that means I gotta think about how to represent that if at all in my API. the reason for my questions (also the previous ones) is that I want to make a little library that lets you make animated JXLs out of frames and patches |
|
2025-12-01 05:38:01
|
but I am not looking to make an entire encoder from scratch, so it'll take already encoded frames, which means I need to make sure users know to either only pass in frames from XYB-encoded images, or from non-XYB images in the same colorspace, and to make sure that flag is respected, things like that |
|
|
|
Traneptora
|
2025-12-02 06:00:15
|
libjxl alloc's a 16-bit modular buffer if this flag is true, afaik |
|
2025-12-02 06:00:36
|
if it's true, then overflowing a 16-bit signed integer overflows the modular buffer |
|
2025-12-02 06:00:51
|
I'm not sure if overflowing has signed-integer sematics specified, or if it's noncompliant |
|
|
|
veluca
|
2025-12-02 06:03:27
|
Ah that could be |
|
2025-12-02 06:03:36
|
I am fairly sure it's noncompliant |
|
|
|
jonnyawsom3
|
2025-12-02 06:09:14
|
This file has the 16-bit flag incorrectly set to true, grabbed from [this](https://github.com/libjxl/libjxl/issues/4234) issue, doesn't look like libjxl decodes anything obviously incorrectly though so maybe it's doing full float regardless |
|
|
|
Traneptora
|
2025-12-02 06:09:22
|
the actual libjxl behaior is overflow. that's used a lot in jxl art |
|
|
|
veluca
|
2025-12-02 06:10:33
|
I doubt any decoder will do anything else but overflow, at least in release builds |
|
|
|
Traneptora
|
2025-12-02 06:10:43
|
looks like libjxl and jxlatte decode it as expected, jxl-oxide messes up b/c it's noncompliant and overflows |
|
2025-12-02 06:10:52
|
and rust semantics might mess that up |
|
2025-12-02 06:10:57
|
curious how jxl-rs handles it |
|
|
|
Tirr
|
2025-12-02 06:13:23
|
the image is invalid in that case |
|
2025-12-02 06:14:31
|
and jxl-rs only has 32-bit buffers for now, so it will behave the same as libjxl |
|
|
|
jonnyawsom3
|
2025-12-02 06:16:20
|
jxl-rs decodes it fine, some values off by 1 or 2 compared to libjxl |
|
|
|
Tirr
|
2025-12-02 06:16:22
|
I doubt libjxl has 16-bit buffers, it has typedef for only 32-bit integer type at least https://github.com/libjxl/libjxl/blob/53042ec537712e0df08709524f4df097d42174bc/lib/jxl/modular/modular_image.h#L25 |
|
|
|
Traneptora
|
2025-12-02 10:58:24
|
see for example: https://github.com/libjxl/libjxl/blob/53042ec537712e0df08709524f4df097d42174bc/lib/jxl/dec_group.cc#L130 |
|
2025-12-02 10:58:53
|
sure, this is HF blocks, but it does internally differentiate |
|
2025-12-02 11:01:24
|
I'm not sure where the code to do this is with modular though |
|
|
|
Tirr
|
2025-12-03 01:15:53
|
the overflow happens in LF image |
|
|
|
jonnyawsom3
|
2025-12-04 08:08:57
|
Hey <@794205442175402004>, do you know if chunked encoding might be breaking the global palette?
I was running some tests on this image, and effort 4 of v0.8 is beating effort 9 on main |
|
2025-12-04 08:09:00
|
https://cdn.discordapp.com/attachments/794206087879852107/1445963476253151377/Test.png?ex=693241ed&is=6930f06d&hm=a9a987d882521ca61f45dbcc6774e6e6df98e58aa4f62317bc0dfcf2534a4de3& |
|
|
|
_wb_
|
2025-12-04 08:09:50
|
that does sound like something that is possible. Chunked and global don't mix well 🙂 |
|
|
|
jonnyawsom3
|
2025-12-04 08:10:36
|
Effort 10 or enabling patches brings it back to v0.8 at 4x smaller, but effort 10 with patches disabled only increases by a hundred bytes, so the gain isn't from more coding tools |
|
|
|
Traneptora
|
2025-12-05 12:03:32
|
worth mentioning that they added an Annex N extension which is just padding. data inside it is discarded |
|
2025-12-05 12:03:51
|
reason for this is Frames are byte-aligned, but the zero pad to byte happens after the ICC profile |
|
2025-12-05 12:12:45
|
so if you were to change the size of the ImageHeader such that the ICC profile would start at a different bit alignment, you'd have to shift the entire ICC profile but stop shifting when you get to the frame hader, which requires you to know how long the ICC profile is |
|
2025-12-05 12:12:50
|
unfortunately this requires entropy decoding it, to determine its length |
|
2025-12-05 12:13:32
|
So, in theory, in the image header you can use the extensions bundle to shift the size of the header |
|
|
|
Magnap
|
2025-12-05 01:02:05
|
what's the mapping between `--quality` and `--distance` for cjxl? |
|
|
|
lonjil
|
2025-12-05 01:12:12
|
```c
float JxlEncoderDistanceFromQuality(float quality) {
return quality >= 100.0 ? 0.0
: quality >= 30
? 0.1 + (100 - quality) * 0.09
: 53.0 / 3000.0 * quality * quality - 23.0 / 20.0 * quality + 25.0;
}
``` |
|
|
Jyrki Alakuijala
|
2025-12-05 11:48:05
|
the sooner 'quality' as a 0..100 concept goes away, the better -- distance is just a better way to say the same thing 🙂 |
|
|
|
lonjil
|
|
|
Magnap
|
2025-12-06 03:37:12
|
That's why I wanted the function, because Krita only lets you set the "quality" and I wanted to know how to get the distance I want 😅 |
|
|
MissBehavior
|
2025-12-07 11:49:51
|
how fit in heic container? |
|
|
username
|
2025-12-08 01:31:25
|
I assume you mean heif? (heic is heif with H265 inside) I don't think libjxl supports writing/creating HEIF files with JXL inside and also I don't think there's anything currently (including Apple devices) that even supports HEIF files with JXL inside.
unless you are asking something else? If so can you provide more details as to what you are trying to do exactly. |
|
|
|
MissBehavior
|
2025-12-08 01:32:14
|
I made a silly assumption, i'm really just trying to keep my exif data |
|
|
|
username
|
2025-12-08 01:33:30
|
ah well JXL supports storing EXIF metadata so you should be fine on that front as long as the metadata is getting read from the HEIC file and properly passed to the JXL |
|
|
|
MissBehavior
|
2025-12-08 01:33:47
|
cjxl is not working for that unfortunatley |
|
2025-12-08 01:34:12
|
so I have to do a manual exif entry it seems |
|
2025-12-08 01:34:18
|
is there a better way |
|
|
|
username
|
2025-12-08 01:47:12
|
I haven't really messed around with cjxl on that front much unfortunately so I'm unsure. You might have to end up setting up a script that uses [ExifTool](https://exiftool.org/) to extract the exif data out into a file and then have that get passed into cjxl alongside the image you are encoding so something along the lines of `cjxl input.heic -x exif=input.exif output.jxl` for cjxl's side of things maybe? |
|
|
NovaZone
|
2025-12-08 02:11:22
|
If ur fine with a gui https://github.com/JacobDev1/xl-converter |
|
2025-12-08 02:12:17
|
Default is to wipe but supports both encoder exif and exiftool listed above |
|
2025-12-08 02:12:50
|
Also uses bog standard vanilla libs, which is very nice |
|
|
|
MissBehavior
|
2025-12-08 04:10:26
|
thank you |
|
2025-12-08 04:10:35
|
the both of you have been really helpful |
|
|
|
Traneptora
|
2025-12-08 07:39:33
|
ffmpeg has a PR rn to write exif data |
|
2025-12-08 07:39:39
|
merge soon tm |
|
|
|
MissBehavior
|
2025-12-08 11:54:36
|
thank you 🙏🏼 |
|
|
Laserhosen
|
2025-12-08 02:15:50
|
Question: if the decoder gives you both an encoded color profile and an ICC, is the encoded profile always the "canonical" one of the two? |
|
2025-12-08 02:16:02
|
That's the impression I get from decode.h... but the reason I ask is I'm trying to prevent cjxl from adding an unnecessary ICC profile when converting JXL -> JXL when the input has an encoded profile. |
|
2025-12-08 02:16:10
|
And currently `DecodeImageJXL` is too eager to set `ppf->primary_color_representation = PackedPixelFile::kIccIsPrimary`.
It does this whenever the decoded JXL's "original" profile can be fetched as an ICC... which is very often the case even when it never had an ICC. |
|
2025-12-08 02:20:24
|
My blunt-instrument fix is setting this to `kColorEncodingIsPrimary` whenever a valid encoded profile exists. But I'm guessing it's more subtle than that... |
|
|
|
Traneptora
|
2025-12-08 03:43:15
|
the header of a JXL file stores one or the other, not both |
|
2025-12-08 03:44:09
|
if you ask the library for an ICC profile when it has enum values, it'll synthesize one |
|
|
|
Laserhosen
|
2025-12-08 03:44:18
|
But as far I know, the decoder can't tell you which is "real". |
|
|
|
Traneptora
|
2025-12-08 03:44:18
|
so you have to ask if for enum values first |
|
2025-12-08 03:44:29
|
it can, you just have to ask it for enum values |
|
2025-12-08 03:44:38
|
if that fails it means that inside the file is ICC |
|
|
|
Magnap
|
2025-12-08 03:45:21
|
(with `JxlDecoderGetColorAsEncodedProfile`, ftr) |
|
|
|
Laserhosen
|
2025-12-08 03:45:27
|
So if the enum is present it's always correct to use that (at least until you need an actual ICC to do a color conversion, or you're writing a file that doesn't support the encoded type) |
|
|
|
Traneptora
|
2025-12-08 03:46:05
|
yea, if the enum values are present, they are perfectly descriptive |
|
2025-12-08 03:46:32
|
or rather describe the space as well as a corresponding ICC profile could |
|
|
|
Magnap
|
2025-12-08 03:46:44
|
which says this in [its documentation](https://libjxl.readthedocs.io/en/latest/api_decoder.html#_CPPv434JxlDecoderGetColorAsEncodedProfilePK10JxlDecoder21JxlColorProfileTargetP16JxlColorEncoding):
> When rendering an image on a system where ICC-based color management is used, `JxlDecoderGetColorAsICCProfile` should generally be used first as it will return a ready-to-use profile (with the aforementioned caveat about HDR). When knowledge about the nominal color space is desired if available, `JxlDecoderGetColorAsEncodedProfile` should be used first. |
|
|
|
Traneptora
|
2025-12-08 03:47:15
|
yea, if you're rendering to an ICC thing you should just get the ICC anyway and not worry too much |
|
|
|
Laserhosen
|
2025-12-08 03:47:28
|
Thanks, but that makes me wonder why `primary_color_representation` exists, if the enumerated values are always "better" to use when possible |
|
|
|
Traneptora
|
2025-12-08 03:47:54
|
they aren't "better" in the case where you're just throwing the data to something that supports ICC color management |
|
2025-12-08 03:48:02
|
if you're rendering to the screen, it's preferred for example to just use ICC |
|
2025-12-08 03:48:23
|
but, if, for example you're wrapping libjxl and want to decode to a PNG and set its cICP tag, that's when ICC profiles are less useful |
|
|
|
Magnap
|
2025-12-08 03:48:29
|
it _does_ sound weird to me that it would be set to `kIccIsPrimary` if the color profile is actually the structured encoding kind, but I am not very familiar with the libjxl API |
|
|
|
Laserhosen
|
2025-12-08 03:49:36
|
Of course, in a situation where you're forced to use an ICC, you'd use the ICC, but I'm not sure what the flag is telling us. |
|
|
|
Traneptora
|
2025-12-08 03:49:38
|
it's worth mentioning that if inside the JPEG XL file is an ICC profile, but it's XYB encoded, it's not always possible to use it |
|
2025-12-08 03:49:57
|
the flag is internal to libjxl, clients don't see it |
|
2025-12-08 03:50:08
|
clients just see the functions you call and their responses |
|
2025-12-08 03:50:51
|
for example if the ICC profile describes how to encode RGB -> PCS but not vice versa, then you can't get the pixels in the space described by it |
|
2025-12-08 03:51:07
|
you can encode *from it* but not *to it* |
|
2025-12-08 03:51:22
|
an example of such ICC profiles are the ones in XYB jpegs produced by jpegli |
|
|
|
Magnap
|
2025-12-08 03:51:38
|
the use case of JXL→JXL is even mentioned specifically in [a comment](https://github.com/libjxl/libjxl/blob/53042ec537712e0df08709524f4df097d42174bc/lib/extras/packed_image.h#L209C1-L217C26) that to me sounds like it should be `kColorEncodingIsPrimary` when the color profile was a structured encoding:
```cpp
// `primary_color_representation` indicates whether `color_encoding` or `icc`
// is the “authoritative” encoding of the colorspace, as opposed to a fallback
// encoding. For example, if `color_encoding` is the primary one, as would
// occur when decoding a jxl file with such a representation, then `enc/jxl`
// will use it and ignore the ICC profile, whereas `enc/png` will include the
// ICC profile for compatibility.
// If `icc` is the primary representation, `enc/jxl` will preserve it when
// compressing losslessly, but *may* encode it as a color_encoding when
// compressing lossily.
``` |
|
|
|
Laserhosen
|
2025-12-08 03:51:51
|
Or to put it another way, when cjxl calls `EncodeImageJXL` with a PackedPixelFile that has a valid enumerated profile, why would it ever want to write an ICC? |
|
|
|
Traneptora
|
2025-12-08 03:52:24
|
it wouldn't, since libjxl will parse ICC profiles that can be represented as enums and store the enums instead |
|
2025-12-08 03:52:36
|
so the synthesize -> parse in theory roundtrips |
|
2025-12-08 03:52:39
|
it's just a bit slower |
|
2025-12-08 03:52:52
|
if it doesn't roundtrip, that's a bug |
|
|
|
Laserhosen
|
2025-12-08 03:54:02
|
Yes, at the moment doing a JXL-> JXL conversion through cjxl will add an ICC where there wasn't one. It will probably round trip, but it makes the file bigger than it needs to be. |
|
2025-12-08 03:54:14
|
(Lossless) |
|
2025-12-08 03:55:11
|
So maybe the fix should be in `EncodeImageJXL` to just... always use the enums if they're set. |
|
|
|
spider-mario
|
2025-12-08 03:57:25
|
the goal of `primary_color_representation` was exactly to prevent that (https://github.com/libjxl/libjxl/commit/c29089110b6b15c0cd9b3a3897310da2eb22999b) |
|
2025-12-08 04:01:54
|
what I vaguely remember from back then is that we used to just check whether there was an ICC profile in `PackedPixelFile` when converting to CodecInOut, but there always was one as a fallback (or there needed to be), so we introduced this to signal whether it’s there _just_ as a fallback |
|
|
|
Laserhosen
|
2025-12-08 04:03:49
|
That suggests the behavior in dec/jxl.cc is wrong, as in this case the ICC *is* just a fallback, but it's being treated as primary. |
|
2025-12-08 04:04:30
|
So my original blunt instrument fix might be right? |
|
|
|
spider-mario
|
2025-12-08 04:04:38
|
it’s possible that it has regressed since then (I’m pretty sure it worked at the time) |
|
2025-12-08 04:04:41
|
could be worth bisecting |
|
2025-12-08 04:05:08
|
I’ll give that a quick try |
|
|
|
Laserhosen
|
2025-12-08 04:08:06
|
Yeah, the original version of that code makes more sense to me. |
|
|
|
spider-mario
|
2025-12-08 04:09:00
|
just confirmed that it used to work (and that now it doesn’t), so I have two anchors for my `git bisect` and I can proceed |
|
2025-12-08 04:13:56
|
oh no, I’m the one who broke it |
|
2025-12-08 04:14:19
|
https://github.com/libjxl/libjxl/pull/3880 is the culprit |
|
2025-12-08 04:14:59
|
I wish I could have been more explicit about what it supposedly fixed |
|
2025-12-08 04:18:54
|
https://xkcd.com/979/ this but with my own past self |
|
|
|
Laserhosen
|
2025-12-11 08:04:24
|
^ Should I create an issue on github for this? (or a PR that reverts the jxl.cc part of #3880 as I've kind of done that anyway?) |
|
2025-12-11 08:04:40
|
No rush, just don't want it to disappear 🙂 |
|
|
|
jonnyawsom3
|
2025-12-19 07:15:55
|
Continuing from https://discord.com/channels/794206087879852103/803574970180829194/1451653791349870748
AFAIK, `num_color_channels` is only ever 1 or 3, Greyscale or RGB. The rest is done with color profiles and extra channels |
|
|
|
Tirr
|
2025-12-19 07:16:15
|
<@279366206249500683> use [JxlDecoderGetExtraChannelInfo](https://libjxl.readthedocs.io/en/latest/api_decoder.html#_CPPv429JxlDecoderGetExtraChannelInfoPK10JxlDecoder6size_tP19JxlExtraChannelInfo) to get more information about extra channels |
|
|
awxkee
|
2025-12-19 07:18:10
|
Hm, strange I didn't see that one, thanks! |
|
2025-12-19 07:18:22
|
How would be CMYKOGV signalled? |
|
|
|
Tirr
|
2025-12-19 07:18:43
|
that's... not specified afaict |
|
2025-12-19 07:19:48
|
I'd signal OGV channels using `kNonOptional` with extra channel names, but anyone can do whatever they want I guess |
|
2025-12-19 07:20:28
|
or maybe spot colors if that's appropriate? |
|
|
|
awxkee
|
2025-12-19 07:21:18
|
I actually have no idea, here is perhaps someone with strong understanding of spec is needed |
|
2025-12-19 07:21:24
|
CMYKOGV is just common CMYK |
|
2025-12-19 07:21:42
|
Might be less common than 4 channels CMYK itself, but still it's simple CMYK |
|
|
|
jonnyawsom3
|
2025-12-19 07:22:55
|
Only CMYK is formally defined in the spec, but as Tirr said, it could easily be encoded with the appropriate channel names too. It just wouldn't decode in your standard image viewer |
|
|
|
awxkee
|
2025-12-19 07:23:44
|
Yeah that's the point, I want to understand what to expect for those cases in my libraries |
|
2025-12-19 07:24:18
|
If I'd just want to encode for myself this is easy as I could encode in any way I want to |
|
|
|
Tirr
|
2025-12-19 07:27:17
|
any spec conforming decoders should interpret CMYK channels correctly, but there's no guarantee that they can add information from OGV channels since they might not know how to interpret those channels |
|
|
|
username
|
2025-12-19 07:56:24
|
If a library isn't designed to look for the extra OGV channels then what happens next depends on if said extra channels are set as `JXL_CHANNEL_UNKNOWN` or `JXL_CHANNEL_OPTIONAL`
IIRC `UNKNOWN` means that the decoder *should* refuse to decode the file while `OPTIONAL` means that it will still try to decode the image but ignore and not use the extra channels if it doesn't understand them. |
|
|
|
veluca
|
2025-12-19 09:45:48
|
<@794205442175402004> anything we should do here to specify this better? |
|
2025-12-19 09:46:02
|
(I admit I haven't the foggiest on what OGV channels might be) |
|
2025-12-19 09:47:10
|
for everyone else: how do such images get exchanged nowadays? |
|
|
|
awxkee
|
2025-12-19 10:20:31
|
PDF/X and thousand of incompatible formats are used |
|
|
|
_wb_
|
2025-12-19 10:30:28
|
orange green violet iirc, just more inks |
|
2025-12-19 10:32:15
|
In principle you could treat them as 3 spot colors on top of CMYK, with the advantage that it should be possible to make existing jxl decoders actually render it somewhat correctly (besides not being able to apply any actual color management to the OGV part) |
|
2025-12-19 10:33:54
|
or you could use optional or non-optional extrachans for this, with the name field saying what it actually is.
(optional or non-optional depending on whether you still want to just show the CMYK or if you want conforming decoders to give up and say "i cannot handle this" unless they can handle it) |
|
2025-12-19 10:36:26
|
but is there a real need to actually interchange 7-channel images? I would assume you can just interchange images in whatever color space has a wide enough gamut for your content, and leave it to the final output device to decide how it can best render that |
|
2025-12-19 10:38:31
|
(i.e. how to best convert it to CMYK or CMYKOGV or some funky 20-color process or whatever, which depends on the actual printing infrastructure that is available, economic considerations, the paper type you want to use, etc etc) |
|
2025-12-19 10:43:11
|
anyway if someone is actually representing 7-channel CMYKOGV in jxl, then just select some convention and then we can add it to some future edition of the spec |
|
|
|
awxkee
|
2025-12-19 10:43:34
|
My plan is rather to understand how to deal with untrivial cases, especially the one I know at this moment |
|
2025-12-19 10:45:27
|
And as possibilities to encode funny images at the moment are narrow, I'd expect that someone will start use it for interesting stuff as jxl gets more broad adoption |
|
2025-12-19 10:46:53
|
So I'm trying to create a plan in my head will I handle now funny profiles that I know, or won't |
|
2025-12-19 10:47:25
|
It seems that at the moment is better don't do that |
|
|
|
jonnyawsom3
|
2025-12-25 05:34:18
|
So what was the plan for level 3?
```/** Control what kind of buffering is used, when using chunked image frames.
* -1 = default (let the encoder decide)
* 0 = buffers everything, basically the same as non-streamed code path
(mainly for testing)
* 1 = buffers everything for images that are smaller than 2048 x 2048, and
* uses streaming input and output for larger images
* 2 = uses streaming input and output for all images that are larger than
* one group, i.e. 256 x 256 pixels by default
* 3 = currently same as 2
*
* When using streaming input and output the encoder minimizes memory usage at
* the cost of compression density. Also note that images produced with
* streaming mode might not be progressively decodable.
*/
JXL_ENC_FRAME_SETTING_BUFFERING = 34,```
Buffering the full image (0), LF blocks (1) and Groups (2) makes sense, but I'm guessing rows would be the only way to use 3 (or however you did PPM streaming)
Not to mention, right now it's "a single LF block or the entire image", "one group or the entire image, but it's actually the LF again" and "3 is the same as 2, which is the same as 1" <https://github.com/libjxl/libjxl/blob/53042ec537712e0df08709524f4df097d42174bc/lib/jxl/enc_frame.cc#L1787> |
|
2025-12-25 05:34:56
|
I've also got some ideas for low color/GIF compression, but that'll take a while and need some help |
|
|
|
runr855
|
2025-12-25 05:10:01
|
Why do I get a ssimulacra2 score of ~93 when comparing a jpeg to the losslessly converted jxl? When using djxl to get the original jpeg back, the files hashes are identical so nothing funny is going on in that regard |
|
|
|
lonjil
|
2025-12-25 05:43:59
|
The JPEG spec is pretty loose w.r.t. the results of the decoding process, so in practice, every JPEG implementation will decode a given file to slightly different pixels. |
|
|
|
jonnyawsom3
|
2025-12-25 05:57:35
|
Libjxl uses higher precision maths, so it won't be identical to the JPEG but it's actually more accurate (and you can always transcode back, which *is* identical) |
|
|
|
Exorcist
|
2025-12-25 06:41:22
|
|
|
2025-12-25 06:43:12
|
|
|
|
|
jonnyawsom3
|
2025-12-27 01:49:16
|
Continuing from https://discord.com/channels/794206087879852103/794206170445119489/1454290097540431967, this image is great at showing the desaturation caused by the B channel in multiple colors (Disable patches when encoding) |
|
2025-12-27 01:52:47
|
Original, VarDCT and Modular. Quick comparison at Distance 1 |
|
|
|
Demiurge
|
2025-12-27 01:57:41
|
https://entropymine.com/imageworsener/gamma/g/feather1.png |
|
2025-12-27 01:58:10
|
I wonder if this image gets noticeably darker after (lossy) encoding? |
|
2025-12-27 01:58:35
|
I'm not at a pc right now. |
|
2025-12-27 02:00:04
|
It's from here:
https://entropymine.com/imageworsener/gamma/ |
|
|
|
A homosapien
|
2025-12-27 02:42:33
|
Doing some simple testing with the lossy modular quant tables, a small ~1-3% increase in file size results almost completely fixes the yellow desaturation issue. |
|
2025-12-27 02:43:00
|
|
|
|
|
jonnyawsom3
|
2025-12-27 02:46:03
|
Original, Main, Fixed |
|
|
|
A homosapien
|
2025-12-27 02:46:15
|
Yellow is back on the menu boys |
|
|
|
jonnyawsom3
|
2025-12-27 02:47:08
|
A year long hypothesis confirmed before the new year |
|
|
|
username
|
2025-12-27 02:50:43
|
oo nice! I'm guessing y'all where testing with lossy modular because the code in libjxl is already setup to easily allow for custom quant tables with lossy modular specifically? |
|
|
|
jonnyawsom3
|
2025-12-27 02:57:41
|
It's just a LUT, so we edited the values for B <https://github.com/libjxl/libjxl/blob/53042ec537712e0df08709524f4df097d42174bc/lib/jxl/enc_modular.cc#L97> |
|
|
|
runr855
|
2025-12-27 03:03:31
|
Is this an actual potential fix for the whole desaturation issue, or a step on the way there? |
|
|
|
jonnyawsom3
|
2025-12-27 03:05:28
|
It's evidence pointing towards the quant tables being the issue, but trying to rebalance them won't be an easy task
For some reason *increasing* the quality of B, has now made B less saturated when we size-match to Main... Still work to do |
|
|
|
runr855
|
2025-12-27 03:07:26
|
I'm with you now, I've only read a bit of Discord and a few Github issues. I thought it already was established that the quant tables was the issue |
|
2025-12-27 03:08:45
|
When you say size-match, is that referring to file size? |
|
|
|
jonnyawsom3
|
2025-12-27 03:09:50
|
I had established that, but we didn't have any concrete evidence, just extrapolations from images. Foxless saying I was wrong in the other channel got me to grab Sapien to help prove it, which now we have |
|
2025-12-27 03:10:37
|
Yeah, a JXL at the same size with the fix has less saturated blue for some reason, even though the fix was giving blue higher quality |
|
2025-12-27 03:11:26
|
Lossy has a dozen different settings and things that influence the encoder, so it might take a while to figure out what's going wrong |
|
|
|
runr855
|
2025-12-27 03:14:38
|
Great that you've found a clue to the real cause. You're all doing great work with JPEG XL 🚀 |
|
|
|
jonnyawsom3
|
2025-12-27 03:19:11
|
So far we've fixed progressive, faster decoding, resampling and LZ77. v0.12 is gonna be juicy side-by-side with jxl-rs |
|
2025-12-27 03:24:32
|
Weirdly, it's almost like it's inverting the saturation. This is modular, so there shouldn't be any DCT artifacts |
|
2025-12-27 03:24:34
|
|
|
|
|
runr855
|
2025-12-27 03:32:01
|
Agree, an interesting consequence |
|
|
|
jonnyawsom3
|
2025-12-27 06:23:09
|
Well we figured out the squeeze in lossy modular is pretty broken, it causes patterns of alternating colors and heavy color bleeding
Ignoring that though, the quant table still worked pretty well. We'll definitely need help to change the VarDCT table though. I can't find any documentation about using parametic tables and it's a bit outside our coding expertise adding a whole new feature (we've mostly been tweaking existing code so far) |
|
|
|
runr855
|
2025-12-28 02:09:16
|
I just want to be certain, if I get a ssimulacra2 score of 100, does that mean the image is not impacted by desaturation? I just want to be sure ssimulacra2 doesn't work in a way that could overlook the problem |
|
|
|
RaveSteel
|
2025-12-28 02:18:35
|
You'll only get a score of 100 for lossless transcode. Only lossy encode is affected by the desaturation |
|
|
|
NovaZone
|
2025-12-28 02:56:37
|
If ur not doing large batches u could also do absolute subtraction with https://github.com/pixop/video-compare |
|
2025-12-28 02:56:58
|
Handles imgs as well |
|
|
|
jonnyawsom3
|
2025-12-28 03:05:33
|
The desaturation specifically effects VarDCT, so if you use `-q` or `-d` then it's affected. *However* it's mostly/only noticeable on the most saturated/vibrant colors like yellow and orange
TLDR; if you didn't notice it before, you're probably fine. Just consider this a 'free' quality/density boost once we have it sorted out |
|
|
|
NovaZone
|
2025-12-28 03:07:06
|
And 0,255,0 xD |
|
|
|
jonnyawsom3
|
2025-12-28 03:16:59
|
Technically anything where B is meant to be 0 |
|
2025-12-28 03:17:44
|
`255,0,0` `0,255,0` `255,255,0` would be the worst cases, with anything in-between showing symptoms |
|
|
|
NovaZone
|
2025-12-28 03:19:53
|
Indeed my own quick subtraction test of a color chart showed similar https://discord.com/channels/794206087879852103/794206087879852106/1397514863655518239 |
|
2025-12-28 03:23:18
|
Hopefully solving de-sat also solves the color shift |
|
|
|
jonnyawsom3
|
2025-12-28 03:48:23
|
Did a quick test using our jank fix, definitely reduced the excess blue in the bottom left |
|
|
|
NovaZone
|
2025-12-28 03:51:12
|
Can tell how much better that is even on my phone xD |
|
2025-12-28 03:51:18
|
Very nice 👍 |
|
2025-12-28 03:52:56
|
Good to know they are at least somewhat related |
|
|
|
username
|
2025-12-28 09:42:43
|
> That looks exactly like it, indeed! Good find. So the encoding color profile should only be set to "original" in lossless mode? That'll need some UI changes as well, I suppose.
https://github.com/darktable-org/darktable/issues/19995#issuecomment-3694556574 |
|
2025-12-28 09:45:06
|
what's the correct API usage to attach a color profile when encoding to something like a lossy XYB VarDCT image? (if possible provide this help to the darktable maintainers) |
|
2025-12-28 09:48:24
|
unless I'm misunderstanding this? |
|
2025-12-28 09:51:41
|
actually yeah it's possible I'm mistaking attached color profile with color profile used for encoding or something it seems |
|
|
|
Demiurge
|
2025-12-28 11:49:56
|
Looks like it murders magenta as well. |
|
2025-12-28 11:56:02
|
When you say it makes blue less saturated compared to main, are you saying this based off objective testing or based off subjective, visual testing? If it's based off objective testing (such as looking at a "diff") then this might not actually be a sign of a problem. It could be intentional - increasing the quality of the range of values our eyes are sensitive to and decreasing the quality outside of that range (our eyes are less sensitive to blue!) |
|
2025-12-28 11:58:09
|
And the quality of the B channel is also responsible for "yellow" which our eyes are very sensitive to! |
|
|
|
jonnyawsom3
|
2025-12-28 10:46:14
|
<@794205442175402004> do you know where the Parametrized quant tables are described in the spec? I see them mentioned but no examples of how we could use them |
|
|
|
runr855
|
2025-12-28 11:23:48
|
Is lossless conversion -d 0.0 at or very close to it's optimal size in the current encoder? Compared to lossy I'd guess lossless meets a hard limit in the spec? |
|
|
|
jonnyawsom3
|
2025-12-28 11:25:54
|
Nowhere close, some of the coding tools aren't used at all and there's near-infinite combinations of them, so there'll probably always be room for improvement unless your source is <#824000991891554375> |
|
|
|
runr855
|
2025-12-28 11:37:28
|
Great to hear, and I guess I could always losslessly re-encode again down the road for the additional gains? |
|
|
|
A homosapien
|
|
|
runr855
|
2025-12-28 11:42:55
|
Thanks |
|
|
|
_wb_
|
2025-12-29 08:20:09
|
Annex I.2.4, let me screenshot some of that |
|
2025-12-29 08:20:47
|
|
|
2025-12-29 08:21:11
|
|
|
2025-12-29 08:21:32
|
|
|
2025-12-29 08:21:53
|
|
|
2025-12-29 08:22:29
|
|
|
2025-12-29 08:23:04
|
|
|
2025-12-29 08:24:29
|
|
|
|
|
Tirr
|
2025-12-29 08:24:44
|
(if you rather prefer the actual code: https://github.com/libjxl/jxl-rs/blob/main/jxl/src/frame/quant_weights.rs#L1004) |
|
|
|
_wb_
|
2025-12-29 08:24:52
|
|
|
2025-12-29 08:26:16
|
it's a pretty large amount of spec, considering we have only been using all-default and RAW quant tables for now |
|
2025-12-29 08:29:01
|
iirc libjxl does have code for the encoder to take a quant table description, encode it, and decode it again so the encoder is sure to be using the same tables as the decoder |
|
|
|
username
|
2025-12-29 10:28:54
|
<@794205442175402004> what is the stance on using RGB for lossy? it seems like projects are either using/doing this by accident or intentionally exposing it to users both via `uses_original_profile` in the API however when encoding at efforts 7 and higher this causes strange unintentional visual artifacts.
Should the docs for the API warn/protect against using/doing this or should this be tracked on the libjxl repo as a bug or both?
https://github.com/darktable-org/darktable/issues/19995 |
|
|
Emre
|
2025-12-30 03:03:55
|
> what is the stance on using RGB for lossy?
from what I know, human vision != RGB
We are more sensitive to luminance than chrominance.
YCbCr separates brightness and color differences.
RGB channels are highly correlated and edges appear in all three channels. YCbCr decorrelates the signal and it improves transform efficiency (DCT, wavelets) and rate-distortion. |
|
2025-12-30 03:04:21
|
On the other hand, I would also like to know why, despite this YUV444 is pretty popular and good for lossy images. |
|
2025-12-30 03:04:49
|
It probably still allows uneven quantization |
|
2025-12-30 03:05:58
|
It's still YCbCr and it can strongly decorrelate. AFAIK you can't do that with RGB. Transforms have to code edges in R, G, B; all separately. |
|
2025-12-30 03:06:40
|
I wonder if this is probably another reason why XYB was invented. |
|
|
|
Tirr
|
2025-12-30 03:09:21
|
api docs say `use_original_profile` shouldn't be set for lossy https://libjxl.readthedocs.io/en/latest/api_metadata.html#_CPPv4N12JxlBasicInfo21uses_original_profileE
> Note that for lossy compression, this should be set to false for most use cases, [...] |
|
|
|
jonnyawsom3
|
2025-12-30 05:00:48
|
I left a comment explaining. While it's not recommended, RGB compression *is* supported, and is even an option in cjxl via `--disable_perceptual_optimizations`. As seen in <#803645746661425173> though, VarDCT is quite broken with it, Modular works quite well |
|
2025-12-30 05:45:38
|
Oh <@384009621519597581>, I meant to ask. If it's not too much of a pain to get it set up again, what about exposing these quant tables as a config file?
<https://github.com/libjxl/libjxl/blob/53042ec537712e0df08709524f4df097d42174bc/lib/jxl/enc_modular.cc#L93-L98>
Then we can find both good VarDCT weights and decent Modular quant tables without having to re-compile, hopefully ending up with better sharpness and color retention when we figure out how to edit the VarDCT quant table |
|
|
|
_wb_
|
2025-12-30 11:23:03
|
I think the main issue is that most applications don't read the API documentation very carefully, and that this variable is poorly named since it seems to suggest that if you set it to false, you will lose the color space signaling (which is not true), or something like that. |
|
|
|
RaveSteel
|
2025-12-30 11:24:20
|
I was surprised that setting the distance to 0 did not result in a lossless image without that parameter |
|
|
|
jonnyawsom3
|
2025-12-30 11:29:23
|
The API throws an error if you do that now |
|
|
|
RaveSteel
|
2025-12-30 11:30:32
|
how recent is "now"? |
|
|
|
jonnyawsom3
|
2025-12-30 12:05:41
|
4 years ago https://github.com/libjxl/libjxl/pull/1270 |
|
|
|
RaveSteel
|
2025-12-30 12:18:51
|
Hm, I can't recall getting an error message when i tried it last week |
|
|
|
_wb_
|
2025-12-30 12:30:59
|
setting the distance to 0 is not the same thing as calling SetFrameLossless(). Though probably we should make it do the same thing, actually... |
|
|
|
RaveSteel
|
2025-12-30 12:57:47
|
I had used SetFrameLossless and set distance to 0, because neither resulted in a lossless image, so I tried both |
|
2025-12-30 12:59:25
|
Only after I correctly also set uses_original_profile did it result in a lossless image |
|
2025-12-30 12:59:40
|
but without it, setting distance to 0 resulted in a lossy image |
|
|
|
AccessViolation_
|
2025-12-30 02:36:02
|
oh ye I'll see if I can get to that later today |
|
2025-12-30 02:36:12
|
I'll mark the message as unread so I see the ping and don't forget |
|
|
|
jonnyawsom3
|
2025-12-31 07:40:07
|
I just found another example but with pure Red this time. Compressing in RGB with Resampling 2 manages to look better than VarDCT or Modular in Main and v0.8... We're gonna need help to get those new quant tables implemented, and to find the right values for a new set. The AC seems bitstarved for both X and B, but B is between 2-4x worse. That's why chroma noise has been fairly common in certain images or low qualities |
|
2025-12-31 08:45:54
|
More tests later, and it looks like Gaborish is causing problems too. Ringing and sharpening the chroma artifacts caused by the quant tables (I think) |
|
2025-12-31 11:22:24
|
Yet another thought, the B in XYB is Blue-Yellow, and in JXL the B channel is stored as B-Y, with Y being luminance. By that logic, saturated yellow should naturally make the B at it's lowest value, so quantization rounding down should only make yellows *more* saturated, not less. I'll have to dig through the code after New Year's and figure out what kind of rounding is going on |
|
2026-01-03 04:45:10
|
I know there's nothing inherently wrong with them, but the AI PRs feel a bit... Hrmmmm
https://github.com/libjxl/jxl-rs/pull/604 |
|
|
|
tee
|
2026-01-04 05:41:44
|
```cjxl -j 1 -d 0 -e 7 ./85.png ./85.jxl
JPEG XL encoder v0.11.1 794a5dcf [AVX2,SSE4,SSE2]
libpng warning: iCCP: profile 'icc': 'GRAY': Gray color space not permitted on RGB PNG
Getting pixel data failed.```
Seems to be a known bug, but is there any workaround? Running it through oxipng first doesn't seem to help: `oxipng -o max --strip tEXt,zTXt,iTXt,eXIf,tIME --force` |
|
|
|
_wb_
|
2026-01-04 08:08:48
|
you'll have to either strip that icc profile or convert the PNG to grayscale |
|
|
|
spider-mario
|
2026-01-04 09:22:06
|
this one should work |
|
2026-01-04 09:25:14
|
ah, imagemagick apparently dropped the ICC |
|
2026-01-04 09:25:17
|
that might be the reason |
|
|
|
AccessViolation_
|
2026-01-04 04:12:29
|
sorry for the wait, I'm working on this now |
|
2026-01-04 04:36:02
|
well, whenever this ends |
|
2026-01-04 04:36:56
|
first request of the day hit a rate limit apparently |
|
|
|
tee
|
2026-01-04 11:42:43
|
Is stripping the png icc profile a lossless operation? Or is it worth keeping the icc profile?
`oxipng -o max --strip iCCP --force` |
|
|
dogelition
|
2026-01-04 11:57:00
|
lossless in the sense that the pixel data is preserved, but the appearance in any viewer that would make use of that icc profile will probably be changed (unless it's just the standard srgb profile i guess)
(also this should probably be in another channel unless it's specifically about libjxl's output, e.g. <#805176455658733570>) |
|
|
|
tee
|
2026-01-05 02:36:38
|
```cjxl -j 1 -d 0 -e 7 "./001.jpg" "./001.jxl" -v -v -v -v
JPEG XL encoder v0.11.1 794a5dcf [AVX2,SSE4,SSE2]
Read JPEG image with 1637805 bytes.
Encoding [JPEG, lossless transcode, effort: 7]
Error while decoding the JPEG image. It may be corrupt (e.g. truncated) or of an unsupported type (e.g. CMYK).
EncodeImageJXL() failed.```
Jpg colorspace is sRGB according to imagemagick identify. Github issue says to use jpegtran, but that doesn't seem to help: `jpegtran -copy none -perfect -optimize -outfile "./001.jpg" "./001.jpg"` |
|
|
whatsurname
|
2026-01-05 08:12:19
|
<@&807636211489177661> scam |
|
|
|
spider-mario
|
2026-01-05 10:21:41
|
I don’t think jpegtran can work in-place like this; it needs different input and output filenames |
|
2026-01-05 10:21:55
|
(what I often do is use `.jpg` for one and `.jpeg` for the other) |
|
2026-01-05 10:22:32
|
it also only takes the input as a command-line argument |
|
2026-01-05 10:22:39
|
the output needs to be a redirection AFAIK |
|
2026-01-05 10:23:19
|
ah, missed the `-outfile` |
|
2026-01-05 10:23:25
|
I stand corrected |
|
2026-01-05 10:24:16
|
I most often use it as:
```console
$ jpegtran -copy none -optimise -progressive < input.jpg > output.jpg
``` |
|
|
Gonzalo Muñoz
|
2026-01-05 05:19:04
|
Hi! Quick question about the reference implementation: I know JPG reconstruction can be bitwise lossless (i.e. like zip), but I haven't found anything about the rest of the formats. Can we restore them verbatim as files, or it's more like pixel-perfect kind of lossless? |
|
|
|
AccessViolation_
|
2026-01-05 05:28:04
|
JPEG is the only format supported by JXL's lossless recompression like that |
|
|
|
Gonzalo Muñoz
|
2026-01-05 05:28:22
|
OK, thanks |
|
|
|
_wb_
|
2026-01-06 06:55:31
|
We could define the equivalent of `jbrd` for some other formats too in principle, I have made an experimental PSD recompression thing, but abandoned it since the PSD format is too poorly documented and changes with every new Photoshop release. |
|
2026-01-06 06:59:33
|
For formats that have entropy coding with many different ways of encoding the same uncompressed data, it will be tricky to keep the bitstream reconstruction data small though. E.g. for PNG, in general, the overhead would be significant (need to store the details of which lz77 matches were used), though you could have shortcuts for common PNG encoders like libpng. |
|
|
|
jonnyawsom3
|
2026-01-06 07:13:47
|
Reminds me of my proposal to Krita https://bugs.kde.org/show_bug.cgi?id=500877 |
|
|
|
AccessViolation_
|
2026-01-06 09:27:36
|
I feel like reversible recompression at the byte level for lossless formats wouldn't have nearly as many use cases regardless |
|
2026-01-06 09:29:21
|
if you can get the pixel data and metadata back in the original format, it'll be semantically identical. you just couldn't use it in file system compression or http content-encoding or things like that |
|
|
|
jonnyawsom3
|
2026-01-06 09:55:47
|
Now if only we could get metadata back https://github.com/libjxl/libjxl/issues/2641 |
|
|
|
Exorcist
|
2026-01-06 11:11:16
|
The people use metadata as main payload:
https://xkcd.com/1172/ |
|
|
|
monad
|
2026-01-07 07:45:10
|
example of PNG recompression leveraging preflate: <https://gist.github.com/lifthrasiir/5c24058f21ce6fba231cf1bfba45bf28> |
|
|
|
jonnyawsom3
|
2026-01-08 06:00:59
|
<@384009621519597581> looking back on this gave me some thoughts https://discord.com/channels/794206087879852103/822105409312653333/1457816277975306250 |
|
2026-01-08 06:02:45
|
Edge detection could help find image features, something similar to the VarDCT selection could find areas of identical color, or LZ77 could find repetitions of patterns and tiles |
|
2026-01-08 06:03:14
|
Lots of different ways to extract different types of patches |
|
|
|
Magnap
|
2026-01-08 07:54:01
|
Something akin to the matrix profile could also work? If adapted to 2D. For a vector of length n and a subsequence length, for each subsequence of that length it's the most similar (by Euclidean distance) subsequence elsewhere in the vector and its similarity, and there's an algorithm to find it in O(n²) (independent of the subsequence length) |
|
|
|
AccessViolation_
|
2026-01-08 08:04:36
|
it would be cool if we had different algorithms for patch detection at different effort levels, so even lower effort levels benefit from *some* attempts at replacing repeating elements with patches |
|
2026-01-08 08:07:16
|
effort 2 for lossless could be incredibly simple, just splitting everything into 8x8 or 16x16 blocks and deduplicating those. if repeating blocks have sufficient entropy (e.g. they're not one color, since then patching likely won't be worth it), replace the blocks with patches |
|
2026-01-08 08:10:30
|
like literally just creating a hashmap of 8x8/16x16 blocks to keep track of their count and positions and their estimated entropy (entropy calculated once, as soon as a block appears for the second time) should be enough. then replace higher entropy, more common blocks with patches |
|
2026-01-08 08:13:41
|
I would almost say this would probably be fast enough for e1, but it's probably good to keep e1 really simple just so we have something that's very very fast |
|
|
|
veluca
|
2026-01-08 09:35:50
|
there's no way that's fast enough for e1 |
|
2026-01-08 09:35:51
|
😛 |
|
|
|
AccessViolation_
|
2026-01-08 09:57:49
|
oh welp |
|
|
|
Magnap
|
2026-01-09 09:46:38
|
Not sure who to ping to ask, but out of curiosity, how does libjxl detect patches at the moment? |
|
|
|
monad
|
2026-01-09 10:14:44
|
https://github.com/libjxl/libjxl/blob/53042ec537712e0df08709524f4df097d42174bc/lib/jxl/enc_patch_dictionary.cc#L227 |
|
|
|
_wb_
|
2026-01-09 10:16:02
|
@veluca made that, basically it detects regions of solid color first (4x4 iirc), then looks for patches (that are not just solid color) in the neighborhood of those, finds ones that occur more than once, does some fuzzy matching in case of lossy, exact matching in case of lossless, and then packs them into a patch frame. I'm not sure if I understand all the details of it. It works quite well but I'm sure there are many directions to explore to improve this, in particular to make it more general and not require solid background. |
|
|
|
monad
|
2026-01-09 10:18:23
|
patch candidates also need to be completely surrounded by the detected background color |
|
|
|
AccessViolation_
|
2026-01-09 10:20:56
|
interesting, I didn't know we already did fuzzy matching. I knew we did kAdd to make them effectively lossless, but thought we only did exact matching currently |
|
2026-01-09 10:21:46
|
that's nice |
|
|
|
veluca
|
2026-01-09 10:29:38
|
wonder why that didn't tag me 😄 |
|
|
|
AccessViolation_
|
2026-01-09 10:30:08
|
if you don't press enter but keep typing it doesn't turn it into a mention |
|
|
|
veluca
|
2026-01-09 10:30:42
|
and one thing it certainly doesn't do is detect largeish repeated patterns such as sprites or repeated icons |
|
|
|
spider-mario
|
2026-01-09 10:33:34
|
I keep fantasising about using JXL as a format for NES/SNES/GB/etc. screenshots that directly reuses the underlying sprites as patches |
|
|
|
AccessViolation_
|
2026-01-09 10:33:50
|
a while ago I tried to think of some sort of stateful algorithm that could cheaply detect repeating sequences, by utilizing something like a move to front transform, and the transform buffer would be watched, but that didn't really turn into anything |
|
2026-01-09 10:34:26
|
it would not be as precise as LZ77, more just like a signal of "there might be repeating stuff here" |
|
2026-01-09 10:36:52
|
recently I was also thinking about Firefox's built-in full-page screenshot tool that was used to generate the massive Wikipedia PNG that we used to benchmark JXL and AVIF, and I thought that since Firefox knows where the lossy images, html/css-defined graphics, and text are, it could easily put those three distinct layers compressed independently |
|
2026-01-09 10:38:32
|
it's not one of those screenshot tools that scrolls the viewport and stitches them together, it seems to actually hook into the renderer to create the capture, so there's already a lot of potential there |
|
2026-01-09 10:41:43
|
|
|
2026-01-09 10:41:44
|
:D |
|
2026-01-09 10:50:35
|
I don't remember the details of this exactly but IIRC NES sprites and large backgrounds are also limited to a fixed, low number of colors per larger block, but the final frame as a whole can have many colors. so depending on the type of content you can also create up to four patch reference frames (or even more transparent layers which are all blended together), grouping a few of those color palettes together, so you get really tiny palette indices, too |
|
|
|
jonnyawsom3
|
2026-01-09 11:01:56
|
If only we could JPEG transcode into patch frames |
|
2026-01-09 11:04:17
|
As Monad said
> patch candidates also need to be completely surrounded by the detected background color
I assume it's kAdd so they can fill in the gap with the solid background color, otherwise removing the patch could cause more entropy than it saves. kReplace and zeroing residuals could work though |
|
|
|
AccessViolation_
|
2026-01-09 02:19:40
|
I was compressing a screenshot of the community website, and it has these artifacts. could these have been accidentally detected as patches? |
|
2026-01-09 02:20:16
|
|
|
2026-01-09 02:34:37
|
source PNG |
|
2026-01-09 02:35:50
|
```
% cjxl source.png q50.jxl -q 50 --patches 1
JPEG XL encoder v0.11.1 0.11.1 [AVX2,SSE4,SSE2]
``` |
|
|
Mine18
|
2026-01-09 03:01:45
|
https://www.youtube.com/watch?v=ZWQ0591PAxM
this video covers the techniques used here |
|
|
|
AccessViolation_
|
2026-01-09 03:02:47
|
yeah I linked to that as well under the message that was forwarded |
|
2026-01-09 03:03:22
|
it's a cool project, I should get the game on Steam at one point. apparently it comes with a NES emulator if you do so it just works like a normal program |
|
2026-01-09 03:03:59
|
I saw they even created a level editor (itself external to the game, for obvious reasons) |
|
|
|
username
|
2026-01-09 08:14:54
|
oh hey this reminds me of when I did this: https://discord.com/channels/794206087879852103/803574970180829194/1043213863304302632 |
|
|
|
AccessViolation_
|
2026-01-09 08:16:56
|
It's kind of a neat benchmark encoders that aim to detect screen content because there's gradients and non-flat background all over the place |
|
2026-01-09 08:18:07
|
unlike for example the massive wikipedia screenshot we used, which is just: use patches for letters and you've done a large part of the optimization |
|
2026-01-11 02:10:31
|
for photographic images, I wonder if it's better to slightly denoise, then build an MA tree based on that, then update the residuals (and histograms) so it results in the original image, rather than trying to build the MA tree on the noisy image |
|
2026-01-11 02:11:30
|
I've been wondering about this for a while, but recently I saw that video encoders do this thing where they denoise a frame before doing motion prediction, because the noise makes it less accurate, so now I think there might be something to it |
|
|
|
jonnyawsom3
|
2026-01-11 02:25:22
|
MA is Modular, which is only used on the DC of VarDCT, so it's already 8x downsampled |
|
|
|
AccessViolation_
|
2026-01-11 02:40:29
|
ah sorry, I meant for lossless compression of photographic images |
|
|
|
A homosapien
|
2026-01-12 03:55:26
|
So basically a lossless gaborish? |
|
|
|
AccessViolation_
|
2026-01-12 10:39:15
|
gaborish is an encode-time sharpeding and decode-time blurring, so kind of the opposite |
|
2026-01-12 10:40:22
|
the thing about MA trees is that they can't really be 'wrong', just inefficient. with the right residuals you'll always get the right image back. you could theoretically take the MA tree from one image and apply it to another, fix it up with the right residuals, and it'll render correctly, it'll just compress poorly probably |
|
2026-01-12 10:44:09
|
though, with the way MA trees are currently built - trying a bunch of options and seeing which is the best - it might not matter if the image is denoised before that 🤔 |
|
|
|
jonnyawsom3
|
2026-01-14 08:38:02
|
I think I just realised it's a one-line change to get ProgressiveDC loading in libjxl based browsers again... Oh well, better 3 years late than never |
|
2026-01-14 08:49:54
|
Scratch that, not implemented 😔 |
|
|
|
monad
|
|
|
whatsurname
|
2026-01-18 06:01:10
|
Is https://github.com/libjxl/conformance/blob/master/testcases/animation_newtons_cradle/input.jxl lossy? Every frame after frame 9 has artifacts |
|
|
|
jonnyawsom3
|
2026-01-18 06:14:24
|
Might be related to this https://github.com/libjxl/libjxl/issues/3134 |
|
2026-01-18 06:17:23
|
Yeah, looks like it's lossless but the disposal regions weren't loaded correctly `JPEG XL animation, 480x360, (possibly) lossless, 8-bit RGB+Alpha` |
|
|
|
Demiurge
|
2026-01-19 09:13:14
|
<@238552565619359744> underrated JXL champ |
|
|
|
jonnyawsom3
|
2026-01-19 09:23:31
|
Not quite |
|
2026-01-19 11:23:01
|
After finding an image that requires a massive amount of dithering and causes moire patterns from the repetition, I'm trying to think if we can do the blue noise dither more efficiently. Using a smaller LUT but offsetting it per channel to avoid obvious repetition, maybe a 1D LUT offset per row, not sure what kind of results or performance impact it would have though. Feel free to make suggestions |
|
|
|
Exorcist
|
2026-01-19 11:47:08
|
adding noise before noise shaping (error diffusion) |
|
|
|
jonnyawsom3
|
2026-01-19 11:48:01
|
We don't do noise shaping |
|
|
|
Exorcist
|
2026-01-19 11:51:33
|
The textbook dithering for uniform quantization is:
1. add TPDF noise
2. noise shaping (also known as error diffusion) |
|
2026-01-19 11:51:55
|
There is no such thing that "directly add blue noise" |
|
2026-01-19 11:53:09
|
[white noise become blue] is the result of noise shaping |
|
|
|
jonnyawsom3
|
2026-01-20 12:20:20
|
Maybe I'm getting confused then, we just compare the decimal against a blue noise LUT and round up or down depending on the LUT's value, there's no second step |
|
|
|
lonjil
|
2026-01-20 12:20:32
|
One way of generating blue noise is to take white noise and do some funny stuff to it, but that's not at all like error diffusion dithering. Once you have your blue noise texture, you apply it exactly like white noise dither, that is to say, as a simple threshold map. |
|
2026-01-20 12:20:45
|
That's exactly correct. |
|
|
|
jonnyawsom3
|
2026-01-20 12:23:23
|
Right now it's a 32x32 LUT that's identical for every channel, I'm thinking of either making it a 1x1024 LUT and offsetting based on row, or offsetting the existing LUT per channel. Both should avoid repetition better without much cost, could probably halve the LUT and get some performance back if it goes well
Could also offset based on frame number, like photon noise does for some temporal dithering |
|
|
Adrian The Frog
|
2026-01-20 04:40:06
|
it will only be blue in one dimension if you use a 1x1024 LUT... |
|
|
|
jonnyawsom3
|
2026-01-20 04:43:50
|
...good point |
|
|
|
Exorcist
|
2026-01-20 07:10:56
|
> compare the decimal against a blue noise LUT and round up or down depending on the LUT's value
[stochastic rounding by remainder] is equal as [add 1-LSB noise then floor]
and, you need prove your "blue noise LUT" is better than standard RPDF noise |
|
|
eddie.zato
|
2026-01-20 07:20:55
|
The last release of libjxl was over a year ago, and it's not version 1.0. I'm interested in learning about the current state of development, as all my images are in jxl format. |
|
|
|
_wb_
|
2026-01-20 08:02:00
|
focus has shifted to jxl-rs for now, for obvious reasons. But when the chrome/firefox dust settles, we should go back to libjxl and get a v1.0 released, or at least a v0.12 |
|
|
|
jonnyawsom3
|
2026-01-20 08:41:17
|
The firefox covered in Chrome dust |
|
|
Pheubel
|
2026-01-20 09:36:55
|
forgive my ignorance, but if the focus is to get jxl-rs to reach feature parity with libjxl, wouldn't it make more sense to keep focus on the rust implementation? or is it more for projects that implemented the reference implementation |
|
2026-01-20 09:37:50
|
i don't really know how big the overlap is between libjxl and jxl-rs's dev team |
|
|
|
veluca
|
2026-01-20 09:39:04
|
pretty high 😛 |
|
2026-01-20 09:39:27
|
but jxl-rs doesn't have an encoder (yet?) |
|
|
|
AccessViolation_
|
2026-01-20 09:40:08
|
yet!! |
|
|
|
veluca
|
2026-01-20 09:41:34
|
I might try to make a lossless encoder once I am happy with the decoder |
|
2026-01-20 09:41:42
|
and a lossy encoder once I am happy with the lossless encoder |
|
2026-01-20 09:41:47
|
keyword being _might_ |
|
|
|
jonnyawsom3
|
2026-01-20 09:43:05
|
jxl-rs needs images to decode in the first place, so fixing up libjxl benefits both (Eg, making faster decoding work for all the decoders) |
|
|
|
AccessViolation_
|
2026-01-20 09:44:11
|
if there's going to be encoding, I really hope it's going to be nice and modular so it'll be relatively easy to plug new encoding methods into the CLI. imagine if some new idea we want to test could just be in its own Rust module, and selected with a CLI flag similar to `tune=` in avifenc |
|
2026-01-20 09:46:06
|
I personally think that with such an expressive format, making experimentation like that relatively easy will be really valuable to JXL enoding as a whole. there will always be unique tunes and heuristics to try out |
|
|
|
jonnyawsom3
|
2026-01-20 10:04:01
|
JXL was aiming not to have tunes, and do everything heuristically so you get the optimal output with minimum setup, but it could be useful for experimental build options... Though, I don't think there's been any experimental options, PRs just wait until they're ready for release. Closest currently would be lossy_palette |
|
|
|
AccessViolation_
|
2026-01-20 10:14:51
|
I think that's a great philosophy for the canonical encoder mode that people will end up running, but if the heuristics are rather monolithic, it's going to require changing the code in a bunch of places to experiment with new ideas. it might be more in-line with the philosophy to call them "experiments" rather than "tunes", and if they turn out valuable, they can be implemented into the canonical encoder mode? |
|
2026-01-20 10:17:35
|
like if we wanted to try this, for example:
<https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9505599>
it might take a lot of time to implement, only to find out it's not worth it. I think it would be nice if you could throw this method of encoding into its own module so it can be experimented with, and if fruitful, implemented into the default encoder |
|
|
|
Demiurge
|
2026-01-20 10:22:57
|
You need a large LUT. And an offset might result in unforeseen problems. |
|
|
|
jonnyawsom3
|
2026-01-20 10:26:36
|
The existing LUT is already a 1-2% decode speed penalty. Offsetting the LUT per channel should only be an improvement, since then it would spread the repetition between the channels instead of just laying it on top |
|
|
|
veluca
|
2026-01-20 10:27:05
|
is it 32x32 but actually 48x32 in memory? |
|
2026-01-20 10:27:33
|
also, floats? |
|
|
|
Demiurge
|
2026-01-20 10:27:36
|
I think you're supposed to use 3 different LUTs. One per channel. |
|
2026-01-20 10:28:05
|
But I could be wrong |
|
2026-01-20 10:28:13
|
It's been a while since I read about dithering |
|
|
|
jonnyawsom3
|
2026-01-20 10:28:23
|
<https://github.com/libjxl/libjxl/blob/main/lib/jxl/render_pipeline/stage_write.cc#L64> |
|
|
|
veluca
|
2026-01-20 10:28:25
|
if all the above are true, then it's ~6kB of memory, which is a lot |
|
|
|
jonnyawsom3
|
2026-01-20 10:29:46
|
Yeah, hence seeing if I can cut it down to a 16x16 with the offsets to hide the repetitions a bit more |
|
|
|
veluca
|
2026-01-20 10:30:05
|
uh, I am not sure I truly understand the logic there |
|
2026-01-20 10:30:45
|
in part because I don't remember why you'd ever use LoadDup128 |
|
|
|
Demiurge
|
2026-01-20 10:31:06
|
A large LUT is the cost that must be paid for blue noise dither... |
|
|
|
jonnyawsom3
|
2026-01-20 10:31:32
|
I originally wanted to use RGB blue noise, but that would've made the LUT even bigger and we were already getting a slowdown, so we found a decent compromise |
|
|
|
veluca
|
2026-01-20 10:32:14
|
I wonder how good a 16x16 8-bit LUT would be |
|
|
|
jonnyawsom3
|
2026-01-20 10:32:30
|
The LUT was originally an 8x8 bayer, all we did was swap the values and adjust the matrix size to match, didn't look into it much after since it just works™ |
|
|
|
veluca
|
2026-01-20 10:33:28
|
so the part that confuses me is that I'd expect this code to do Weird Things (tm) at the right border of an image |
|
2026-01-20 10:33:37
|
or something like that |
|
|
|
Demiurge
|
2026-01-20 10:33:44
|
I don't think it's possible to make it too small without paying the price for it. |
|
|
|
veluca
|
2026-01-20 10:33:46
|
but whatever 😄 |
|
|
|
Demiurge
|
2026-01-20 10:34:04
|
A large LUT is just the price that must be paid |
|
|
|
jonnyawsom3
|
2026-01-20 10:34:36
|
It looks better than bayer and that's all we were aiming for, but since v0.12 is delayed I've been thinking about more tweaks we could fit into it |
|
|
|
Demiurge
|
2026-01-20 10:44:23
|
Maybe there is a different way of encoding it and applying it that could be more efficient on memory and throughput |
|
|
|
jonnyawsom3
|
2026-01-20 10:51:47
|
An old idea was to run the Photon Noise decode path, since it's blue noise-ish, but it also has a hefty speed penalty |
|
|
|
veluca
|
2026-01-20 11:15:42
|
yeah no 😄 |
|
|
|
jonnyawsom3
|
2026-01-20 12:00:53
|
I think it'll boil down to "try a bunch and see what works"
If we can lower the LUT size, yippee |
|
|
|
veluca
|
2026-01-20 01:02:19
|
I'd be surprised if it can't be converted to 8-bit entries or worst case 16-bit entries |
|
|
|
username
|
2026-01-20 01:10:03
|
the 32x32 LUT was the lowest I could go before patterns started becoming noticeable. Months later I found out that NVidia seemingly came to a similar conclusion: https://developer.nvidia.com/blog/rendering-in-real-time-with-spatiotemporal-blue-noise-textures-part-2/#tiling_blue_noise |
|
2026-01-20 01:21:56
|
I had tried regenerating 16x16 patterns over and over again but I kept noticing tiling |
|
|
|
ignaloidas
|
2026-01-20 01:25:37
|
wouldn't that hurt SIMD perf tho? You need to load as f16/u8 and then convert to f32 |
|
|
|
veluca
|
2026-01-20 01:27:04
|
that can be done fairly quickly though |
|
2026-01-20 01:27:35
|
even with offsetting? |
|
|
|
username
|
2026-01-20 01:27:58
|
haven't tried offsetting |
|
|
|
veluca
|
2026-01-20 01:28:02
|
also you can load the relevant parts of the f32 bit pattern |
|
2026-01-20 01:28:39
|
say, the top 8 bits of mantissa for a float in [1, 2), and then you subtract 1.5 |
|
2026-01-20 02:12:38
|
ah, actually offsetting might be a bad idea because it makes gray no longer gray |
|
|
|
jonnyawsom3
|
2026-01-20 04:21:45
|
Ah right, hmm... Though, so long as the noise is still blue, it should average to grey, no? |
|
|
|
veluca
|
2026-01-20 04:36:27
|
sure, but still |
|
|
|
AccessViolation_
|
2026-01-20 05:20:42
|
speaking of, I wonder if that's also why JXL has color-bleeding artifacts rather than pure luma artifacts like some other codecs: this might be because the three color channels have different quantization tables |
|
2026-01-20 05:21:24
|
I don't remember whether I've brought this up before. it seems plausible |
|
2026-01-20 05:22:51
|
though I guess since they're fairly decorrelated thanks to the color space, you might still get that even if you used the same quant tables for all channels. since every channel is different data |
|
2026-01-20 05:24:02
|
I remember being surprised the artifacts in AVIF were mostly luma-based. there was no color bleeding, just darkening/brightening |
|
2026-01-20 05:24:26
|
sorry, this is off-topic |
|
|
|
Laserhosen
|
2026-01-20 06:02:19
|
A bit late, but here's the little thing I made for playing with multi-frame JXLs.
https://github.com/alistair7/jxltk
In its current form it's probably not that helpful to the libjxl tools, but I've found it useful as a way of accessing some of the blending options in the API that cjxl doesn't expose. |
|
|
|
jonnyawsom3
|
2026-01-20 08:09:23
|
Cranking up the chroma quant table quality fixed the bleed and desaturation, but trying to add new tables to libjxl is a few grades above my level |
|
|
|
RaveSteel
|
2026-01-20 08:55:42
|
does it support muxing JPEG transcoded JXLs together without bloating up the filesize? |
|
|
|
jonnyawsom3
|
2026-01-20 09:14:54
|
> It's strictly limited to the capabilities of the libjxl API, meaning almost every operation involves re-encoding pixels. For this reason, it defaults to lossless compression. Even so, any special effects (noise, splines, spots, ...) in the input will get rasterised in the output. |
|
|
|
RaveSteel
|
|
|
Laserhosen
|
2026-01-20 10:12:42
|
Yeah, last time I checked (which admittedly was a while ago) the API refused to accept multiple jpeg frames, or mix pixel frames with jpeg frames |
|
|
|
username
|
2026-01-21 12:51:52
|
in voice chat we found out that the use of `LoadDup128` was actually causing the the 32x32 LUT to get loaded wrong: https://cdn.discordapp.com/attachments/794206087879852107/1463492790141980764/bababababababababab.webp
Additionally we managed to add and test RGB offsetting to the dither which looks quite nice in relevant cases! (I don't have any visual examples on hand) |
|
|
|
veluca
|
2026-01-21 12:53:55
|
that makes sense, LoadDup128 doesn't *feel* like the right thing to use |
|
|
|
jonnyawsom3
|
2026-01-21 12:54:46
|
We made the blue noise bluer |
|
|
|
username
|
2026-01-21 12:55:16
|
LoadDup128 did seem to work fine when the LUT was wayy smaller |
|
|
|
veluca
|
2026-01-21 12:56:55
|
so one thing I still don't like is that with this code, the dithering result depends on the offset |
|
2026-01-21 12:57:58
|
say offset is (31, 0): then you'd load, for pixel 32, the dither value at position (0, 1). If instead you process directly pixel 32, you'd correctly load dither value (0, 0) |
|
2026-01-21 12:59:09
|
AFAICT the easiest solution is to instead have a 48x32 table |
|
2026-01-21 12:59:23
|
(i.e. 16-pad each row) |
|
|
|
jonnyawsom3
|
2026-01-21 01:23:19
|
I think our brains are all melting from staying up all night working on this, but I think we're starting to understand. Right now the offset can cause the next row to be sampled, meaning it's not maintaining the blue noise pattern? So you're suggesting instead of padding at the end, we pad each row so the offset loops it back around |
|
|
|
veluca
|
2026-01-21 01:24:42
|
yep |
|
2026-01-21 01:25:04
|
to be clear, it's still a huge improvement over whatever the old code was doing 😄 |
|
|
|
username
|
2026-01-21 01:26:45
|
context for anyone else looking in here we are talking about this PR: https://github.com/libjxl/libjxl/pull/4559 |
|
|
|
_wb_
|
2026-01-21 01:28:40
|
Whenever we have found a good way to dither, it would be good to make jxl-rs do something similar, and also to make as many viewers as possible also do this if possible whenever they downscale images. E.g. it would be nice if Chrome would also apply this when showing downscaled images. |
|
|
|
jonnyawsom3
|
2026-01-21 01:38:54
|
|
|
2026-01-21 01:38:55
|
Made a quick demo to show it off |
|
|
|
A homosapien
|
2026-01-21 01:51:52
|
Simple yet effective, it looks even better still. I just implemented your padding idea to the PR. |
|
|
|
jonnyawsom3
|
2026-01-21 01:58:56
|
I assume you mean downsample in bitdepth? But yes, replicating the libjxl behaviour would be nice now that we have it figured out. Still need to fix the 'Linear when uint' bug too though... |
|
|
|
veluca
|
2026-01-21 01:59:27
|
Later 😛 |
|
|
|
username
|
2026-01-21 02:13:12
|
speaking of, aren't there compat concerns relating to the Linear vs LinearSRGB stuff? |
|
2026-01-21 02:15:23
|
currently the API seems to not match the docs and the PR that's open for it also doesn't match the docs it seems |
|
|
|
veluca
|
2026-01-21 02:15:48
|
ok so now how about this...
1: convert the dithering pattern to u8 in range 1 to 255 (inclusive)
2: load a vector of u8s, zero-extend them to u32 (two calls to PromoteTo in highway)
3: OR the u32s with 0x47000000 and subtract 32768.5 |
|
2026-01-21 02:16:26
|
that should give the same noise pattern (but quantized to 8 bits of precision, which I would guess is enough), but use 1/4 of the memory |
|
|
|
A homosapien
|
2026-01-21 02:16:46
|
maybe for a future PR 😅 |
|
|
|
veluca
|
2026-01-21 02:16:50
|
hahah |
|
2026-01-21 02:17:21
|
fair enough |
|
|
|
MissBehavior
|
2026-01-21 04:26:13
|
I'm ngl struggling for time to learn to script the cli, and metadata to automate conversion. Is there a gui that can do it in bulk? |
|
|
|
username
|
2026-01-21 04:29:01
|
this maybe? https://github.com/JacobDev1/xl-converter (WARNING: this tool strips metadata by default!) |
|
|
|
MissBehavior
|
2026-01-21 04:30:18
|
Thank you! Hopefully it can take cli args or give access to all encoder settings. This might be what I needed! |
|
|
|
username
|
2026-01-21 04:33:07
|
hmm I'm not 100% sure if it does. another option is this tool which is JXL focused and allows overriding the cli args: https://github.com/kampidh/jxl-batch-converter/ |
|
|
|
MissBehavior
|
2026-01-21 04:33:52
|
I am going to try |
|
2026-01-21 04:33:57
|
both. |
|
2026-01-21 04:33:59
|
Ty |
|
|
|
username
|
2026-01-21 04:36:57
|
oh also the second tool uses the cjxl cli tool as it's backend and I would recommend pointing it to a nightly libjxl binary since there's been a bunch of improvements compared to the latest stable release: https://artifacts.lucaversari.it/libjxl/libjxl/latest/ |
|