|
lonjil
|
2024-09-30 09:32:36
|
they're both more washed out
|
|
|
damian101
|
|
lonjil
now explain this. from left to right: d1_wood.jxl, original downscaled with libplacebo and written with ffmpeg to a png file, source_wood.jxl
|
|
2024-09-30 09:32:37
|
does the one in the middle look right on your monitor, or the one on the left?
|
|
|
|
lonjil
|
2024-09-30 09:32:46
|
the middle one
|
|
|
|
damian101
|
2024-09-30 09:33:04
|
then your monitor most likely uses gamma 2.2
|
|
2024-09-30 09:33:17
|
less likely some other gamma
|
|
|
|
lonjil
|
2024-09-30 09:33:27
|
I wonder if there's a database of this kind of information
|
|
|
Demiurge
|
2024-09-30 09:36:27
|
Hmmm... If I use the lcms srgb profile I don't get the weirdness happening anymore
|
|
2024-09-30 09:36:33
|
after converting to lossy
|
|
2024-09-30 09:36:46
|
only when I use the icc profile
|
|
|
|
lonjil
|
2024-09-30 09:37:22
|
I put it into sRGB mode and now it looks like shit
|
|
|
|
Demiurge
|
2024-09-30 09:38:31
|
https://color.org/srgbprofiles.xalter
|
|
2024-09-30 09:39:05
|
This profile causes the weird color change after lossy conversion
|
|
2024-09-30 09:39:19
|
|
|
2024-09-30 09:39:57
|
Look what happens if I use lcms srgb profile instead!
|
|
2024-09-30 09:40:37
|
Now they both look identical
|
|
2024-09-30 09:40:53
|
and there's no color shift after cjxl -d 1
|
|
2024-09-30 09:41:12
|
(well, not identical, -d 1 is still noticeably blurry.)
|
|
2024-09-30 09:41:30
|
But the brightness looks the same at least. In waterfox.
|
|
|
|
lonjil
|
2024-09-30 09:42:04
|
Now it looks the same as my own downscale brightness-wise
|
|
|
|
Demiurge
|
2024-09-30 09:42:40
|
libplacebo looks cool. It's rare to find gamma-correct dither algorithms.
|
|
2024-09-30 09:43:11
|
Maybe if libjxl used libplacebo it would have less issues like this :)
|
|
|
|
lonjil
|
2024-09-30 09:43:50
|
Does it have any issues? So far I've only seen issues with that weird ICC profile :P
|
|
|
Demiurge
libplacebo looks cool. It's rare to find gamma-correct dither algorithms.
|
|
2024-09-30 09:44:12
|
I think it just converts to linear light
|
|
|
|
Demiurge
|
2024-09-30 09:44:18
|
What's so weird about it? I thought it was super official and whatever.
|
|
|
lonjil
I think it just converts to linear light
|
|
2024-09-30 09:45:33
|
no, dithering must be done as the final step, after it's already been converted into whatever gamma the output format is in.
|
|
2024-09-30 09:46:00
|
that's why it's so rare for it to be done correctly
|
|
|
|
lonjil
|
2024-09-30 09:46:24
|
<@853026420792360980> do you know what libplacebo's behavior is?
|
|
|
Traneptora
|
|
lonjil
<@853026420792360980> do you know what libplacebo's behavior is?
|
|
2024-09-30 09:57:39
|
for what, specifically?
|
|
|
|
lonjil
|
2024-09-30 09:58:17
|
Pashifox mentioned "gamma correct dither"
|
|
2024-09-30 10:00:43
|
But idk if libplacebo does anything special
|
|
2024-09-30 10:01:46
|
I mean in my case it was just downscaling an image with libplacebo in ffmpeg but idk if that's what Pashifox reacted to or if they found something elsewhere when googling
|
|
2024-09-30 10:02:06
|
<@1028567873007927297> what were you referring to specifically?
|
|
|
|
Demiurge
|
2024-09-30 10:19:26
|
oh
|
|
2024-09-30 10:19:30
|
I was referring to the readme
|
|
2024-09-30 10:19:36
|
first time I heard of libplacebo
|
|
2024-09-30 10:19:54
|
it says in the readme that it has gamma correct dithering algorithms which is pretty rare.
|
|
2024-09-30 10:22:28
|
most dithering algorithms just average values directly instead of trying to maintain the same brightness
|
|
|
|
Traneptora
|
|
Demiurge
most dithering algorithms just average values directly instead of trying to maintain the same brightness
|
|
2024-09-30 10:41:02
|
libplacebo does most image processing in linear light, including downscaling
|
|
|
|
lonjil
|
|
Demiurge
most dithering algorithms just average values directly instead of trying to maintain the same brightness
|
|
2024-09-30 10:52:10
|
wouldn't the difference be extremely tiny? At 8 bits, the sRGB curve looks essentially linear at the plus minus 1 scale
|
|
|
|
Demiurge
|
2024-09-30 10:59:17
|
No, the difference in apparent brightness is pretty noticeable from a difference, between a gamma correct dither and a naive dither. In fact it's often used as a monitor test.
|
|
2024-09-30 10:59:42
|
Anyways... Why is libjxl having such a hard time with the srgb profile from color.org?
|
|
2024-09-30 11:02:50
|
It behaves the same whether using vips jxlsave or using cjxl too.
|
|
2024-09-30 11:05:51
|
and if it happens with this color profile I guarantee this extremely noticeable color shift happens with lots of images that come with embedded profiles. As evidenced by the numerous open bug reports from people trying to convert images into lossy jxl from images that have embedded color profiles
|
|
2024-09-30 11:06:41
|
it's been a problem since 0.7
|
|
|
|
damian101
|
|
Demiurge
Anyways... Why is libjxl having such a hard time with the srgb profile from color.org?
|
|
2024-09-30 11:28:58
|
maybe that one does not specify a transfer curve
|
|
2024-09-30 11:29:12
|
Which is actually often desired behaviour
|
|
2024-09-30 11:30:50
|
conversion to lossy JXL of course has to assume some transfer, which seems to be sRGB
|
|
2024-09-30 11:32:21
|
However, I definitely think during decoding, the transfer curve tag should be set to unspecified, despite decoding happening to sRGB, to stay consistent with the source.
|
|
|
|
lonjil
|
|
maybe that one does not specify a transfer curve
|
|
2024-09-30 11:39:06
|
a 61kB sRGB ICC profile that doens't specify a transfer curve? What exactly would it be doing?
|
|
2024-09-30 11:42:48
|
if I decode the weird looking d1_wood.jxl file from <#803645746661425173>, the PNG it produces looks like this: ```
PNG signature found: d1_wood.jxl.png
Chunk: IHDR, Size: 25, Offset: 8, CRC32: 8f68727b
Chunk: gAMA, Size: 16, Offset: 33, CRC32: 31e8965f
Chunk: iCCP, Size: 345, Offset: 49, CRC32: 6d1c0317
Chunk: cHRM, Size: 44, Offset: 394, CRC32: 1b1c9a7f
Chunk: cICP, Size: 16, Offset: 438, CRC32: 9aa2f9d9
Chunk: IDAT, Size: 8204, Offset: 454, CRC32: f0f9d064
Chunk: 90 more IDAT chunks
Chunk: IEND, Size: 12, Offset: 743315, CRC32: ae426082
Size: 1500x937, Color: 8-bit RGB
gAMA: 100000
ICC Profile Length: 536
cHRM: wp: 31270, 32900, r: 64000, 33001, g: 30000, 60000, b: 15000, 6000
cICP: BT709, LinearLight, RGB, Full
```
so that's a lot of different color related chunks, and unless ICC profile length is in some odd unit, weird that the reported size doesn't match what jxlinfo reported about the file 🤔
|
|
2024-09-30 11:43:48
|
Why is it tagged as linear light, that doesn't make any sense
|
|
2024-09-30 11:44:04
|
<@1028567873007927297> so I can reproduce, can you tell me the exact process you used to create that file?
|
|
|
|
Demiurge
|
2024-09-30 11:53:13
|
`vipsthumbnail --vips-progress -a -i ./sRGB_v4_ICC_preference.icc -e ./sRGB_v4_ICC_preference.icc --intent perceptual -s 1500x1500> -o 'source_%s.jxl[lossless,effort=2,keep=icc]' ./wood.jpg`
|
|
2024-09-30 11:53:46
|
also works with `-i srgb`
|
|
2024-09-30 11:53:58
|
but the problem goes away when doing `-e srgb`
|
|
2024-10-01 12:31:03
|
If you just type `srgb` it uses some internal lcms profile data
|
|
|
Meow
|
2024-10-01 08:32:26
|
Is visually lossless JXL (about -d 0.5) a good idea for storing not-very-important and non-photographic images?
|
|
|
Quackdoc
|
2024-10-01 08:34:44
|
it depends on you
|
|
2024-10-01 08:35:09
|
are you planing to transcode the image a couple thousand times over and over? Probably not, transcoding it once or twice? probably fine
|
|
|
jonnyawsom3
|
2024-10-01 08:42:54
|
Generally for non-photo I find that by the time the distance is low enough, the lossless isn't much bigger instead
|
|
|
Kleis Auke
|
|
Demiurge
If you just type `srgb` it uses some internal lcms profile data
|
|
2024-10-01 09:03:17
|
libvips uses the the CC0-licensed `sRGB-v4.icc` profile from <https://github.com/saucecontrol/Compact-ICC-Profiles#srgbscrgb> for the magic `srgb` string in libvips 8.15 (see PR <https://github.com/libvips/libvips/pull/3400>).
|
|
|
|
Quackdoc
|
2024-10-01 09:04:26
|
I wonder how accurate the ICC is... I hate that this is a concern
|
|
|
|
Meow
|
|
Quackdoc
are you planing to transcode the image a couple thousand times over and over? Probably not, transcoding it once or twice? probably fine
|
|
2024-10-01 11:35:39
|
I have a quick action on macOS and it can do batch transcoding for hundreds or even thousands without any problem
|
|
|
|
lonjil
|
2024-10-01 11:36:28
|
ok this is weird.
Left: d1_wood.jxl, as decoded to png by djxl.
Middle: d1_wood.jxl.
Right: d1_wood.jxl, as decoded to png by jxl-oxide.
|
|
|
|
Meow
|
2024-10-01 11:37:16
|
That's only for PNG or JPEG at very high quality
|
|
|
|
lonjil
|
|
lonjil
if I decode the weird looking d1_wood.jxl file from <#803645746661425173>, the PNG it produces looks like this: ```
PNG signature found: d1_wood.jxl.png
Chunk: IHDR, Size: 25, Offset: 8, CRC32: 8f68727b
Chunk: gAMA, Size: 16, Offset: 33, CRC32: 31e8965f
Chunk: iCCP, Size: 345, Offset: 49, CRC32: 6d1c0317
Chunk: cHRM, Size: 44, Offset: 394, CRC32: 1b1c9a7f
Chunk: cICP, Size: 16, Offset: 438, CRC32: 9aa2f9d9
Chunk: IDAT, Size: 8204, Offset: 454, CRC32: f0f9d064
Chunk: 90 more IDAT chunks
Chunk: IEND, Size: 12, Offset: 743315, CRC32: ae426082
Size: 1500x937, Color: 8-bit RGB
gAMA: 100000
ICC Profile Length: 536
cHRM: wp: 31270, 32900, r: 64000, 33001, g: 30000, 60000, b: 15000, 6000
cICP: BT709, LinearLight, RGB, Full
```
so that's a lot of different color related chunks, and unless ICC profile length is in some odd unit, weird that the reported size doesn't match what jxlinfo reported about the file 🤔
|
|
2024-10-01 11:38:18
|
here is the umbrialpng output for the jxl-oxide png, compare to the umbrielpng output for the djxl png above ```
PNG signature found: d1_wood.jxl.jxl-oxide.png
Chunk: IHDR, Size: 25, Offset: 8, CRC32: 8f68727b
Chunk: iCCP, Size: 355, Offset: 33, CRC32: 693c72e7
Chunk: cICP, Size: 16, Offset: 388, CRC32: 9c693b32
Chunk: IDAT, Size: 2024005, Offset: 404, CRC32: 08289b7d
Chunk: IEND, Size: 12, Offset: 2024409, CRC32: ae426082
Size: 1500x937, Color: 8-bit RGB
ICC Profile Length: 572
ICC profile matches sRGB profile
cICP represents sRGB space
cICP: BT709, IEC61966-2-1, RGB, Full
```
|
|
|
|
veluca
|
2024-10-01 11:43:18
|
what's weird about the djxl output?
|
|
2024-10-01 11:43:25
|
seems like dithering
|
|
|
|
lonjil
|
2024-10-01 11:46:13
|
I think it's dithered in linear RGB and then converted to sRGB
|
|
2024-10-01 11:47:32
|
actually, GIMP just told me that the ICC profile that djxl output is linear RGB
|
|
2024-10-01 11:48:22
|
The original file, and the ICC profile <@1028567873007927297> used when saving it, which is stored inside the JXL file, are sRGB
|
|
2024-10-01 11:48:57
|
jxl-oxide outputs an sRGB PNG file, but djxl ouputs an 8-bit linear RGB PNG file
|
|
|
|
veluca
|
2024-10-01 11:52:12
|
weird
|
|
2024-10-01 11:52:22
|
I thought we got rid of that awful behaviour
|
|
|
|
lonjil
|
2024-10-01 11:53:08
|
yeah, I recall something like that happening and being fixed before 🤔
|
|
|
|
Meow
|
|
Generally for non-photo I find that by the time the distance is low enough, the lossless isn't much bigger instead
|
|
2024-10-01 12:27:46
|
-d 0.5 is much smaller than -d 0 if this is originally lossless
|
|
|
Quackdoc
are you planing to transcode the image a couple thousand times over and over? Probably not, transcoding it once or twice? probably fine
|
|
2024-10-01 02:10:05
|
Oh what you mentioned is generation loss. Then it would be only once generally
|
|
2024-10-01 02:11:47
|
I have hundreds to do
|
|
|
|
Quackdoc
|
2024-10-01 08:55:28
|
will jxl-rs have an officially supported c api?
|
|
|
|
veluca
|
2024-10-02 07:31:41
|
who knows 😛 probably at some point it will but we're not thinking about it yet
|
|
|
lonjil
jxl-oxide outputs an sRGB PNG file, but djxl ouputs an 8-bit linear RGB PNG file
|
|
2024-10-02 07:32:02
|
can you open an issue about it?
|
|
|
|
lonjil
|
|
|
jonnyawsom3
|
2024-10-02 01:51:19
|
Had an idea but might not be as good as it sounds. Checking EXIF data for an ISO tag and then automatically adding that level of photon noise during lossy encoding
JPEGs would just transcode anyway, so that away 90% of images
RAW files likely have a different noise pattern to what JXL would generate, making the image worse instead of better
You can't disable noise with djxl to get the 'original' back like you can with spot colors (Even though they haven't been used in years)
|
|
|
AccessViolation_
|
2024-10-02 01:56:32
|
I kind of assumed that'd be how it worked
|
|
2024-10-02 02:00:13
|
1. denoise image
2. compress image
3. noise synthesis when decoding
|
|
2024-10-02 02:01:56
|
I exported a high resolution raw to a developed PNG in Darktable and converted that PNG with `cjxl image.png image.jxl` but `--noise` and `--photon_noise_iso` didn't do anything in terms of file size. I guess it currently just adds noise instead of attempting to replace it
|
|
2024-10-02 02:07:06
|
Which is a shame because with noisy images like this, especially in low bitrate settings, removing photon noise before compressing it would allow you to use your limited bits on details that actually matter (though I have no idea how much this would save since my experiments didn't reveal much). I know AV1 also has noise synthesis so I tried to compare it by doing the same with AVIF to see if noise synthesis is used and what savings it would give, but it turns out `avifenc` is a pain to configure
|
|
|
Oleksii Matiash
|
|
Had an idea but might not be as good as it sounds. Checking EXIF data for an ISO tag and then automatically adding that level of photon noise during lossy encoding
JPEGs would just transcode anyway, so that away 90% of images
RAW files likely have a different noise pattern to what JXL would generate, making the image worse instead of better
You can't disable noise with djxl to get the 'original' back like you can with spot colors (Even though they haven't been used in years)
|
|
2024-10-02 02:07:07
|
But different cameras have very different noise with the same declared ISO sensitivity
|
|
|
|
AccessViolation_
|
2024-10-02 02:08:59
|
I should try denoising the raw image, then exporting, then adding noise back in with noise synthesis 🤔
|
|
|
CrushedAsian255
|
2024-10-02 02:35:58
|
We need to be careful with denoising as if it’s too strong it can cause its own artefacting
|
|
|
|
jonnyawsom3
|
|
Oleksii Matiash
But different cameras have very different noise with the same declared ISO sensitivity
|
|
2024-10-02 02:42:19
|
Yeah, hence my comment about making it worse instead of better
|
|
|
AccessViolation_
I exported a high resolution raw to a developed PNG in Darktable and converted that PNG with `cjxl image.png image.jxl` but `--noise` and `--photon_noise_iso` didn't do anything in terms of file size. I guess it currently just adds noise instead of attempting to replace it
|
|
2024-10-02 02:46:03
|
Photon noise is just a filter applied to the decoder, so very little filesize. The noise option is meant to learn the noise pattern of the image and replicate it on decode, but it's in a similar state as dots where the code *runs* but won't actually do much other than use a few gigs of RAM
|
|
|
|
Oleksii Matiash
|
|
Yeah, hence my comment about making it worse instead of better
|
|
2024-10-02 02:46:09
|
Old cameras often did nothing but setting tag to exif. I.e. data was the same for all or most sensitivity settings
|
|
|
|
CrushedAsian255
|
|
Photon noise is just a filter applied to the decoder, so very little filesize. The noise option is meant to learn the noise pattern of the image and replicate it on decode, but it's in a similar state as dots where the code *runs* but won't actually do much other than use a few gigs of RAM
|
|
2024-10-02 02:47:05
|
What are dots?
|
|
|
|
jonnyawsom3
|
2024-10-02 02:49:37
|
Originally intended to encode things like stars, sparks or CMYK spots, it would use elipses to create round and oblong objects (If I recall), it's almost entirely replaced by Patches now
|
|
|
|
CrushedAsian255
|
2024-10-02 02:50:18
|
Are splines currently being used for anything?
|
|
|
|
AccessViolation_
|
2024-10-02 02:50:50
|
if you're familiar with splines it's similar to that, but ellipses instead of lines
|
|
|
|
CrushedAsian255
|
2024-10-02 02:51:42
|
Do dots allow you to set gradient colours like splines?
|
|
2024-10-02 02:51:53
|
Or is it a static colour
|
|
|
|
AccessViolation_
|
2024-10-02 02:53:41
|
Idk about colors. I do remember reading you could define the drop off with a Gaussian function
|
|
|
|
CrushedAsian255
|
2024-10-02 02:55:49
|
Not sure how common perfect ovals would be in actual images
|
|
2024-10-02 02:55:58
|
So not sure how often they would be used
|
|
|
|
AccessViolation_
|
2024-10-02 02:59:38
|
iirc one of the mentioned use cases was stars. A small point light with a gradient intensity drop off
|
|
|
spider-mario
|
|
AccessViolation_
I should try denoising the raw image, then exporting, then adding noise back in with noise synthesis 🤔
|
|
2024-10-02 03:10:17
|
I did have an experimental tool that took original + denoised image and tried to infer the noise level from the difference
|
|
2024-10-02 03:10:49
|
it needs the original (or denoised) and not just the difference so that it can know which signal level each observed deviation corresponds to
|
|
2024-10-02 03:11:04
|
and it leaves the denoising up to the user
|
|
2024-10-02 03:12:36
|
also, there is a scale mismatch I never got around to debugging between photon_noise in libjxl and in libaom, and that tool gave results in libaom scale
|
|
|
_wb_
|
2024-10-02 05:35:30
|
The approach I would try is something like this:
- Denoise by applying EPF at some amplitude proportional to the distance target
- Estimate noise
- Encode denoised image and signal noise (or if noise is below some threshold, just encode original and don't signal noise)
|
|
|
|
AccessViolation_
|
2024-10-02 05:47:55
|
How selectively can you apply noise? I remember it only being possible to add it based on brightness using some formula, or I may be confusing it with the `photon_noise_iso` thing. Or is there a layer that can be used sort of as a gain map for the noise?
|
|
|
|
lonjil
|
2024-10-02 05:53:37
|
While evaluating the original full resolution version of the image I posted in <#803645746661425173>, I thought libjxl did quite a good job denoising bright parts of the image at d=1 e=3 without hurting any real details. What sort of stuff does libjxl do with those settings?
|
|
|
|
_wb_
|
|
AccessViolation_
How selectively can you apply noise? I remember it only being possible to add it based on brightness using some formula, or I may be confusing it with the `photon_noise_iso` thing. Or is there a layer that can be used sort of as a gain map for the noise?
|
|
2024-10-02 05:59:10
|
Noise is applied throughout the frame, only modulated by brightness (it can be signaled how it is modulated). If you really want to locally control noise, an encoder has to do something like this:
- encode a frame without noise
- patch-copy it to a second frame, with noise
- put an alpha channel on the second frame so both frames get blended however you want
It wouldn't be particularly efficient to decode, but it would work.
|
|
|
lonjil
While evaluating the original full resolution version of the image I posted in <#803645746661425173>, I thought libjxl did quite a good job denoising bright parts of the image at d=1 e=3 without hurting any real details. What sort of stuff does libjxl do with those settings?
|
|
2024-10-02 06:02:46
|
I think it does a low-amplitude EPF (decoder only), and of course some high freq coefficients get quantized away which is also a form of denoising I guess 🙂
|
|
|
|
lonjil
|
|
|
spider-mario
|
2024-10-02 08:11:12
|
why is Adobe DNG Converter on the command line not creating any DNG file
|
|
2024-10-02 08:11:28
|
it seemingly exits successfully, but there is no new file
|
|
|
|
_wb_
|
2024-10-02 08:20:48
|
Why can't they make it open source so the foss world can remain up to date more easily?
|
|
2024-10-02 08:23:18
|
Currently I don't think darktable, rawtherapee etc can open iPhone DNGs. What's the point of DNG if there's no interoperability anyway?
|
|
|
|
lonjil
|
2024-10-02 08:25:14
|
well it's interoperable between Adobe products and cameras that produce DNGs !
|
|
|
|
spider-mario
|
2024-10-02 08:41:51
|
to be fair, the format specification is open, and some tools (PTGui, RawDigger) have implemented support for jxl dng
|
|
2024-10-02 08:41:59
|
but yeah, more open-source tooling would be nice
|
|
|
RaveSteel
|
|
_wb_
Currently I don't think darktable, rawtherapee etc can open iPhone DNGs. What's the point of DNG if there's no interoperability anyway?
|
|
2024-10-02 08:49:37
|
One of the main problems is that Adobe's DNG SDK, which libraw uses, is not published under a copyleft license. So projects like rawtherapee are hesitating to use it, sadly
|
|
|
|
_wb_
|
2024-10-02 08:49:44
|
I wonder why darktable is not even trying to support ProRAW. Are those not just standard DNG files?
|
|
|
|
RaveSteel
|
2024-10-02 08:54:17
|
Huh, darktable indeed does not support standard ProRAW
|
|
2024-10-02 08:54:25
|
Never noticed because I use Rawtherapee mostly
|
|
|
|
spider-mario
|
2024-10-02 08:56:30
|
as far as I’ve read, ProRAW is standard linear DNG 1.6
|
|
|
|
RaveSteel
|
2024-10-02 08:57:18
|
ProRAW seems to be using an unsupported compression scheme
|
|
2024-10-02 08:57:22
|
https://github.com/darktable-org/darktable/issues/7578
|
|
|
|
lonjil
|
2024-10-02 08:58:50
|
AFAIK ProRAW only ever used JPEG compression
|
|
|
|
RaveSteel
|
2024-10-02 09:00:09
|
If Apple published techincal documents we could have a look, but they don't
|
|
2024-10-02 09:00:13
|
Or at least I've found none
|
|
2024-10-02 09:17:22
|
Not a technical specification but a nice breakdown of ProRAW by the author of Halide
https://archive.is/adjeH
|
|
|
qdwang
|
|
lonjil
AFAIK ProRAW only ever used JPEG compression
|
|
2024-10-02 09:37:13
|
Yes, my code can decode ProRAW and it’s only JPEG 92 lossless and nothing special
|
|
|
|
RaveSteel
|
2024-10-02 09:38:25
|
There is a pull request in darktable's raw decoder library opened 2020, but it's not making much progress
|
|
|
|
qdwang
|
2024-10-02 09:52:47
|
In the last reply of the thread, someone mentioned dnglab, it should also decode ProRAW. But sadly it’s completely rust(just like my code), so the decode part cannot be ported to rawspeed easily.
|
|
|
Dejay
|
2024-10-02 09:55:54
|
Is it (theoreticallyy) possible to lossless transcode jpeg2000 to JpegXL?
|
|
|
|
lonjil
|
2024-10-02 10:04:09
|
some jpeg2000 files, not all, and only using JXL's regular lossless mode
|
|
|
|
Dejay
|
2024-10-02 10:14:07
|
I was wondering about lossy encoded jp2 files, for lossless transcode like with jpg so it becomes a smaller size
|
|
|
|
Demiurge
|
2024-10-03 12:44:37
|
Well jxl has a wavelet transform
|
|
2024-10-03 12:45:48
|
idk if they are similar enough that the transform coefficients could be re-used
|
|
2024-10-03 12:46:25
|
without converting it back into pixels
|
|
|
|
Dejay
|
2024-10-03 01:53:23
|
Ahh so in case of jpg it just copies the DCT and compresses those coefficients better. And theoretically one could copy those wavelet DWT too
|
|
|
|
CrushedAsian255
|
2024-10-03 02:35:47
|
Do jpeg xl blocks have to line up to some form of “supergrid” like how AVIF superblocks
|
|
2024-10-03 02:35:52
|
Or can they just go anywhere
|
|
|
|
Demiurge
|
|
Dejay
Ahh so in case of jpg it just copies the DCT and compresses those coefficients better. And theoretically one could copy those wavelet DWT too
|
|
2024-10-03 02:49:30
|
I don't know if theoretically they are similar enough to be transformed with "minimal loss."
|
|
2024-10-03 02:49:57
|
But old JPEG is for sure
|
|
|
|
jonnyawsom3
|
|
spider-mario
why is Adobe DNG Converter on the command line not creating any DNG file
|
|
2024-10-03 06:00:04
|
Ah, so you had to endure the torture too. A shame I've seen it so late (Or early now I guess...)
The parameters are case sensitive, order dependant and with no help function and no warnings or errors on failure. So essentially try every combination until you get a file out
|
|
|
RaveSteel
https://github.com/darktable-org/darktable/issues/7578
|
|
2024-10-03 06:04:34
|
https://github.com/Beep6581/RawTherapee/pull/6214
|
|
|
|
_wb_
|
2024-10-03 06:39:19
|
Modular Squeeze is not really a wavelet in the same way as JPEG 2000's DWT. It can be described as a modified, non-linear Haar wavelet, but Haar is kind of the simplest possible wavelet, compared to the DWTs that J2K is doing.
|
|
2024-10-03 06:40:40
|
So no, lossy j2k cannot be losslessly converted to jxl, except by first decoding to pixels (but that's not _really_ lossless and it's also not going to be effective)
|
|
|
|
CrushedAsian255
|
|
_wb_
Modular Squeeze is not really a wavelet in the same way as JPEG 2000's DWT. It can be described as a modified, non-linear Haar wavelet, but Haar is kind of the simplest possible wavelet, compared to the DWTs that J2K is doing.
|
|
2024-10-03 06:41:30
|
isn't squeeze basically just mid/side between adjacent pixels?
|
|
|
|
_wb_
|
2024-10-03 06:43:48
|
not completely, it's not just converting `A, B` to `(A+B)/2, A-B`, since if you would then quantize the residuals, it would just be a blocky, bandy mess.
|
|
2024-10-03 06:50:32
|
the `(A+B)/2` part is there just like that (which gives box-filter progressive previews), but instead of `A-B`, it stores `A-B-T` where `T` is a nonlinear "tendency" term based on neighboring samples: basically if the local neighborhood is smooth (monotonic), `T` corresponds to linear interpolation, while if there is a local nonmonotonicity, `T` is just zero, to avoid the ringing you would otherwise get around hard edges when quantizing these residuals.
|
|
2024-10-03 06:51:47
|
there are some comments in https://github.com/libjxl/libjxl/blob/main/lib/jxl/modular/transform/squeeze.h to explain how it works
|
|
|
|
CrushedAsian255
|
2024-10-03 06:52:54
|
```theoretically range could be 2.5
// times larger (2 times without the
// 'tendency'), but there should be lots
// of zeroes```
does this mean squeeze residual channels get more bits
|
|
|
|
_wb_
|
2024-10-03 06:58:31
|
yes, see https://github.com/libjxl/libjxl/blob/main/lib/jxl/enc_modular.cc#L895
|
|
|
|
Demiurge
|
|
_wb_
So no, lossy j2k cannot be losslessly converted to jxl, except by first decoding to pixels (but that's not _really_ lossless and it's also not going to be effective)
|
|
2024-10-03 07:48:07
|
They aren't similar enough to theoretically convert or reuse some of the resulting data?
|
|
|
|
_wb_
|
2024-10-03 09:32:11
|
J2K uses these: https://en.m.wikipedia.org/wiki/Cohen%E2%80%93Daubechies%E2%80%93Feauveau_wavelet
|
|
2024-10-03 09:33:49
|
CDF 9/7 is used for lossy J2K. It's pretty different from Squeeze.
|
|
|
|
CrushedAsian255
|
2024-10-03 11:19:49
|
Could you keep the wavelet data and then use jxl as entropy coding instead of whatever it’s currently using
|
|
|
|
lonjil
|
2024-10-03 11:42:14
|
no
|
|
2024-10-03 11:42:37
|
because jxl doesn't have those wavelets
|
|
2024-10-03 11:43:48
|
you could extrace the wavelet data, encode it as an image (looking completely nonsensical) and then compress that with jxl, but it wouldn't work well and obviously nothing would display the original image from the jxl
|
|
|
|
_wb_
|
2024-10-03 12:17:35
|
I guess you could make a new format that is basically J2K with different entropy coding, but I don't think it would be super useful compared to JPEG recompression. JPEG is a lot more ubiquitous than JPEG 2000....
|
|
|
|
CrushedAsian255
|
2024-10-03 01:16:12
|
Also at that point you could do the same for anything
|
|
|
|
Dejay
|
|
_wb_
I guess you could make a new format that is basically J2K with different entropy coding, but I don't think it would be super useful compared to JPEG recompression. JPEG is a lot more ubiquitous than JPEG 2000....
|
|
2024-10-03 04:28:55
|
Well PDF still use jpeg2000, and djvu use wavelet. But you couldn't really convert them "lossless" anyway
|
|
2024-10-03 04:29:11
|
And it's a rather silly idea 😄
|
|
|
|
lonjil
|
2024-10-03 04:30:17
|
I wonder when JXL will be added to the PDF standard
|
|
|
|
Dejay
|
2024-10-03 04:32:03
|
I think there are so many pdf tools and libraries that adding that would be extremely problematic.
|
|
|
username
|
2024-10-03 04:33:13
|
the PDF standard has been planning on adding backwards incompatible stuff for quite a while now iirc
|
|
|
|
lonjil
|
2024-10-03 04:37:23
|
I mean, presumably every update to PDF added stuff that older viewers couldn't handle?
|
|
|
|
Dejay
|
2024-10-03 04:40:41
|
It would be easier to add a sort of standard for transparently hooking in specialized compressors into file systems and file transmission. PackJPG does this, recompresses jpg with better compression. Precomp-cpp uses that and can do the same for pdf and zip file.
|
|
|
|
_wb_
|
2024-10-03 04:56:26
|
Someone from PDF/R told me they're planning to add jxl as a payload option there.
|
|
|
yoochan
|
2024-10-03 05:04:30
|
What does /R means?
|
|
|
TheBigBadBoy - 𝙸𝚛
|
|
yoochan
What does /R means?
|
|
2024-10-03 05:08:09
|
<https://www.pdf2go.com/dictionary/pdf_r>
PDF Raster
|
|
|
|
Dejay
|
2024-10-03 05:19:05
|
It seems you could also lossless convert PDF/R to jxl
|
|
|
Foxtrot
|
2024-10-03 07:03:45
|
Yes, do it. Then upload ton of PDFs with JXL on web and when Chrome cant display them be like "I guess Chrome doesnt support PDF"
|
|
|
|
lonjil
|
2024-10-03 07:05:27
|
PDF already supports a bunch of image formats that no browser supports, so presumably they do actually ship such codecs, just only inside the PDF implementation.
|
|
|
|
Foxtrot
|
2024-10-03 07:06:25
|
oh no, my plan foiled again
|
|
|
|
spider-mario
|
2024-10-03 08:50:16
|
maybe one will be able to embed a frameless PDF
|
|
2024-10-03 08:50:48
|
I believe the term is usually “chromeless”, but I fear it might interfere with the message
|
|
|
|
lonjil
|
2024-10-03 08:54:57
|
I remember some technical marketing video from Mozilla about chromeless browsers or something and it got a lot of hate
|
|
2024-10-03 09:04:33
|
And alas, anything not supported by the browser might be decoded for PDF via Wasm, so much slower.
|
|
|
|
jonnyawsom3
|
2024-10-03 10:33:10
|
Noticed this user on Reddit mention filing an issue at Apple internally about broken HDR in the Photos app, so I opened the profile and saw this https://www.reddit.com/r/jpegxl/comments/1fg4iul/comment/ln02n9o/
|
|
2024-10-03 10:33:35
|
Not entirely sure what it means... An embedded preview, transcoding? Hmmm...
|
|
|
|
username
|
|
Not entirely sure what it means... An embedded preview, transcoding? Hmmm...
|
|
2024-10-03 10:36:28
|
probably related to this https://discord.com/channels/794206087879852103/822105409312653333/1290977610599497729
|
|
2024-10-03 10:38:51
|
I hate how the comments section on Reddit works, I didn't even see what the comment you linked to was a reply to until just now when I opened the full post
|
|
2024-10-03 10:39:35
|
either way what I linked to is probably still relevant
|
|
|
|
jonnyawsom3
|
2024-10-03 10:41:10
|
This is just a normal JXL file, so unless they're explicitly storing a thumbnail in EXIF or something then I don't know what they're doing
|
|
|
|
username
|
2024-10-03 10:43:17
|
I'm a bit tired but uh what is a normal JXL file? because if you mean ProRaw then it's not
|
|
|
|
jonnyawsom3
|
2024-10-03 10:44:15
|
A .jxl, not a .DNG or whatever format they label it as. Exported as a JXL from software
|
|
|
|
username
|
2024-10-03 10:46:59
|
their comment is a bit confusing but they might just be talking about ProRaw despite the comment they are replying to
|
|
|
|
Demiurge
|
|
_wb_
I guess you could make a new format that is basically J2K with different entropy coding, but I don't think it would be super useful compared to JPEG recompression. JPEG is a lot more ubiquitous than JPEG 2000....
|
|
2024-10-03 11:20:48
|
that sounds like htj2k
|
|
|
|
lonjil
|
2024-10-03 11:22:14
|
Isn't HTJ2K just a subset of J2K?
|
|
|
|
Demiurge
|
2024-10-04 04:43:27
|
I think it's j2k with a faster entropy coder?
|
|
|
|
_wb_
|
2024-10-04 05:07:43
|
Yes, it's J2K with everything the same except the entropy coder.
|
|
|
|
CrushedAsian255
|
|
_wb_
Yes, it's J2K with everything the same except the entropy coder.
|
|
2024-10-04 05:16:16
|
is it like prefix code instead of ANS or something?
|
|
|
|
_wb_
|
2024-10-04 05:34:39
|
I don't know the details but I think it boils down to using something that parallelizes/SIMDifies while the old J2K had inherently sequential and branchy entropy coding.
|
|
|
|
jonnyawsom3
|
2024-10-04 05:40:17
|
> HTJ2K achieves these improvements by introducing a new HT block coder, which is a drop-in replacement for the
> original JPEG 2000 Part 1 block coder (J2K-1) and allows truly reversible transcoding to/from J2K-1.
|
|
|
|
CrushedAsian255
|
|
_wb_
I don't know the details but I think it boils down to using something that parallelizes/SIMDifies while the old J2K had inherently sequential and branchy entropy coding.
|
|
2024-10-04 06:06:36
|
just wondering, is jpeg xl modular SIMDable? The pixel-by-pixel nature feels very sequential to me
|
|
|
|
_wb_
|
2024-10-04 06:55:36
|
No it isn't, at least not in the general case. Special cases can be done without branching though, and besides the entropy decoding, the other stuff like RCTs etc SIMDifies well.
|
|
2024-10-04 06:56:19
|
E.g. e1-3 encoded images have a fast decode path that avoids branching
|
|
2024-10-04 06:56:48
|
Also we have groups so it does parallelize easily
|
|
2024-10-04 06:59:11
|
But individual streams of entropy coding are kind of inherently sequential, that's kind of inevitable. With hindsight maybe we should have used multiplexed ANS streams to allow SIMD entropy decode, but that gets quite complicated...
|
|
|
|
CrushedAsian255
|
2024-10-04 07:13:18
|
Don’t groups sometimes reference each other? Or are they completely unique and separate
|
|
|
|
_wb_
|
2024-10-04 07:18:29
|
They can be decoded independently, as far as the entropy decode is concerned. There are filters like Gabor ish/EPF that do cross group boundaries, but those can be simdified.
|
|
|
|
CrushedAsian255
|
2024-10-04 08:13:35
|
so is decode
```
decodedGroups = parallel.foreach(groups,group=>decodeGroup(group));
image = stitch(decodedGroups);
return applyFilters(image);
```
|
|
|
|
jonnyawsom3
|
2024-10-04 09:30:57
|
If using `-E` then channels have to wait for the ones before, I think, and layers with blending have to wait for the layer beneath, but each channel itself can use a thread per group
|
|
|
nocpu
|
2024-10-04 02:46:31
|
https://discord.com/channels/794206087879852103/806898911091753051/1283154358950821908
|
|
|
|
jonnyawsom3
|
2024-10-04 03:26:51
|
Yeah, what about it?
|
|
|
|
Dejay
|
2024-10-05 12:40:08
|
Does JXL still support this? (or plans to support this)
> 4.1.2 Screen readers/clicking on image text
> Examples of text in images include event flyers, company brochures and documents in general. For these, it would be desirable to store the actual text into optional metadata, such that users can interact with it (click on links, copy text).
|
|
|
|
CrushedAsian255
|
2024-10-05 03:30:02
|
does libjxl currently support varblocks smaller than 8x8?
|
|
|
Dejay
Does JXL still support this? (or plans to support this)
> 4.1.2 Screen readers/clicking on image text
> Examples of text in images include event flyers, company brochures and documents in general. For these, it would be desirable to store the actual text into optional metadata, such that users can interact with it (click on links, copy text).
|
|
2024-10-05 03:45:20
|
maybe an extension box could be added to support that
|
|
2024-10-05 03:45:34
|
not sure if the base spec has anything along that like
|
|
|
|
_wb_
|
|
CrushedAsian255
does libjxl currently support varblocks smaller than 8x8?
|
|
2024-10-05 05:59:11
|
Yes
|
|
|
|
CrushedAsian255
|
|
_wb_
Yes
|
|
2024-10-05 06:00:14
|
I couldn't find any, maybe jxlatte doesn't display them properly
<@853026420792360980>
|
|
|
Tirr
|
2024-10-05 06:03:26
|
jxl can do 2x2, 4x4 and 4x8 DCTs but they are still within 8x8 varblock boundary
|
|
|
|
CrushedAsian255
|
|
Tirr
jxl can do 2x2, 4x4 and 4x8 DCTs but they are still within 8x8 varblock boundary
|
|
2024-10-05 06:03:42
|
so like 8x8 subdivide?
|
|
2024-10-05 06:04:05
|
do larger varblocks have to stay within certain boundaries?
|
|
|
|
Tirr
|
2024-10-05 06:04:11
|
like, 2x2 DCTs are always grouped in 16 DCT blocks and form 8x8 varblock
|
|
|
|
CrushedAsian255
|
2024-10-05 06:04:13
|
im guessing they can't cross group boundaries
|
|
|
|
Tirr
|
2024-10-05 06:04:51
|
likewise four 4x4 DCTs and two 4x8 DCTs
|
|
|
|
CrushedAsian255
|
2024-10-05 06:05:50
|
that make sense
|
|
|
|
Tirr
|
2024-10-05 06:06:00
|
you can see Section I.1 of the spec
|
|
|
|
CrushedAsian255
|
2024-10-05 06:06:07
|
do the 8x8 and higher have to stick within a certain grid?
|
|
|
|
Tirr
|
2024-10-05 06:06:19
|
and I.9
|
|
2024-10-05 06:06:50
|
varblocks can't cross 256x256 group boundaries
|
|
|
|
CrushedAsian255
|
2024-10-05 06:08:26
|
is something like this allowed?
|
|
2024-10-05 06:08:47
|
|
|
|
|
embed
|
|
CrushedAsian255
|
|
2024-10-05 06:08:50
|
https://embed.moe/https://cdn.discordapp.com/attachments/794206170445119489/1292005548232802337/Screenshot_2024-10-05_at_16.08.23.jxl?ex=6702296f&is=6700d7ef&hm=59ad61d8cc1e2f7e611014e7ca3d44449574fbf11dffaaf354de29cfc8f4a9ec&
|
|
|
|
CrushedAsian255
|
|
Tirr
varblocks can't cross 256x256 group boundaries
|
|
2024-10-05 06:09:49
|
fair, that would be kinda silly and defeat the point of the groups
|
|
|
|
Tirr
|
|
https://embed.moe/https://cdn.discordapp.com/attachments/794206170445119489/1292005548232802337/Screenshot_2024-10-05_at_16.08.23.jxl?ex=6702296f&is=6700d7ef&hm=59ad61d8cc1e2f7e611014e7ca3d44449574fbf11dffaaf354de29cfc8f4a9ec&
|
|
2024-10-05 06:10:42
|
yeah this seems valid
|
|
|
|
Dejay
|
2024-10-05 06:11:32
|
Is there a spec available somewhere free? Or only iso?
|
|
|
|
Tirr
|
2024-10-05 06:11:56
|
there's community draft of the spec in <#1021189485960114198>
|
|
|
|
Dejay
|
|
Tirr
there's community draft of the spec in <#1021189485960114198>
|
|
2024-10-05 06:22:42
|
Thanks, I think I found it
|
|
|
|
jonnyawsom3
|
|
Tirr
jxl can do 2x2, 4x4 and 4x8 DCTs but they are still within 8x8 varblock boundary
|
|
2024-10-05 08:08:51
|
Ahh, so that's why VarDCT visualisation shows so many colors for the 8x8 block. It's different mixes of the smaller ones
|
|
|
|
_wb_
|
2024-10-05 08:09:55
|
Yes, and also the corner-cut variants of 8x8
|
|
|
|
Tirr
|
2024-10-05 08:14:03
|
and "Hornuss" transform which does... something. I know this is some kind of 4x4 transform but don't understand exactly what it is
|
|
|
|
_wb_
|
2024-10-05 08:19:06
|
It's meant for cases where DCT doesn't work well, originally we called it "identity" but it's not really just the identity transform
|
|
|
|
Tirr
|
2024-10-05 08:20:35
|
is it something like "almost identity but hacked to have average of the block at topleft corner"?
|
|
|
|
Traneptora
|
|
CrushedAsian255
I couldn't find any, maybe jxlatte doesn't display them properly
<@853026420792360980>
|
|
2024-10-05 08:21:31
|
Varblocks can't be smaller than 8x8
|
|
|
|
CrushedAsian255
|
|
Tirr
is it something like "almost identity but hacked to have average of the block at topleft corner"?
|
|
2024-10-05 08:22:16
|
maybe it's average in top left and everything else is differecne from average
|
|
|
|
Traneptora
|
2024-10-05 08:22:46
|
The smaller DCT sizes still use 8x8 varblocks
|
|
|
|
veluca
|
|
Tirr
is it something like "almost identity but hacked to have average of the block at topleft corner"?
|
|
2024-10-05 08:22:49
|
pretty much 😛
|
|
|
|
CrushedAsian255
|
|
veluca
pretty much 😛
|
|
2024-10-05 08:22:58
|
is it defined in the spec?
|
|
|
|
Traneptora
|
2024-10-05 08:23:35
|
spec is descriptive, not information
|
|
|
|
CrushedAsian255
|
|
Traneptora
spec is descriptive, not information
|
|
2024-10-05 08:23:47
|
?
|
|
|
|
veluca
|
2024-10-05 08:23:48
|
I mean, the spec defines a formula, it doesn't say how I came up with it or why -- but the idea was indeed to find a way to put the average in the topleft corner without things being horrible numerically
|
|
|
|
Traneptora
|
2024-10-05 08:24:01
|
it defines hornuss transform but doesn't provide any logic behind background or motivation
|
|
|
|
CrushedAsian255
|
2024-10-05 08:24:21
|
is what you mean as : the spec defines the what, but not the why or how?
|
|
|
|
Traneptora
|
|
|
Tirr
|
2024-10-05 08:24:30
|
transforms are defined as just code in the spec
|
|
|
|
CrushedAsian255
|
2024-10-05 08:24:42
|
is there a good place to learn about the why?
|
|
|
|
Traneptora
|
2024-10-05 08:24:55
|
"ask here"
|
|
|
|
veluca
|
2024-10-05 08:24:57
|
asking here is your best bet I'm afraid
|
|
|
|
Traneptora
|
2024-10-05 08:25:27
|
there hasn't been a full writeup about the how & why of JXL by any core devs
|
|
|
|
CrushedAsian255
|
2024-10-05 08:25:36
|
do we know if this is going anywhere?
https://discord.com/channels/794206087879852103/1254012901816274956
|
|
|
|
Tirr
|
2024-10-05 08:25:58
|
well it basically depends on me 😅
|
|
|
|
CrushedAsian255
|
2024-10-05 08:26:02
|
maybe someone should create a google for a technical writeup
|
|
|
Tirr
well it basically depends on me 😅
|
|
2024-10-05 08:26:23
|
it's open source so other people on the server can pr
|
|
|
|
Traneptora
|
2024-10-05 08:27:06
|
Ive thought about making an "unpacking JXL" blog post targeted at people roughly familiar with JPEG
|
|
|
|
veluca
|
2024-10-05 08:27:35
|
happy to review such a blogpost if it gets written 🙂
|
|
|
|
Traneptora
|
2024-10-05 08:27:35
|
but time
|
|
|
|
CrushedAsian255
|
|
Traneptora
but time
|
|
2024-10-05 08:27:45
|
yeah
|
|
|
|
veluca
|
|
Traneptora
but time
|
|
2024-10-05 08:27:45
|
yeah, that...
|
|
2024-10-05 08:28:04
|
it's also why we didn't do something similar yet
|
|
2024-10-05 08:28:21
|
time is probably better spent with the Rust reimplementation, cropped decode API, or whatever else
|
|
|
|
CrushedAsian255
|
|
veluca
time is probably better spent with the Rust reimplementation, cropped decode API, or whatever else
|
|
2024-10-05 08:28:40
|
ig if people are curious they can just come here
|
|
|
|
jonnyawsom3
|
|
_wb_
Yes, and also the corner-cut variants of 8x8
|
|
2024-10-05 08:29:00
|
Not sure I've heard of that before
|
|
|
|
CrushedAsian255
|
2024-10-05 08:29:20
|
i remember hearing about them but not sure what they are specifically
|
|
|
|
veluca
|
2024-10-05 08:29:41
|
I believe we ended up calling it AFV transform
|
|
|
|
Traneptora
|
2024-10-05 08:29:53
|
That is them, yes
|
|
|
|
veluca
|
2024-10-05 08:30:00
|
it's actually a 2d DCT-like transform
|
|
|
|
Traneptora
|
2024-10-05 08:30:23
|
they are a 4x8 DCT, a 4x4 DCT, and a 4x4 Hornuss kinda wrapped up
|
|
2024-10-05 08:30:29
|
iiuc
|
|
|
|
veluca
|
2024-10-05 08:30:49
|
the 4x4 that's not a DCT is not a Hornuss
|
|
|
|
Traneptora
|
2024-10-05 08:30:56
|
ah nvm
|
|
|
|
veluca
|
2024-10-05 08:31:24
|
I could describe it as one of the eigenvectors of the 4x4 grid graph with the top-left 3 pixels being split from the rest of the grid, but I am not sure that description is going to be particularly useful 😛
|
|
|
|
CrushedAsian255
|
2024-10-05 08:31:26
|
for the not 8x8 / non-dct blocks what is stored in the LF image?
|
|
|
|
Traneptora
|
2024-10-05 08:31:45
|
Always the average
|
|
|
|
Tirr
|
2024-10-05 08:31:49
|
average of the 8x8 block, yes
|
|
|
|
Traneptora
|
2024-10-05 08:31:50
|
of the 8x8
|
|
|
|
Tirr
|
2024-10-05 08:32:07
|
so you can use LF image as 8x downsampled image
|
|
|
|
veluca
|
|
veluca
I could describe it as one of the eigenvectors of the 4x4 grid graph with the top-left 3 pixels being split from the rest of the grid, but I am not sure that description is going to be particularly useful 😛
|
|
2024-10-05 08:32:27
|
in a less technical way, you can think of it as DCTing separately 3 pixels in the corner and the rest of the 4x4 block
|
|
|
|
CrushedAsian255
|
2024-10-05 08:32:31
|
for the > 8x8 blocks i thought i remember reading it was the smaller coefficients
|
|
|
|
veluca
|
2024-10-05 08:32:39
|
well, it's not *really* the average for >8x8
|
|
2024-10-05 08:32:49
|
I mean, it's close to it, but a bit more complicated
|
|
2024-10-05 08:33:24
|
it's the IDCT of the (N/8 x N/8) lowest-frequency coefficients
|
|
|
|
Tirr
|
2024-10-05 08:33:40
|
the point is that LF image is always 8x downsampled one
|
|
|
|
Traneptora
|
2024-10-05 08:33:46
|
The LF coefficients are always the average of the corresponding 8x8 block. for larger varblock sizes, eg 32x32, you end up with a 4x4 grid of lf coefficients. you have to forward DCT these
|
|
|
|
jonnyawsom3
|
2024-10-05 08:33:55
|
Now I'm wondering how hard it'd be to show all these in the visualisation too. Maybe with a few extra lines on the shortest side saying what each color and shape means
|
|
|
|
veluca
|
2024-10-05 08:34:00
|
which is *some* 8x downsampled representation of the NxN block, but not just per-block average
|
|
|
|
Traneptora
|
2024-10-05 08:34:08
|
ah, hm
|
|
|
Now I'm wondering how hard it'd be to show all these in the visualisation too. Maybe with a few extra lines on the shortest side saying what each color and shape means
|
|
2024-10-05 08:34:42
|
show what, specifically?
|
|
|
|
CrushedAsian255
|
|
veluca
it's the IDCT of the (N/8 x N/8) lowest-frequency coefficients
|
|
2024-10-05 08:34:44
|
are the values actually used in the decoding for >8x8?
|
|
|
|
Traneptora
|
|
|
veluca
|
2024-10-05 08:34:50
|
yup
|
|
2024-10-05 08:35:36
|
making sure that the 8x8 downsampled representation would be consistently available even with larger transforms was quite an effort, from what I remember of it 😄
|
|
|
|
Traneptora
|
2024-10-05 08:35:42
|
you take the LF coefficients and apply forward DCT, lay them in the upper left corner of the HF coefficients, and then apply IDCT to get pixels
|
|
|
|
jonnyawsom3
|
|
Traneptora
show what, specifically?
|
|
2024-10-05 08:37:24
|
The sub-8x8 DCT blocks, Hornuss and AFV. Ideally with a glossary to know what colors and shapes mean what.
Essentially an extended version of the current VarDCT visualisations
|
|
|
|
Traneptora
|
2024-10-05 08:38:14
|
That could be done, but a glossary is harder
|
|
2024-10-05 08:38:21
|
where do you put it?
|
|
2024-10-05 08:38:39
|
like ux design question
|
|
2024-10-05 08:39:04
|
actually rendering those sub boundaries is doable
|
|
2024-10-05 08:39:40
|
the issue is some ger really fine, like dct2
|
|
|
|
CrushedAsian255
|
|
Traneptora
you take the LF coefficients and apply forward DCT, lay them in the upper left corner of the HF coefficients, and then apply IDCT to get pixels
|
|
2024-10-05 08:39:49
|
like this?
|
|
2024-10-05 08:39:56
|
|
|
|
|
Traneptora
|
2024-10-05 08:40:31
|
yes
|
|
2024-10-05 08:40:58
|
essentially
|
|
2024-10-05 08:41:25
|
there's a few extra steps but that's the fundamental mathematical concept
|
|
|
|
CrushedAsian255
|
2024-10-05 08:41:30
|
so then decode is
|
|
|
|
Traneptora
|
2024-10-05 08:41:45
|
basically yes
|
|
|
|
CrushedAsian255
|
|
Traneptora
there's a few extra steps but that's the fundamental mathematical concept
|
|
2024-10-05 08:41:58
|
> few extra steps
things like quantisation and normalisation?
|
|
2024-10-05 08:42:06
|
and entropy coding
|
|
|
|
jonnyawsom3
|
|
Traneptora
where do you put it?
|
|
2024-10-05 08:42:24
|
I was thinking of adding it to the shortest side of the image. So on very high aspect ratios it doesn't increase memory usage twofold, but then it's very annoying to make the layout account for multiple columns or rows of the glossary on small images.
If you can see what I mean
|
|
|
|
CrushedAsian255
|
2024-10-05 08:43:12
|
for the AFV, you could add a dot to the top/bottom left/right
|
|
2024-10-05 08:43:18
|
for 4x4 drawing the lines should work
|
|
2024-10-05 08:43:36
|
so then it's only really 2x2 and Horuness that need special markings
|
|
|
|
Traneptora
|
|
CrushedAsian255
> few extra steps
things like quantisation and normalisation?
|
|
2024-10-05 08:44:17
|
extra steps basically like scaling
|
|
|
|
CrushedAsian255
|
2024-10-05 08:44:33
|
so just linear operations to make the numbers play nice?
|
|
|
|
Traneptora
|
2024-10-05 08:46:11
|
for the most part, yea
|
|
|
|
Tirr
|
2024-10-05 08:46:23
|
to match smaller DCT coeffs to larger DCTs (see Section I.8)
|
|
|
|
Traneptora
|
2024-10-05 08:47:09
|
after the forward DCT is run, each of the new coeffs is scaled by a small ish amount based on its position
|
|
2024-10-05 08:48:50
|
the upper left is always scaled by 1.0, but the one at y=0, x=1 or y=1, x=0 is scaled by about 1.2x
|
|
2024-10-05 08:49:04
|
it's just an adjustment to make the math work out
|
|
|
|
CrushedAsian255
|
|
Traneptora
the upper left is always scaled by 1.0, but the one at y=0, x=1 or y=1, x=0 is scaled by about 1.2x
|
|
2024-10-05 08:49:33
|
> scaled by 1.0
doesn't that mean just no scaling?
|
|
|
|
Traneptora
|
2024-10-05 08:49:37
|
yes
|
|
2024-10-05 08:49:49
|
basically when you roundtrip a DCT you end up with an extra linear factor
|
|
|
|
CrushedAsian255
|
|
Traneptora
basically when you roundtrip a DCT you end up with an extra linear factor
|
|
2024-10-05 08:50:11
|
is that deliberate?
|
|
|
|
Traneptora
|
2024-10-05 08:50:16
|
it's a result of the math
|
|
2024-10-05 08:50:37
|
a regular fourier transform, forward, and then backward, is not a perfect roundtrip, you actually get `2pi` times what you started with
|
|
2024-10-05 08:51:02
|
but the transformation is linear, so what you end up doing is you add a `1/2pi` factor to either the forward definition or to the inverse definition
|
|
2024-10-05 08:51:06
|
that way it's a proper roundtrip
|
|
2024-10-05 08:51:29
|
but whether you add the `1/2pi` linear multiple to the forward side, or to the inverse side, is sorta up2u
|
|
2024-10-05 08:51:34
|
provided you're consistent with it
|
|
2024-10-05 08:51:40
|
because it's all linear
|
|
2024-10-05 08:51:50
|
it just factors out of the sum/integral
|
|
2024-10-05 08:52:00
|
for DCT it's the same but it's a factor of 2 instead pf 2pi
|
|
2024-10-05 08:52:46
|
for JXL, there's a `sqrt(2)` on both sides, rather than just a 2 on one side
|
|
2024-10-05 08:53:10
|
that way the coefficients are the same for both forward and inverse and the sqrt(2) can be baked in
|
|
2024-10-05 08:53:32
|
the downside is that you then have to "invert" this baking in when you do this kinda roundtrip shenangians
|
|
2024-10-05 08:54:05
|
it's a bit complicated about the math behind it but it's ultimately a result of the DCT not perfectly roundtripping, so you have to adjust
|
|
|
|
veluca
|
2024-10-05 08:54:54
|
actually, that's not quite it
|
|
|
|
Traneptora
|
2024-10-05 08:55:15
|
unless I misunderstood something, which I think I may have
|
|
|
|
veluca
|
2024-10-05 08:57:07
|
it's easier to think of it for a 1d DCT and a 2x downscale
|
|
2024-10-05 08:57:49
|
actually wait, let's see if I was smart and I left comments in libjxl when I figured this all out
|
|
|
|
CrushedAsian255
|
2024-10-05 08:57:51
|
is a 2d dct just Column1D(Row1D(data)) ?
|
|
|
veluca
actually wait, let's see if I was smart and I left comments in libjxl when I figured this all out
|
|
2024-10-05 08:58:05
|
are there that many comments in libjxl?
|
|
|
|
veluca
|
2024-10-05 08:58:17
|
https://github.com/libjxl/libjxl/blob/main/lib/jxl/dct_scales.h#L18
|
|
|
CrushedAsian255
are there that many comments in libjxl?
|
|
2024-10-05 08:58:52
|
I spent a few days getting the math wrong multiple times, after all of that I decided to get smart and leave traces for the next time I would question my sanity 😛
|
|
|
|
spider-mario
|
2024-10-05 10:08:23
|
along similar lines, I left this here for anyone who would be curious as to why this is the correct formula: https://github.com/libjxl/libjxl/blob/57dd9253b9bc0d1875584666b3c151636925e338/lib/jxl/cms/jxl_cms.cc#L812-L834
|
|
|
|
Traneptora
|
|
CrushedAsian255
is a 2d dct just Column1D(Row1D(data)) ?
|
|
2024-10-05 11:49:23
|
yes
|
|
2024-10-05 11:49:43
|
also doesn't matter which order
|
|
|
|
CrushedAsian255
|
|
Traneptora
also doesn't matter which order
|
|
2024-10-05 11:50:24
|
I thought the spec defines a specific order
|
|
|
|
Traneptora
|
|
CrushedAsian255
I thought the spec defines a specific order
|
|
2024-10-05 12:03:42
|
it doesn't matter because you get the same result
|
|
2024-10-05 12:03:55
|
whether you do row or column first
|
|
|
|
Demiurge
|
|
Dejay
Does JXL still support this? (or plans to support this)
> 4.1.2 Screen readers/clicking on image text
> Examples of text in images include event flyers, company brochures and documents in general. For these, it would be desirable to store the actual text into optional metadata, such that users can interact with it (click on links, copy text).
|
|
2024-10-05 02:38:38
|
good question. first time I heard about this
|
|
2024-10-05 02:38:43
|
sounds like djvu
|
|
|
|
Dejay
|
|
Demiurge
sounds like djvu
|
|
2024-10-05 07:58:00
|
Yeah. It would be easy to add this to metadata and write some javascript to do this, but being part of the spec would be much better.
|
|
2024-10-05 08:01:07
|
Btw PDF supports the same bi-tonal / JBIG compression for scanned document than djvu, so I guess that is why djvu faded out.
|
|
2024-10-05 08:02:59
|
That is probably not new to most people here but I'm just learning about this stuff
|
|
2024-10-05 08:05:34
|
Could be an interesting feature for a smartphone to scan documents into bi-tonal / jbig compressed JpegXL.
|
|
|
|
Demiurge
|
2024-10-05 08:06:55
|
I knew about djvu. It is awesome for raster graphics. But it never took off because the killer feature was not freely available. Multi layer image decomposition with a mix of dct, wavelet, and bitonal coding in one image
|
|
2024-10-05 08:07:25
|
jxl can also benefit from this but no such encoder exists
|
|
2024-10-05 08:08:43
|
libjxl is not optimized at all for bitonal
|
|
|
|
Dejay
|
2024-10-05 08:10:35
|
Well cjxl can already find patches if they are "perfect" and lossless. You'd need to add a special mode, but just automatic bitonal / whats-it-called foreground background separation is tricky
|
|
|
|
Demiurge
|
2024-10-05 08:10:54
|
Such optimizations are possible but not implemented by any encoder
|
|
|
|
Dejay
|
2024-10-05 08:11:19
|
Synthetic image patch test
|
|
2024-10-05 08:11:58
|
Without patches that image is much bigger. So it really is just a question of the encoder
|
|
|
|
jonnyawsom3
|
2024-10-05 08:31:35
|
Monad made some custom patch detection that worked best on tiles https://discord.com/channels/794206087879852103/803645746661425173/1280929625555603480
|
|
2024-10-06 05:25:57
|
Is there a reason JXL isn't much better at Palette than PNG/GIF other than coding features being limited per group?
|
|
|
HCrikki
|
2024-10-06 07:10:57
|
wasnt some optimization for rgb images with less than 256 unique colors reverted at some time?
|
|
|
|
Dejay
|
2024-10-06 07:49:39
|
So how do I incite cjxl to use palette? '--modular_palette_colors=255'? I'm trying to convert a png with grayscale palette colors, gimp does it but cjxl ends up with a larger size and RGB colorspace
|
|
|
|
jonnyawsom3
|
2024-10-06 07:52:12
|
The PNG is likely saved as palette RGB instead of grayscale, so cjxl is making an RGB image with palette too. I'd suggest try running oxipng or ECT on the png to see if it works after
|
|
|
|
Dejay
|
|
The PNG is likely saved as palette RGB instead of grayscale, so cjxl is making an RGB image with palette too. I'd suggest try running oxipng or ECT on the png to see if it works after
|
|
2024-10-06 07:57:35
|
Ah thanks, yeah seems a fully grayscale color palette fixed it
|
|
2024-10-06 08:12:07
|
So when I save this png from "basically grayscale palette" to fully gray, cjxl works, but then gimp still exports this new png to jxl better than cjxl. Obviously I'm doing something wrong (just using cjxl -d 0 -e 9). Or gimp is doing something righter.
```
Length Name
------ ----
58029 b026.grayscale.palette.d0.e9.jxl
56798 b026.grayscale.palette.gimp.d0.e9.jxl
75311 b026.grayscale.palette.png
62077 b026.original.palette.d0.e9.jxl
60501 b026.original.palette.gimp.d0.e9.jxl
80078 b026.original.palette.png
```
|
|
2024-10-06 08:12:18
|
|
|
|
|
jonnyawsom3
|
2024-10-06 08:13:30
|
Oh, try `-e 10` on cjxl
|
|
2024-10-06 08:14:12
|
Since 0.10 the effort levels were tweaked to allow better multithreading
|
|
|
|
Dejay
|
|
Since 0.10 the effort levels were tweaked to allow better multithreading
|
|
2024-10-06 08:15:21
|
Oh I see, because gimp is older they use slower more efficient jxl settings
|
|
2024-10-06 08:15:28
|
Yeah e 10 works
|
|
|
|
jonnyawsom3
|
2024-10-06 08:18:01
|
The old e9 got moved to e10 so everyone would get a performance uplift, since it's not a huge difference in filesize but goes from fully multithreaded to single threaded
|
|
2024-10-06 08:19:08
|
If you wanted to push it, just for fun, you can add `-g 3 -I 100` too. Higher memory usage and time but gives a nice bump to compression
|
|
|
|
Dejay
|
2024-10-06 08:23:38
|
Thanks again. Ok so cjxl not using a palette for the original didn't actually hurt compression. But it still feels off
|
|
|
|
jonnyawsom3
|
2024-10-06 08:27:18
|
It probably was using a palette, but kept the bitdepth the same. Since palette can be for the entire image or per group of 128 x 128 up to 1024 x 1024 pixels
|
|
|
|
Dejay
|
2024-10-06 09:01:37
|
Ok so apparently converting an image with a grayscale palette (3-channel image) to a pure grayscale (1-channel) saves about 120-150 bytes lol
|
|
2024-10-06 09:01:59
|
Which makes sense of course
|
|
|
|
Meow
|
2024-10-07 06:40:35
|
Several months ago I tested some screentone images and PNG performed significantly better than JXL
|
|
|
|
Demiurge
|
2024-10-07 07:03:45
|
I never see png perform better. Even at effort=3
|
|
|
|
CrushedAsian255
|
|
Meow
Several months ago I tested some screentone images and PNG performed significantly better than JXL
|
|
2024-10-07 07:04:05
|
What settings were you using?
|
|
|
|
Meow
|
2024-10-07 07:05:35
|
Only -d 0
|
|
|
|
Demiurge
|
2024-10-07 07:08:38
|
I would be curious to see if I can repeat this
|
|
2024-10-07 07:08:52
|
Are you able to share the images?
|
|
|
Orum
|
2024-10-07 07:31:33
|
I wonder if the png encoder was changing something like bit depth?
|
|
|
|
_wb_
|
2024-10-07 07:32:30
|
Clean screentone patterns can have repetitions that are good for png's DEFLATE.
|
|
|
|
AccessViolation_
|
2024-10-07 08:08:41
|
Does anyone know the reason JPEG XL went with lower limits for bit depth, channel amount and dimensions than JPEG 2000? Was it a trade off with header size?
|
|
|
|
Meow
|
2024-10-07 08:09:33
|
|
|
2024-10-07 08:09:34
|
Don't click if you are anthropomorphobia
|
|
2024-10-07 08:13:36
|
Their JXL counterparts are 12.9% and 306.8% (!) larger respectively
|
|
|
|
Orum
|
2024-10-07 08:19:46
|
```372066 SPOILER_Screentone_Alpha.jxl
375002 SPOILER_Screentone_Alpha.png```
|
|
2024-10-07 08:20:11
|
ran the png through `ect -9` as well
|
|
2024-10-07 08:20:50
|
anyway args used for `cjxl` were: `-d 0 -e 9 --streaming_output -g 3 -I 100`
|
|
|
|
CrushedAsian255
|
|
AccessViolation_
Does anyone know the reason JPEG XL went with lower limits for bit depth, channel amount and dimensions than JPEG 2000? Was it a trade off with header size?
|
|
2024-10-07 08:22:01
|
JPEG 2000’s limits are kind of overboard
|
|
2024-10-07 08:22:17
|
32 bit float / 24 bit int were picked for practical reasons
|
|
|
|
Meow
|
|
Orum
anyway args used for `cjxl` were: `-d 0 -e 9 --streaming_output -g 3 -I 100`
|
|
2024-10-07 08:22:56
|
Very advanced for casual users🫠
|
|
|
|
Orum
|
|
|
Meow
|
2024-10-07 08:25:16
|
Saving 0.8% is still not that impressive however
|
|
|
|
Orum
|
2024-10-07 08:26:06
|
when it's much, *much* faster than `ect -9`, it is
|
|
2024-10-07 08:27:02
|
still, webp does an even better job (here vs the original png, not the ect optimized one):
```311044 SPOILER_Screentone_Alpha.webp
389450 SPOILER_Screentone_Alpha.png```
|
|
2024-10-07 08:28:09
|
any time webp beats jxl, you know there is room for improvement on the jxl side
|
|
|
|
CrushedAsian255
|
2024-10-07 08:28:22
|
Lossy or lossless
|
|
|
|
Orum
|
2024-10-07 08:28:29
|
always lossless
|
|
2024-10-07 08:29:14
|
well, possibly some metadata fell on the cutting room floor with webp <:ChizuWink:857418269647700003>
|
|
|
Jyrki Alakuijala
|
|
_wb_
Clean screentone patterns can have repetitions that are good for png's DEFLATE.
|
|
2024-10-07 09:13:27
|
sometimes a combination of deflate and bit packing and prediction -- JPEG XL's LZ77-like approach is more powerful than that of PNG, with 2d references (i.e., cheaper way to copy from previous row) -- but we don't utilize it much or prioritize it in search
|
|
|
Dejay
Thanks again. Ok so cjxl not using a palette for the original didn't actually hurt compression. But it still feels off
|
|
2024-10-07 09:17:19
|
JPEG XL has two kinds of 'palettes', one kind can densify a channel (such as RGBXYB) and the other kind can record RGB triplets.
|
|
|
Orum
any time webp beats jxl, you know there is room for improvement on the jxl side
|
|
2024-10-07 09:18:54
|
I guided the design of JPEG XL lossless and while the team was benchmarking it a bit differently than I did for WebP lossless design, possibly leading to different prioritization of format features, I paid attention that what I found impactful in WebP lossless design got reflected on the JPEG XL side -- we just haven't build a great encoder yet for lossless
|
|
|
|
CrushedAsian255
|
|
Jyrki Alakuijala
I guided the design of JPEG XL lossless and while the team was benchmarking it a bit differently than I did for WebP lossless design, possibly leading to different prioritization of format features, I paid attention that what I found impactful in WebP lossless design got reflected on the JPEG XL side -- we just haven't build a great encoder yet for lossless
|
|
2024-10-07 09:48:45
|
So Modular is FUIF+WebP LL+PNG?
|
|
|
|
Meow
|
|
Orum
well, possibly some metadata fell on the cutting room floor with webp <:ChizuWink:857418269647700003>
|
|
2024-10-07 09:50:07
|
Also noticed that the resulted WebP's colour space becomes RGB
|
|
|
|
CrushedAsian255
|
|
Jyrki Alakuijala
sometimes a combination of deflate and bit packing and prediction -- JPEG XL's LZ77-like approach is more powerful than that of PNG, with 2d references (i.e., cheaper way to copy from previous row) -- but we don't utilize it much or prioritize it in search
|
|
2024-10-07 09:51:17
|
> we don’t utilise it much or prioritise it in search
I still think it could be helpful once it matures to have tunes for libjxl, the default should still do a good job for any image but the more you know about the input the better, so you could eg apply specific features like patches if the image is a screenshot where they may not work as well for a photographic
|
|
2024-10-07 09:52:45
|
Would help optimise the search space
|
|
|
|
Meow
|
2024-10-07 09:53:24
|
This kind of images is very popular for comics
|
|
|
|
CrushedAsian255
|
2024-10-07 09:54:23
|
Specifically for things like comics would have a specific structure to them that can help optimise searching
|
|
2024-10-07 09:54:58
|
Ie usually black and white, also things like dots and splines would be extra effective
|
|
|
|
jonnyawsom3
|
2024-10-07 10:01:19
|
I think better heuristics are wanted rather than tune options. If you give it text you assume Patches would work, if you give it pixel art it scales and upsamples (Only 2, 4 or 8 and NN, but still a neat idea for e11 or something...)
Then you have to consider groups, if you tune the entire image when areas are different, it could be worse than letting it run automatically
|
|
|
|
CrushedAsian255
|
|
I think better heuristics are wanted rather than tune options. If you give it text you assume Patches would work, if you give it pixel art it scales and upsamples (Only 2, 4 or 8 and NN, but still a neat idea for e11 or something...)
Then you have to consider groups, if you tune the entire image when areas are different, it could be worse than letting it run automatically
|
|
2024-10-07 10:02:35
|
oh, so it could use different internal "tunings" heuristics on each group?
|
|
|
|
jonnyawsom3
|
2024-10-07 10:03:51
|
Long ago there was a PR that would change groups to lossless if they compressed better. So for Manga and some screenshots it would encode a lossless image instead of lossy, for photos it would use VarDCT
|
|
|
|
CrushedAsian255
|
|
Long ago there was a PR that would change groups to lossless if they compressed better. So for Manga and some screenshots it would encode a lossless image instead of lossy, for photos it would use VarDCT
|
|
2024-10-07 10:07:00
|
i can't remember can jxl have vardct and modular groups in the same image?
|
|
2024-10-07 10:07:13
|
im pretty sure you can use patches to do that but i mean like in a singular frame
|
|
|
|
jonnyawsom3
|
2024-10-07 10:07:37
|
Actually you're right, it was patches not groups
|
|
|
|
CrushedAsian255
|
|
Long ago there was a PR that would change groups to lossless if they compressed better. So for Manga and some screenshots it would encode a lossless image instead of lossy, for photos it would use VarDCT
|
|
2024-10-07 10:08:22
|
maybe not lossless but lossy modular could help
|
|
|
|
jonnyawsom3
|
2024-10-07 10:08:56
|
https://github.com/libjxl/libjxl/pull/1395
|
|
|
|
CrushedAsian255
|
2024-10-07 10:10:20
|
only problem is how does this affect progressive Loading?
|
|
|
|
Dejay
|
|
Long ago there was a PR that would change groups to lossless if they compressed better. So for Manga and some screenshots it would encode a lossless image instead of lossy, for photos it would use VarDCT
|
|
2024-10-07 10:13:40
|
That would be nice too for JPG reencode either lossless transcode or visually lossless, sometimes one or the other is smaller
|
|
|
|
jonnyawsom3
|
2024-10-07 10:14:54
|
Well that PR only applies to lossy, Jpeg transcoding is meant to be lossless by definition so suddenly making it lossless isn't a good idea
|
|
|
|
Dejay
|
2024-10-07 10:16:03
|
Yeah but if you want d=1 and lossless transcode is smaller that would be fine too
|
|
|
|
CrushedAsian255
|
|
Dejay
Yeah but if you want d=1 and lossless transcode is smaller that would be fine too
|
|
2024-10-07 10:17:29
|
like if the user specifically requests a distance?
|
|
|
|
Dejay
|
|
CrushedAsian255
like if the user specifically requests a distance?
|
|
2024-10-07 10:19:18
|
Yeah in the use case you just want to compress a bunch of images you don't care too much about, just don't want to add compression artifacts or loss of detail
|
|
2024-10-07 10:20:15
|
The visually lossless feature is really a brilliant killer feature, kinda fire and forget
|
|
|
|
CrushedAsian255
|
|
Dejay
The visually lossless feature is really a brilliant killer feature, kinda fire and forget
|
|
2024-10-07 10:20:45
|
i personally use -d 0.6 to -d 0.8 just to give it some headroom but it's still helpful for quickly compressing something
|
|
|
|
Dejay
|
2024-10-07 10:23:44
|
I figure if you want to further edit you need to go lower. Or if you need to zoom
|
|
|
|
Meow
|
|
CrushedAsian255
i personally use -d 0.6 to -d 0.8 just to give it some headroom but it's still helpful for quickly compressing something
|
|
2024-10-07 10:41:44
|
I use -d 0.5 instead, a well-known visually lossless value
|
|
2024-10-07 10:44:20
|
That's close to JPEG at quality 95 and I often used it before I even knew JXL
|
|
|
|
jonnyawsom3
|
2024-10-07 11:28:44
|
Back when I first found JXL I ended up with -d 0.3 since I always zoom in far more than I should
|
|
|
|
CrushedAsian255
|
|
Back when I first found JXL I ended up with -d 0.3 since I always zoom in far more than I should
|
|
2024-10-07 11:28:53
|
at that point use lossless lol
|
|
2024-10-07 11:29:07
|
when i first found JXL (and to an extent still do) i just use lossless for everything
|
|
|
|
jonnyawsom3
|
2024-10-07 11:29:51
|
The only times I've used lossy the past year has been benchmarking and using jpegli to upload 8K images to Discord
|
|
|
|
Dejay
|
2024-10-07 10:55:35
|
Is there a debug tool to analyze the anatomy of a jxl file?
|
|
2024-10-07 10:55:38
|
Something like "jxlinfo -v -v"? I'd be curious for things like "has palette" and number of colors, number of patches, size of VarDCs, number of frames
|
|
|
|
CrushedAsian255
|
|
Dejay
Something like "jxlinfo -v -v"? I'd be curious for things like "has palette" and number of colors, number of patches, size of VarDCs, number of frames
|
|
2024-10-07 11:18:01
|
`jxl-oxide -I --all-frames --with-offset` has a lot of output, not sure if it's what you were specifically looking for
|
|
|
|
Dejay
|
|
CrushedAsian255
`jxl-oxide -I --all-frames --with-offset` has a lot of output, not sure if it's what you were specifically looking for
|
|
2024-10-07 11:31:20
|
Thanks, didn't realize it had a command line tool 🙂
|
|
|
kr1tzy
|
2024-10-08 02:28:11
|
Does a jxl 010 binary template exist yet? That’d be very useful for what <@134333014892216320> mentioned
|
|
|
|
CrushedAsian255
|
2024-10-08 02:49:47
|
is 010 like imhex?
|
|
|
Dejay
Something like "jxlinfo -v -v"? I'd be curious for things like "has palette" and number of colors, number of patches, size of VarDCs, number of frames
|
|
2024-10-08 02:50:05
|
for size of VarDCTs you can use jxlatte
|
|
|
|
Dejay
|
|
kr1tzy
Does a jxl 010 binary template exist yet? That’d be very useful for what <@134333014892216320> mentioned
|
|
2024-10-08 03:40:58
|
Thanks, this binary template is an interesting approach but the specification seems a bit too complex, also compression etc. You basically need a library to parse and output what you're interested in
|
|
2024-10-08 03:41:46
|
Basically I need to set up a linux PC for development and start playing with libjxl
|
|
|
|
kr1tzy
|
|
CrushedAsian255
is 010 like imhex?
|
|
2024-10-08 03:49:54
|
Not sure if imhex supports templates but 010 is a hex editor that you can define structures you expect and it parses/highlights them in the hex and gives u the details of what you’re looking at. For jpeg xl it’d pick out the header, boxes, frames, etc. A JPEG one does exist so I imagine an XL variation will happen at some point.
|
|
|
|
CrushedAsian255
|
2024-10-08 03:51:54
|
Does it support reading on a bit level?
|
|
2024-10-08 03:52:04
|
Hardly anything in jxl is byte aligned
|
|
|
|
kr1tzy
|
2024-10-08 03:52:45
|
I’m just now getting into jxl. Are there any robust parsers floating around atm?
|
|
|
|
CrushedAsian255
|
|
kr1tzy
I’m just now getting into jxl. Are there any robust parsers floating around atm?
|
|
2024-10-08 03:53:28
|
`jxl-oxide -I --all-frames --with-offset`
|
|
|
|
Dejay
|
2024-10-08 03:53:45
|
Doh... I've been wondering why this jxl file has some noise applied to the background and can't be recompressed the same with lossless... then I remembered that I played around with the --photon_noise_iso=3200 haha
|
|
|
|
kr1tzy
|
|
CrushedAsian255
Does it support reading on a bit level?
|
|
2024-10-08 03:54:16
|
Everything I’ve built for it has been byte aligned but I imagine it’s possible. I’ll have to loop back on this.
|
|
|
|
CrushedAsian255
|
2024-10-08 03:55:04
|
i wish jxl has jxl for jxl support with jxl
|
|
2024-10-08 03:56:31
|
ooh nice the image on https://jpeg.org/jpegxl/ is a jxl
|
|
2024-10-08 03:56:41
|
and in hdr
|
|
|
VcSaJen
|
|
kr1tzy
Not sure if imhex supports templates but 010 is a hex editor that you can define structures you expect and it parses/highlights them in the hex and gives u the details of what you’re looking at. For jpeg xl it’d pick out the header, boxes, frames, etc. A JPEG one does exist so I imagine an XL variation will happen at some point.
|
|
2024-10-08 04:03:43
|
ImHex support patterns, yes.
|
|
|
|
Demiurge
|
|
CrushedAsian255
Hardly anything in jxl is byte aligned
|
|
2024-10-08 07:35:31
|
>:(
|
|
|
Kingproone
|
2024-10-08 11:21:21
|
I just saw the new website at jpegxl.info, looks amazing, I have seen some misalignment on the hdr test page:
|
|
|
|
VcSaJen
|
2024-10-09 02:40:44
|
I know that there are cases where lossless WEBP is better than JXL for screenshots. What about on average on game screenshots? Is it better than lossless WEBP?
|
|
|
|
Quackdoc
|
2024-10-09 02:41:43
|
well if you run your game in 10bit the answer would be always lol
|
|
|
|
CrushedAsian255
|
2024-10-09 02:43:27
|
<@853026420792360980> i found a bug with jxlatte
|
|
|
|
Traneptora
|
|
CrushedAsian255
<@853026420792360980> i found a bug with jxlatte
|
|
2024-10-09 02:52:13
|
yes?
|
|
|
|
CrushedAsian255
|
2024-10-09 02:52:26
|
```
jxlatte: BUG: Index 0 out of bounds for length 0
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at com.traneptora.jxlatte.frame.vardct.HFCoefficients.<init>(HFCoefficients.java:57)
at com.traneptora.jxlatte.frame.group.PassGroup.<init>(PassGroup.java:73)
at com.traneptora.jxlatte.frame.Frame.decodePassGroups(Frame.java:342)
at com.traneptora.jxlatte.frame.Frame.decodeFrame(Frame.java:429)
at com.traneptora.jxlatte.JXLCodestreamDecoder.decode(JXLCodestreamDecoder.java:741)
at com.traneptora.jxlatte.JXLCodestreamDecoder.decode(JXLCodestreamDecoder.java:670)
at com.traneptora.jxlatte.JXLDecoder.decode(JXLDecoder.java:36)
at com.traneptora.jxlatte.JXLatte.writeImage(JXLatte.java:281)
at com.traneptora.jxlatte.JXLatte.main(JXLatte.java:347)
```
|
|
2024-10-09 02:52:32
|
i was trying to decode an image
|
|
2024-10-09 02:52:35
|
to show the varblocks
|
|
2024-10-09 02:52:45
|
do you want the jxl file?
|
|
2024-10-09 02:59:06
|
<@853026420792360980>
|
|
|
|
Traneptora
|
2024-10-09 03:09:32
|
yes please
|
|
2024-10-09 03:09:38
|
sorry been a bit busy
|
|
|
|
CrushedAsian255
|
|
Traneptora
yes please
|
|
2024-10-09 03:26:30
|
|
|
|
|
Demiurge
|
2024-10-09 05:14:36
|
It's not the format. It's the libjxl encoder less optimized than Jyrki's lossless webp code
|
|
2024-10-09 05:17:07
|
Hopefully someday soon someone will write a good encoder for JXL too 😂
|
|
|
|
yoochan
|
2024-10-09 05:21:27
|
For some kind of images... For photos, jxl is already better.
|
|
|
|
CrushedAsian255
|
2024-10-09 05:31:29
|
maybe we can put some of the stuff from the WebP lossless encoder into libjxl
|
|
2024-10-09 05:31:33
|
like some of the techniques
|
|
2024-10-09 05:31:42
|
or im guessing he has already done that
|
|
|
|
yoochan
|
|
CrushedAsian255
or im guessing he has already done that
|
|
2024-10-09 05:52:21
|
If I understood well, some tricks of palette reordering could be ported (maybe) but haven't yet (?)
|
|
2024-10-09 05:54:39
|
https://docs.google.com/spreadsheets/u/0/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/htmlview?pli=1
But, except for some problematic files, jxl lossless is far better than webp
|
|
|
|
Demiurge
|
|
CrushedAsian255
or im guessing he has already done that
|
|
2024-10-09 05:59:26
|
webp-lossless only supports an extremely limited very basic 8 bit image type whereas jxl supports anything, so because of how different the formats are, they can't just be ported over, they have to be re-written, even if the same optimizations are possible.
|
|
|
|
jonnyawsom3
|
|
Jyrki Alakuijala
I guided the design of JPEG XL lossless and while the team was benchmarking it a bit differently than I did for WebP lossless design, possibly leading to different prioritization of format features, I paid attention that what I found impactful in WebP lossless design got reflected on the JPEG XL side -- we just haven't build a great encoder yet for lossless
|
|
2024-10-09 08:20:55
|
He talked about it the other day
|
|
|
|
Traneptora
|
|
CrushedAsian255
|
|
2024-10-09 11:01:52
|
found the bug, pushed a fix to git
|
|
|
|
CrushedAsian255
|
|
Traneptora
found the bug, pushed a fix to git
|
|
2024-10-09 11:02:25
|
lmao git gud
|
|
|
|
Traneptora
|
2024-10-09 11:02:50
|
```
git: 'gud' is not a git command. See 'git --help'.
```
|
|
2024-10-09 11:02:58
|
how can I git gud
|
|
|
|
CrushedAsian255
|
2024-10-09 11:03:03
|
alias it to bisect
|
|
2024-10-09 11:03:26
|
so when there is a bug you tell yourself to "git gud" and it helps you fix it
|
|
|
|
Traneptora
|
2024-10-09 11:03:43
|
```sh
git ()
{
if [ "$*" = "gud" ]; then
printf 'git gud --scrub\n';
else
command git "$@";
fi
}
```
|
|
|
|
CrushedAsian255
|
2024-10-09 11:04:05
|
lol
|
|
2024-10-09 11:04:29
|
should i run jxlatte against all my jxls to effectively fuzz?
|
|
|
|
Traneptora
|
2024-10-09 11:04:40
|
if you want that would be helpful
|
|
|
|
lonjil
|
2024-10-09 11:05:41
|
Should it produce PNGs that are extremely similar to what djxl produces without special decoding options? So I can just run a script on a ton of stuff and automatically compare.
|
|