|
jonnyawsom3
|
|
Oleksii Matiash
For jpeg recompression e10 is fast enough to use
|
|
2024-08-27 11:40:14
|
Also `--brotli_effort=11` if you have reconstruction enabled
|
|
|
Oleksii Matiash
|
|
Also `--brotli_effort=11` if you have reconstruction enabled
|
|
2024-08-27 11:41:28
|
Yes, it was part of my recompression script 🙂
|
|
|
|
jonnyawsom3
|
|
CrushedAsian255
i haven't actually tested the speed for -e 11 on JPEG recompression,
|
|
2024-08-27 11:41:44
|
I'm not sure it actually does anything, since e11 was mostly just for Modular mode's numerous settings, and jpeg transcodes are straight to VarDCT. With helps with the DC slightly
|
|
|
HCrikki
|
|
Oleksii Matiash
For jpeg recompression e10 is fast enough to use
|
|
2024-08-27 11:41:46
|
for reconstructible jxls from jpg sources, e7 os literally instant (less than 50ms for anything below 20megapixels on avg desktop system) , you dont really gain anything but extra very few kilobytes from any higher effort at way longer duration cost. post 0.10 e7 is already as good as what was once e9 before 0.10
|
|
|
|
Oleksii Matiash
|
|
I'm not sure it actually does anything, since e11 was mostly just for Modular mode's numerous settings, and jpeg transcodes are straight to VarDCT. With helps with the DC slightly
|
|
2024-08-27 11:42:19
|
I'm not sure, but for me it seems to be related to exif compression
|
|
|
|
veluca
|
2024-08-27 11:42:40
|
I am not even sure efforts higher than 7 really do anything for JPEG recompression...
|
|
|
CrushedAsian255
|
|
Oleksii Matiash
I'm not sure, but for me it seems to be related to exif compression
|
|
2024-08-27 11:42:47
|
it probably also ramps up Brotli effort
|
|
|
|
Oleksii Matiash
|
|
HCrikki
for reconstructible jxls from jpg sources, e7 os literally instant (less than 50ms for anything below 20megapixels on avg desktop system) , you dont really gain anything but extra very few kilobytes from any higher effort at way longer duration cost. post 0.10 e7 is already as good as what was once e9 before 0.10
|
|
2024-08-27 11:42:51
|
I just want these few kilobytes 🤷♂️
|
|
|
veluca
I am not even sure efforts higher than 7 really do anything for JPEG recompression...
|
|
2024-08-27 11:42:56
|
They do
|
|
|
|
jonnyawsom3
|
2024-08-27 11:43:05
|
On a 0.016 MP image, e11 takes 10 minutes of CPU time... So yeah, don't use it on anything outside of pixel art xD
|
|
|
|
veluca
|
|
Oleksii Matiash
They do
|
|
2024-08-27 11:43:18
|
then I forgot what it does xD
|
|
|
|
jonnyawsom3
|
2024-08-27 11:43:36
|
Time for more additions to the effort chart ;P
|
|
|
|
CrushedAsian255
|
|
veluca
then I forgot what it does xD
|
|
2024-08-27 11:44:11
|
probably optimises the modular DC
|
|
|
|
veluca
|
2024-08-27 11:46:03
|
yeah, I probably did that xD and I guess it also does a bit more of an exhaustive search for the histograms, although I doubt _that_ changes much
|
|
|
|
CrushedAsian255
|
|
veluca
yeah, I probably did that xD and I guess it also does a bit more of an exhaustive search for the histograms, although I doubt _that_ changes much
|
|
2024-08-27 11:46:43
|
well, if it saves 84 bytes out of a 3.2 MB image, it's worth it
|
|
|
|
Oleksii Matiash
|
2024-08-27 11:51:40
|
File without -N is -e 9 from libjxl 0.8 or 0.7. Funny enough -e 10 produced larger file than -e 9, and all 0.10.3 results are larger than 0.8 (or 0.7, don't remember). Just for clarity - these files are jpeg-in-jxl
|
|
|
DNFrozen
|
2024-08-27 11:52:38
|
the internal exe file of xl converter that does the jpeg recompression only supports effort 1-10 not 11. is it outdated?
|
|
|
Tirr
|
2024-08-27 11:53:15
|
e11 is "expert" option so it's hidden
|
|
|
|
CrushedAsian255
|
2024-08-27 11:53:22
|
you need to pass `--allow_expert_options -e 11`
|
|
|
|
DNFrozen
|
2024-08-27 11:58:16
|
isn't gif a lossless format (aside from the fact that i only supports 256 color?)
|
|
|
Demiurge
|
|
DNFrozen
the interface is nice. but the effort values max out at 9 and I've heard here that there is also 10 and 11 option for conversion
|
|
2024-08-27 12:01:45
|
You don't want to bother going too deeply into diminishing returns...
|
|
2024-08-27 12:02:04
|
it's not worth it.
|
|
2024-08-27 12:02:08
|
Just don't.
|
|
|
|
DNFrozen
|
2024-08-27 12:02:13
|
i still want to try it with one image now
|
|
|
|
Oleksii Matiash
|
|
DNFrozen
isn't gif a lossless format (aside from the fact that i only supports 256 color?)
|
|
2024-08-27 12:02:18
|
It is lossy in another meaning, I believe. As it only allows 256 colors, it will be dithered in 99.9% cases, so doing lossy on it will both have bad compression ratio and lots of artifacts
|
|
|
|
DNFrozen
|
2024-08-27 12:02:53
|
i don't see a special option for lossless recompression. is that the distance value thats set 0.0 by default? or do i need to set quality to 100?
|
|
|
|
Demiurge
|
2024-08-27 12:03:35
|
It's a special option.
|
|
2024-08-27 12:03:48
|
If it's not a special option then maybe your interface doesn't support it.
|
|
|
|
CrushedAsian255
|
2024-08-27 12:03:49
|
it's an entirely differnet encoding pathway
|
|
2024-08-27 12:03:59
|
it bypasses half of the JXL encoding process
|
|
|
|
Demiurge
|
2024-08-27 12:04:39
|
for cjxl you want to use `-j 1` I think
|
|
|
|
CrushedAsian255
|
2024-08-27 12:05:06
|
PNG to JXL:
PNG -> Pixels -> VarDCT blocks -> JXL
JPEG to JXL:
JPEG -> DCT Coefficients -> JXL
Not converting to pixels is how it manages to not loose any quality
|
|
|
Demiurge
for cjxl you want to use `-j 1` I think
|
|
2024-08-27 12:05:23
|
i thought JXL recompression is on by default
|
|
|
username
|
2024-08-27 12:05:31
|
it is
|
|
|
|
CrushedAsian255
|
2024-08-27 12:05:36
|
i've only had to use `-j` to explicitly turn it off (`-j 0`)
|
|
|
|
Demiurge
|
2024-08-27 12:05:47
|
it is on by default... but it's good to be explicit, so that there's an error if it fails.
|
|
|
|
username
|
2024-08-27 12:06:02
|
XL convertor has it off by default but cjxl has it on by default
|
|
|
|
CrushedAsian255
|
|
username
XL convertor has it off by default but cjxl has it on by default
|
|
2024-08-27 12:06:13
|
why would XL converter do that?
|
|
|
|
DNFrozen
|
2024-08-27 12:06:30
|
oh nvm there is an advanced help page that lists the lossless option
|
|
|
|
Demiurge
|
2024-08-27 12:07:31
|
If you use `-j 1` then you can be sure it won't attempt to decode to pixels.
|
|
2024-08-27 12:08:13
|
And there will be an error if it's unable to do it correctly
|
|
|
|
jonnyawsom3
|
|
username
XL convertor has it off by default but cjxl has it on by default
|
|
2024-08-27 12:09:03
|
That might explain the subpar 10% compression they got on average
|
|
|
|
username
|
|
CrushedAsian255
why would XL converter do that?
|
|
2024-08-27 12:09:39
|
the creator seems to want metadata stripping to be available and on by default for all images put through it and JXL JPEG recompression retains metadata which goes against that XL converters strange want to remove metatdata from everything
|
|
|
|
Demiurge
|
2024-08-27 12:09:49
|
I usually get 30% shaved off PNG and 10% shaved off JPEG
|
|
2024-08-27 12:10:28
|
with well optimized png it's probably more like 10% as well
|
|
2024-08-27 12:10:43
|
or more
|
|
|
|
CrushedAsian255
|
|
username
the creator seems to want metadata stripping to be available and on by default for all images put through it and JXL JPEG recompression retains metadata which goes against that XL converters strange want to remove metatdata from everything
|
|
2024-08-27 12:11:05
|
why would they explicitly want metadata stripping?
|
|
2024-08-27 12:11:17
|
i can't live without my Exif 😦
|
|
|
|
Demiurge
|
2024-08-27 12:11:34
|
With larger amounts of images the gains are less miraculous
|
|
|
|
jonnyawsom3
|
2024-08-27 12:11:45
|
It's also compressed in JXL anyway so takes less space
|
|
|
|
CrushedAsian255
|
|
It's also compressed in JXL anyway so takes less space
|
|
2024-08-27 12:12:34
|
Brotli?
|
|
|
|
jonnyawsom3
|
|
|
Demiurge
|
2024-08-27 12:13:31
|
that's pretty weird... Metadata stripping should be an explicit option that's explicitly asked for...
|
|
|
|
CrushedAsian255
|
2024-08-27 12:13:41
|
i put like 2 weeks solid into making sure every image i have has proper Exif metadata, i would be annoyed if i used a program that decided to strip it all out
|
|
|
|
Demiurge
|
2024-08-27 12:13:48
|
A separate task
|
|
|
|
username
|
2024-08-27 12:14:00
|
I don't know why some people seem to be allergic to metadata, like I genuinely find metadata to be useful and removing it saves barely any space especially with the mentioned JXL compression of metadata. I personally believe that metadata stripping should be opt-in rather then opt-out
|
|
|
|
CrushedAsian255
|
2024-08-27 12:14:36
|
i would say metadata stripping is only really a good idea if you're sharing the images, as you might not want things like GPS data public
|
|
|
|
jonnyawsom3
|
|
username
the creator seems to want metadata stripping to be available and on by default for all images put through it and JXL JPEG recompression retains metadata which goes against that XL converters strange want to remove metatdata from everything
|
|
2024-08-27 12:17:56
|
<@365440150785753088> did you have this enabled?
|
|
|
|
Demiurge
|
2024-08-27 12:18:20
|
https://xl-docs.codepoems.eu/metadata
|
|
|
|
CrushedAsian255
|
2024-08-27 12:19:09
|
> Preserving metadata is challenging because image formats are rarely 100% compatible with one another.
Exif is pretty universal
|
|
|
|
DNFrozen
|
2024-08-27 12:19:55
|
i had selected lossless recompression in the output tab
|
|
|
|
jonnyawsom3
|
2024-08-27 12:20:14
|
Okay, good
|
|
|
|
username
|
|
DNFrozen
i had selected lossless recompression in the output tab
|
|
2024-08-27 12:21:32
|
what about the Metadata option under the "Modify" tab?
|
|
|
|
DNFrozen
|
2024-08-27 12:21:56
|
encoder wipe
|
|
2024-08-27 12:22:47
|
so I'm done with my effort 11 test and the file is bigger than the effort 9 form xl converter
so i messed effort parameter or this is metadata
|
|
|
|
CrushedAsian255
|
2024-08-27 12:23:15
|
how much bigger are we talking?
|
|
|
|
DNFrozen
|
2024-08-27 12:23:22
|
is was almost instat done and not super slow as expected
|
|
|
|
Demiurge
|
2024-08-27 12:23:26
|
the website says that lossless jpeg recompression always preserves metadata regardless of settings
|
|
|
|
CrushedAsian255
|
|
DNFrozen
is was almost instat done and not super slow as expected
|
|
2024-08-27 12:24:15
|
it's probably just the JPEG reconstruction metadata
|
|
2024-08-27 12:24:25
|
it's usually a couple hundred bytes in my experience
|
|
|
|
Demiurge
|
2024-08-27 12:24:47
|
if you modify the metadata I'm pretty sure the checksum will change and the jpeg reconstruction will fail
|
|
2024-08-27 12:25:09
|
so you may as well strip the jbrd box and container
|
|
|
|
CrushedAsian255
|
|
Demiurge
so you may as well strip the jbrd box and container
|
|
2024-08-27 12:25:22
|
i don't know why XL converter doesn't just do that if they really don't want metadata
|
|
2024-08-27 12:25:34
|
you still get the efficiency gains of JPEG->JXL conversion
|
|
2024-08-27 12:25:57
|
correct me if I'm wrong, but excluding the `jbrd` box still means the image is viewable, just it can't be reconstructed
|
|
|
|
Demiurge
|
2024-08-27 12:26:05
|
because being obsessed with removing metadata without being requested to do that is pretty dumb
|
|
|
|
CrushedAsian255
|
2024-08-27 12:27:27
|
IMO it should be metadata is preserved unless asked for removal, unless image is being shared, then it might make sense to remove metadata
|
|
|
|
DNFrozen
|
2024-08-27 12:30:05
|
so the files that i converted using xl converter can't be restored to their original jpg state?
|
|
|
Quackdoc
|
2024-08-27 12:30:52
|
try it and see I dunno
|
|
|
|
username
|
2024-08-27 12:31:59
|
I think there was a mixup or misunderstanding along the way. it seems like you trying the higher effort value got a slightly larger image which then someone assumed was JPEG reconstruction data being stripped but the likely answer is something else
|
|
2024-08-27 12:32:47
|
I don't think XL convertor can strip JXL JPEG reconstrction data but hey I could be wrong
|
|
|
|
DNFrozen
|
2024-08-27 12:32:58
|
yeah i assumed that because effort 11 should not be bigger than effor 9
|
|
2024-08-27 12:33:18
|
|
|
2024-08-27 12:33:29
|
they could be restored
|
|
|
|
Demiurge
|
2024-08-27 12:36:39
|
`cjxl --container=0 --allow_jpeg_reconstruction=0`
|
|
2024-08-27 12:37:14
|
`-x strip=???`
|
|
|
|
DNFrozen
|
2024-08-27 12:39:57
|
are my parameters missing something that could make is smaller?
` --allow_expert_options -e 11 -j 1`
|
|
|
|
Oleksii Matiash
|
2024-08-27 12:42:57
|
I don't know what author of xl converter meant by 'encoder - wipe', but it does not touch jpeg metadata when compressing. So original jpeg can be restored without anything lost
|
|
|
|
DNFrozen
|
2024-08-27 12:49:06
|
oh somehow the `-e 9 -j 1` paramater results in a 200 byte smaller image than `--allow_expert_options -e 11 -j 1`
|
|
2024-08-27 12:50:11
|
not what i expected but somehow effort 11 can be worse that effort 9
|
|
|
|
jonnyawsom3
|
|
DNFrozen
is was almost instat done and not super slow as expected
|
|
2024-08-27 12:51:16
|
Oh, the standard lossless is where e11 shows it's bite, try a PNG input... But I'd say no larger than 1080p if you don't want to wait hours
|
|
|
|
DNFrozen
|
2024-08-27 12:52:46
|
yeah i just expected that e11 means brute force and therefore proven smallest possible result
|
|
|
|
CrushedAsian255
|
2024-08-27 12:52:56
|
is there any actual purpose of e11 other than goofing around and testing?
|
|
2024-08-27 12:53:00
|
like is anyone ever going to use it?
|
|
|
|
jonnyawsom3
|
2024-08-27 12:53:26
|
For 1:1 scale pixel art or icons it's actually been *ok*
|
|
|
|
DNFrozen
|
2024-08-27 12:53:59
|
idn for recompression it was done almost instant so if it had been smaller than i would have used it
|
|
|
|
CrushedAsian255
|
2024-08-27 12:54:23
|
got exiftool working
|
|
2024-08-27 12:54:34
|
apparently `io::compress::brotli` needed dependencies
|
|
2024-08-27 12:54:40
|
i ended up just using `cpan`
|
|
2024-08-27 12:54:53
|
package managers ftw
|
|
|
|
DNFrozen
|
2024-08-27 01:00:57
|
for lossless compression you just set the quality param to 100 right?
|
|
|
Orum
|
2024-08-27 01:01:56
|
I use distance of 0, but I think q 100 works too?
|
|
|
|
CrushedAsian255
|
|
Orum
I use distance of 0, but I think q 100 works too?
|
|
2024-08-27 01:02:42
|
quality just maps to distance
|
|
2024-08-27 01:02:50
|
100 quality is 0 distance
|
|
|
|
DNFrozen
|
2024-08-27 01:04:19
|
what is the difference between distance and quality? if they were the same there would be no need to have both
|
|
|
|
Orum
|
2024-08-27 01:04:50
|
there is no need to have both, but apparently people complain if they don't have a quality scale, because other image encoders use that
|
|
2024-08-27 01:05:31
|
meanwhile distance is more like QP/CRF/etc. in video encoders, where higher number = lower quality, which is what I'm most used to
|
|
|
|
DNFrozen
|
2024-08-27 01:06:00
|
oh ok so using both parametars at the same time makes really no sense at all
|
|
|
|
Orum
|
2024-08-27 01:06:06
|
exactly
|
|
|
|
CrushedAsian255
|
|
DNFrozen
oh ok so using both parametars at the same time makes really no sense at all
|
|
2024-08-27 01:06:08
|
No, you don’t want to do that
|
|
|
|
jonnyawsom3
|
2024-08-27 01:08:20
|
Quality changes across formats and encoders, so (I think) most of us use distance to avoid setting the wrong number based on other formats
|
|
|
|
CrushedAsian255
|
2024-08-27 01:09:58
|
Distance is kinda saying “how far away from the original are you okay with”
|
|
2024-08-27 01:10:16
|
With 0 meaning “no changes please” aka lossless
|
|
|
|
DNFrozen
|
2024-08-27 01:12:29
|
oh god i just tried e11 on a png.
this is really not my new default option for conversion
|
|
|
|
CrushedAsian255
|
2024-08-27 01:12:52
|
Yep lol
|
|
|
|
DNFrozen
|
2024-08-27 01:13:22
|
very well optimized for multiple cores
|
|
|
|
CrushedAsian255
|
2024-08-27 01:14:59
|
Encode images and warm your home at the same time!
|
|
2024-08-27 01:15:25
|
<:JXL:805850130203934781> for the win
|
|
|
|
DNFrozen
|
2024-08-27 01:16:47
|
cpu is actually just using 40w because of undervolting and and fixed limit of 3.5GhZ
|
|
|
|
CrushedAsian255
|
2024-08-27 01:17:11
|
Imagine overclocked JXL
|
|
2024-08-27 01:17:44
|
Maybe encoding -e 11 JXLs should be a new PoW algorithm for cryptocurrencies
|
|
|
|
DNFrozen
|
2024-08-27 01:17:53
|
well it's still going. i want at least 2kb reduction compared to e10
|
|
|
|
CrushedAsian255
|
2024-08-27 01:18:16
|
Don’t be surprised if you save like 29 bytes
|
|
2024-08-27 01:18:41
|
How big of an image did you give it?
|
|
|
|
DNFrozen
|
|
|
CrushedAsian255
|
2024-08-27 01:19:27
|
Resolution?
|
|
|
|
DNFrozen
|
2024-08-27 01:20:03
|
eh i gues 800x1280
|
|
|
|
CrushedAsian255
|
2024-08-27 01:20:29
|
Cya tomorrow
|
|
|
|
Oleksii Matiash
|
2024-08-27 01:20:35
|
You'd have to wait 😅
|
|
|
|
DNFrozen
|
2024-08-27 01:21:12
|
stopping it now would hurt my pride
|
|
2024-08-27 01:21:55
|
can't we throw a few 4090 at this important problem?
|
|
|
|
CrushedAsian255
|
2024-08-27 01:22:23
|
Can cjxl gpu accelerate?
|
|
2024-08-27 01:22:26
|
Imagine that
|
|
|
|
DNFrozen
|
2024-08-27 01:25:49
|
so there are millions of combinations and the lower effort values pick the most likely best based on heuristic?
|
|
2024-08-27 01:26:13
|
|
|
2024-08-27 01:26:30
|
that is actually not that bad
|
|
|
|
jonnyawsom3
|
2024-08-27 02:36:40
|
e11 runs 1 e10 encode per thread with different parameters based on `-E -g -I -P`, ect but it isn't a complete brute force that runs for eternity until it gets no smaller
|
|
|
|
DNFrozen
|
2024-08-27 02:37:47
|
is there a "runs for eternity" parameter?
|
|
|
|
jonnyawsom3
|
2024-08-27 02:37:55
|
There's a brief discussion about it here https://discord.com/channels/794206087879852103/803645746661425173/1275352147004756040 along with some extensive benchmarking against e11 (and the underlying parameters it uses)
|
|
|
|
Just me
|
2024-08-27 03:16:22
|
Squoosh doesn't have effort 11 at all. This may mislead people!
|
|
|
|
jonnyawsom3
|
2024-08-27 03:26:14
|
That's because Squoosh is on version 0.6
|
|
|
|
DNFrozen
|
2024-08-27 05:44:35
|
a neural network that tries to predict the best compression parameters for an imaage would be usefull
|
|
|
|
CrushedAsian255
|
2024-08-28 04:49:31
|
If I edit the Exif metadata of a losslessly compressed JPEG, can i still convert back to JPEG, if not does the image still open?
|
|
2024-08-28 05:35:46
|
i applied https://tryitands.ee mentality, apparently not
|
|
2024-08-28 06:23:26
|
That’s what I observed as well
|
|
2024-08-28 06:23:46
|
I guess if I want to edit JPEG->JXL metadata I need to convert back to JPEG
|
|
|
_wb_
|
2024-08-28 06:28:27
|
Ah, if the Exif blob length changes, it will no longer match the length expected by the jbrd, and it will fail.
|
|
2024-08-28 06:30:53
|
The only way to avoid that would be to also update the jbrd to have a different size for the marker containing the Exif, that is, pretend that the original jpeg had a different sized Exif all along...
|
|
|
|
Demiurge
|
|
_wb_
The only way to avoid that would be to also update the jbrd to have a different size for the marker containing the Exif, that is, pretend that the original jpeg had a different sized Exif all along...
|
|
2024-08-28 06:54:44
|
There should also be a way to detect files that can be losslessly transcoded to JPEG without using jbrd at all.
|
|
2024-08-28 06:56:47
|
Also, from what I heard, at higher qualities 8x8 DCT is good enough and variable size blocks is a waste of time... if I remember correctly.
|
|
|
|
_wb_
|
2024-08-28 07:10:57
|
Yes, that would be useful. There are quite a few conditions that need to hold, most of which would not be true in default vardct when encoding from pixels:
1. only dct8x8 without adaptive quantization
2. quantization factors are all integers
3. no gaborish/epf/patches/splines/upsampling/extra channels/...
4. colorspace is RGB or YCbCr
|
|
|
|
Demiurge
|
2024-08-28 07:13:51
|
XYB too?
Adaptive quantization is pretty nice to have... larger block sizes is also cool if you have for example an extremely large, noisy, blurry image like a lot of cameras produce.
|
|
|
|
CrushedAsian255
|
|
_wb_
Yes, that would be useful. There are quite a few conditions that need to hold, most of which would not be true in default vardct when encoding from pixels:
1. only dct8x8 without adaptive quantization
2. quantization factors are all integers
3. no gaborish/epf/patches/splines/upsampling/extra channels/...
4. colorspace is RGB or YCbCr
|
|
2024-08-28 07:25:58
|
But if the EXIF changes, the image itself is still fully intact?
|
|
|
|
Demiurge
|
2024-08-28 07:34:54
|
why wouldn't it be? It's intact, but the jpeg bitstream reconstruction box (jbrd) has mismatching info about the reconstructed jpeg unless the jbrd is updated to match the new exif.
|
|
2024-08-28 07:36:53
|
because of this, trying to reconstruct the original jpeg is impossible, so there's an error when trying to do so, because of the mismatching info. But the information to decode the image is still intact, and the jbrd can theoretically be repaired.
|
|
2024-08-28 07:37:02
|
to produce a new jpeg file again.
|
|
2024-08-28 07:37:40
|
decode to pixels will always work.
|
|
|
|
Oleksii Matiash
|
2024-08-28 07:42:34
|
If I understand correctly, jpeg can still be restored, just without exif?
|
|
|
|
CrushedAsian255
|
|
Demiurge
why wouldn't it be? It's intact, but the jpeg bitstream reconstruction box (jbrd) has mismatching info about the reconstructed jpeg unless the jbrd is updated to match the new exif.
|
|
2024-08-28 07:44:28
|
Someone should send a patch to Exiftool to allow it to modify said jbrd marker
|
|
2024-08-28 07:48:27
|
I would but I don’t know any Perl
|
|
2024-08-28 07:48:43
|
And also don’t fully understand jbrd’s format
|
|
|
|
Demiurge
|
2024-08-28 07:52:43
|
perl is designed to be fast to get started with
|
|
|
|
CrushedAsian255
|
2024-08-28 07:52:54
|
Might give it a crack
|
|
2024-08-28 08:14:55
|
do Exiftool have an Git or version controls
|
|
|
|
Demiurge
|
2024-08-28 09:07:54
|
https://exiftool.org/#links
|
|
2024-08-28 09:11:56
|
Not sure. You might need to email Phil with a patch...
|
|
2024-08-28 09:14:51
|
https://github.com/exiftool/exiftool/pulls
|
|
|
|
Just me
|
2024-08-28 11:32:49
|
Perl has pretty wild and unrestricted syntax. Modern AOT/JIT compiled languages are better... Even JavaScript... Even Python...
|
|
|
spider-mario
|
2024-08-28 11:40:41
|
eh, perl does lexical scoping better than python
|
|
|
|
Demiurge
|
2024-08-28 11:41:07
|
Well it was meant for just getting things done fast, and letting you do what you want without worrying about the right way
|
|
|
|
spider-mario
|
2024-08-28 11:41:43
|
and will actually catch undeclared variables (or assignment to a nonexisting variable you didn’t mean to use) before running, unlike python (provided that you `use strict;`, which you should)
|
|
|
|
Demiurge
|
2024-08-28 11:41:52
|
It's the "just get it done" language and there's lots of virtue in that.
|
|
|
|
spider-mario
|
2024-08-28 11:44:53
|
```perl
use strict;
use warnings;
my $variable = 42;
if ('a' eq 'a') {
$varjable = 44; # typo
}
print "$variable\n";
```
```console
$ perl test.pl
Global symbol "$varjable" requires explicit package name (did you forget to declare "my $varjable"?) at test.pl line 7.
Execution of test.pl aborted due to compilation errors.
```
|
|
2024-08-28 11:45:19
|
```python
variable = 42
if 'a' == 'a':
varjable = 44 # typo
print(variable)
```
```console
$ python test.py
42
```
|
|
|
|
CrushedAsian255
|
|
spider-mario
```perl
use strict;
use warnings;
my $variable = 42;
if ('a' eq 'a') {
$varjable = 44; # typo
}
print "$variable\n";
```
```console
$ perl test.pl
Global symbol "$varjable" requires explicit package name (did you forget to declare "my $varjable"?) at test.pl line 7.
Execution of test.pl aborted due to compilation errors.
```
|
|
2024-08-29 12:07:02
|
i prefer this
|
|
2024-08-29 12:07:10
|
i want my code to fail at compile. not silently give wrong answers
|
|
|
|
Demiurge
|
2024-08-29 12:16:42
|
Notice that python does not even warn about an unused variable.
|
|
|
|
CrushedAsian255
|
2024-08-29 12:18:43
|
ah my faviourite, `varjable`
|
|
|
|
DNFrozen
|
2024-08-29 04:32:12
|
in general i prefer if the language requires you to specify if a var is int,float,string etc
|
|
|
TheBigBadBoy - 𝙸𝚛
|
|
|
Orum
|
2024-08-29 06:39:21
|
strongly typed?
|
|
|
|
spider-mario
|
2024-08-29 07:05:50
|
it’s a bit orthogonal
|
|
2024-08-29 07:06:08
|
I think you’re thinking of “statically typed”, but even then, not all statically-typed languages require explicit type annotations
|
|
2024-08-29 07:06:12
|
(OCaml, Haskell)
|
|
2024-08-29 07:08:17
|
https://en.wikipedia.org/wiki/Strong_and_weak_typing#Variation_across_programming_languages
> Smalltalk, Ruby, Python, and Self are all "strongly typed" in the sense that typing errors are prevented at runtime and they do little implicit type conversion, but these languages make no use of static type checking: the compiler does not check or enforce type constraint rules.
|
|
2024-08-29 07:08:42
|
> Standard ML, F#, OCaml, Haskell, Go and Rust are statically type-checked, but the compiler automatically infers a precise type for most values.
|
|
2024-08-29 07:12:11
|
OCaml’s type inference goes way beyond the usual “the variable’s type is whatever type the initial value happens to be”:
```console
$ ocaml
OCaml version 4.13.1
# let table = Hashtbl.create 0;; # create a hashtable of size zero and unspecified entry type (for now)
val table : ('_weak1, '_weak2) Hashtbl.t = <abstr>
# Hashtbl.add table 'a' 42;;
- : unit = ()
# table;; # now, it knows the exact type
- : (char, int) Hashtbl.t = <abstr>
```
|
|
2024-08-29 07:15:25
|
(it knows this because:
```console
# Hashtbl.add;;
- : ('a, 'b) Hashtbl.t -> 'a -> 'b -> unit = <fun>
```)
|
|
2024-08-29 07:17:02
|
the Rust compiler was originally written in OCaml, by the way
|
|
2024-08-29 07:17:10
|
(before, of course, being rewritten in Rust)
|
|
2024-08-29 07:17:46
|
they also list it here: https://doc.rust-lang.org/reference/influences.html
|
|
|
|
DNFrozen
|
2024-08-29 08:08:56
|
https://www.youtube.com/watch?v=BIm2ZCikI4Q
|
|
|
|
_wb_
|
2024-08-29 08:09:45
|
I used to have opinions about these things, back when I was doing my PhD on funky declarative (logical/functional) programming languages.
Now I basically don't really care much about it anymore. No amount of clever language design and static or dynamic analysis can prevent people from just implementing a subtly (or not so subtly) wrong thing, or a correct thing inefficiently. All Turing complete languages are just syntactic sugar for assembler.
|
|
|
|
DNFrozen
|
2024-08-29 08:13:37
|
my favorite is java.
generally fast enough for everything you usually need and runs on anything that you care about
|
|
|
yoochan
|
2024-08-29 08:14:42
|
except embedded stuffs like micro controllers
|
|
|
|
DNFrozen
|
2024-08-29 08:14:54
|
c is great too but someone should have just defined how long each variable is wiithout leaving that decision to the compiler
|
|
|
|
Orum
|
2024-08-29 08:15:07
|
I hate java
|
|
|
|
DNFrozen
|
|
|
_wb_
|
2024-08-29 08:16:19
|
You can use uint8_t etc if you want explicit width, otherwise using int and size_t etc kind of makes sense since it depends on architecture what is a good choice for those...
|
|
|
|
Orum
|
2024-08-29 08:16:44
|
Endless problems with security in the jvm, it's a headache to keep updated, and rather bloated to install. Any time I had to put it on a machine it felt like it was just a ticking time-bomb for problems.
|
|
|
|
DNFrozen
|
2024-08-29 08:17:22
|
ok fair point
|
|
|
lonjil
|
2024-08-29 09:20:58
|
Personally I think int never makes sense
|
|
2024-08-29 09:21:40
|
Your code certainly assumes that certain ranges of values will be representable by your variables
|
|
2024-08-29 09:22:08
|
So are you actually writing your code with `int`s which will work when `int` is only 16 bits?
|
|
|
_wb_
I used to have opinions about these things, back when I was doing my PhD on funky declarative (logical/functional) programming languages.
Now I basically don't really care much about it anymore. No amount of clever language design and static or dynamic analysis can prevent people from just implementing a subtly (or not so subtly) wrong thing, or a correct thing inefficiently. All Turing complete languages are just syntactic sugar for assembler.
|
|
2024-08-29 09:49:14
|
Perfect shouldn't be the enemy of good IMO. If clever language design cuts certain problems in half, that's a big win.
|
|
2024-08-29 09:51:57
|
For example, the Linux driver for Apple M1's GPU is written in Rust, and it has never ever had crashes in production, other than due to bugs in the rest of the DRM subsystem (which is written in C). Obviously it was written by a very competent developer, but so are most graphics drivers.
|
|
|
|
_wb_
|
2024-08-29 10:05:30
|
Yeah, I agree that better language design is better than poor language design. It's just not a magic solution that can make bugs impossible.
|
|
|
|
DNFrozen
|
2024-08-29 10:07:16
|
stuff always works until it suddenly doesn't (ask the crowdstrike guys)
|
|
|
Traneptora
|
2024-08-29 10:07:26
|
if I have a 480x600 jpeg reconstruction (4:2:0), then the only LF Group is going to be 60x75 iirc
|
|
2024-08-29 10:07:37
|
and the Cb and Cr channels should be 30x38, right?
|
|
2024-08-29 10:07:39
|
cause you round up
|
|
2024-08-29 10:08:20
|
which means I should have a total of 6780 LF Coefficients read, right? `60*75 + 2*30*38 = 6780`
|
|
2024-08-29 10:08:30
|
unless I'm missing something
|
|
|
|
CrushedAsian255
|
2024-08-30 01:59:41
|
I’m just thinking, isn’t JPEG XL’s responsive nature a match made in heaven with HTTP3’s multiplexing? The server can send the first 8kb or whatever of each image, before sending the rest
|
|
|
|
Demiurge
|
2024-08-30 05:24:42
|
Should a smart http server measure the offset that marks the end of the jxl toc and lqip, and send a partial file by default, waiting for the client to ask for the rest?
|
|
|
|
CrushedAsian255
|
2024-08-31 12:46:30
|
Maybe not wait for the client to ask for the rest, as some clients may not do that
|
|
|
|
yoochan
|
|
Demiurge
Should a smart http server measure the offset that marks the end of the jxl toc and lqip, and send a partial file by default, waiting for the client to ask for the rest?
|
|
2024-08-31 05:59:28
|
I tried to implement something very similar. A gallery where the browser request the rest only when you click on the thumbnail. I failed to finish it yet 😅
|
|
|
|
CrushedAsian255
|
|
yoochan
I tried to implement something very similar. A gallery where the browser request the rest only when you click on the thumbnail. I failed to finish it yet 😅
|
|
2024-08-31 06:23:31
|
what image format? JPEG XL? Progressive JPEG?
|
|
|
|
Demiurge
|
2024-08-31 07:05:12
|
For progressive jpeg there's no way to tell which scan to stop at since the first scan might be a completely wrong color or even no color
|
|
|
|
jonnyawsom3
|
|
CrushedAsian255
I’m just thinking, isn’t JPEG XL’s responsive nature a match made in heaven with HTTP3’s multiplexing? The server can send the first 8kb or whatever of each image, before sending the rest
|
|
2024-08-31 09:29:51
|
Another idea I just had while on a train
If the device is in data saving mode, only request up to a progressive scan/lqip depending on setting
|
|
|
|
CrushedAsian255
|
|
Another idea I just had while on a train
If the device is in data saving mode, only request up to a progressive scan/lqip depending on setting
|
|
2024-08-31 09:30:26
|
Only thing is how does the App know the sizes? Is this stored in the bitstream?
|
|
|
|
jonnyawsom3
|
2024-08-31 09:31:29
|
Well the app would need to query the data saving mode anyway, so it could store the scan length in metadata or whatever the dev prefers
|
|
|
|
CrushedAsian255
|
2024-08-31 09:32:02
|
is your idea to have the logic on the client or the server?
|
|
2024-08-31 09:32:11
|
if it's on the server, yes it would need to query the data saving
|
|
|
RaveSteel
|
|
Another idea I just had while on a train
If the device is in data saving mode, only request up to a progressive scan/lqip depending on setting
|
|
2024-08-31 09:32:17
|
Ngl, JavaScript is already able to read out too much device information for my taste. Now if the Browser was able to tell a Website, that would be different
|
|
|
|
CrushedAsian255
|
2024-08-31 09:32:19
|
but i was thinking this more to be a client-size decision
|
|
2024-08-31 09:33:30
|
is there anything in JPEG XL that stores truncation offsets like in FUIF?
From: <https://github.com/cloudinary/fuif>
> The header of a FUIF file contains a mandatory list of truncation offsets, which makes it easy to know how many bytes should be requested or served. The following offsets are included in the header:
|
|
|
|
jonnyawsom3
|
2024-08-31 09:33:32
|
Yeah, if the sever embedded the scan lengths upon upload/encode, then 'normal' clients could just view it normally, data saving ones could use those TOCs to stop the request early
|
|
|
|
CrushedAsian255
|
|
Yeah, if the sever embedded the scan lengths upon upload/encode, then 'normal' clients could just view it normally, data saving ones could use those TOCs to stop the request early
|
|
2024-08-31 09:33:49
|
like a new HTTP header?
|
|
|
|
jonnyawsom3
|
2024-08-31 09:35:37
|
This was only a rough idea while I'm wedged into the luggage rack on a train, but it seems promising to me for a very bored webdev to implement
|
|
|
|
RaveSteel
|
2024-08-31 09:36:36
|
Is it possible to propose ideas to the responsible parties for Standards like this?
|
|
|
|
jonnyawsom3
|
2024-08-31 09:37:36
|
I mean we already have partial requests, or could just cut the transfer when the client sees the bytes listed beforehand, although I have no doubt that would cause a few issues
|
|
|
|
_wb_
|
|
CrushedAsian255
is there anything in JPEG XL that stores truncation offsets like in FUIF?
From: <https://github.com/cloudinary/fuif>
> The header of a FUIF file contains a mandatory list of truncation offsets, which makes it easy to know how many bytes should be requested or served. The following offsets are included in the header:
|
|
2024-08-31 09:41:13
|
Yes, the TOC in the frame header can be used for this purpose.
|
|
|
|
Demiurge
|
2024-08-31 09:43:51
|
I wonder if it's legal to send 206 partial content and content-range header in response to a request that doesn't have the Range header
|
|
2024-08-31 09:44:09
|
I wonder if the client has to be the one to ask for a range in the first place and not the server
|
|
|
|
jonnyawsom3
|
2024-08-31 09:48:38
|
206 also triggers if the server fails to deliver the full request, and is just treated as "Partial Content"
|
|
|
|
Demiurge
|
2024-08-31 09:49:43
|
Good...
|
|
|
|
yoochan
|
|
CrushedAsian255
what image format? JPEG XL? Progressive JPEG?
|
|
2024-08-31 10:05:59
|
Jxl. I planned to pre fetch the minimal size from the header and store it in the filename like : image.912.jxl so the browser can request the first 912 bytes for preview
|
|
|
|
CrushedAsian255
|
|
_wb_
Yes, the TOC in the frame header can be used for this purpose.
|
|
2024-08-31 10:06:27
|
TOC is mandatory?
|
|
|
|
_wb_
|
2024-08-31 10:38:43
|
Yes. It is also what enables parallel decode.
|
|
|
|
jonnyawsom3
|
2024-08-31 10:39:27
|
Huh, for some reason I thought it was part of the container format
|
|
|
|
CrushedAsian255
|
|
Huh, for some reason I thought it was part of the container format
|
|
2024-08-31 10:41:13
|
same
|
|
|
|
jonnyawsom3
|
2024-08-31 10:46:52
|
> Each pass starts with a table of contents (sizes of each of their DC+AC groups), which enables parallel decoding and/or the decoding of a subset. However, there is no higher-level TOC of passes, as that would prevent appending additional images and could be too constraining for the encoder.
|
|
2024-08-31 10:47:11
|
So no TOC of passes, but there is one at the start of every pass
|
|
|
|
_wb_
|
2024-08-31 10:56:47
|
Passes are all in the same TOC, but it's only one frame at a time. So for multi-layer images you only get the base layer in the first TOC, and for animations only the first frame.
|
|
|
|
jonnyawsom3
|
2024-08-31 11:29:00
|
Ahh right
|
|
|
|
CrushedAsian255
|
2024-08-31 11:33:13
|
i should REALLY stop putting off reading the spec
|
|
|
|
jonnyawsom3
|
2024-08-31 11:48:33
|
There's also the Github format overview, that's where I got that except
|
|
|
|
CrushedAsian255
|
|
There's also the Github format overview, that's where I got that except
|
|
2024-08-31 11:49:07
|
there's also that, but it doesn't really go into that much detail
|
|
|
|
Demiurge
|
2024-09-01 12:50:44
|
Can animations have multiple layers in each step?
|
|
2024-09-01 12:51:11
|
Or a combination of animated and non-animated layers?
|
|
2024-09-01 12:51:26
|
Like a character moving over a static backdrop
|
|
|
|
_wb_
|
2024-09-01 02:03:51
|
Yes. You just mix zero duration frames with nonzero duration frames.
|
|
|
|
CrushedAsian255
|
2024-09-01 02:04:31
|
Can an animation frame reference another frame’s layers?
|
|
2024-09-01 02:04:47
|
Like say “layer 2 from previous frame”?
|
|
2024-09-01 02:05:01
|
Also is there a layer stack limit?
|
|
|
|
_wb_
|
|
CrushedAsian255
Can an animation frame reference another frame’s layers?
|
|
2024-09-01 02:05:34
|
Yes
|
|
|
CrushedAsian255
Also is there a layer stack limit?
|
|
2024-09-01 02:06:41
|
You have up to 4 "slots" available to save decoded frames/layers. These can then be referenced for patches or frame blending.
|
|
|
|
CrushedAsian255
|
|
_wb_
Yes
|
|
2024-09-01 02:09:53
|
So if I want to just copy a previous frame’s layer, then save it into a slot and then set a path that is the entire frame in size
|
|
|
|
_wb_
|
2024-09-01 02:17:35
|
Yes, that would work.
|
|
|
Eugene Vert
|
2024-09-01 03:07:06
|
Are animation timecodes just an addition to frame durations or a replacement for them?
|
|
|
|
_wb_
|
2024-09-01 03:26:54
|
Frame durations are expressed in ticks; the image header defines how long a tick lasts (as a rational number). For non-animations, the tick duration is not explicitly defined and frame durations are implicitly zero (so you can only have layers).
|
|
2024-09-01 03:31:37
|
timecodes are an optional additional thing that can be signalled for every frame. These codes are SMPTE timecodes: it's a uint32 which is interpreted as four uint8 numbers, for hours, minutes, seconds and frame (so I guess you can have at most 256 frames per seconds with that encoding)
|
|
2024-09-01 03:32:32
|
they can be ignored though — only the duration has normative implications for how to play the animation.
|
|
|
|
CrushedAsian255
|
2024-09-01 03:32:51
|
So the time codes are just metadata?
|
|
|
|
_wb_
|
2024-09-01 03:42:53
|
Yes, only constraint is they have to increase when the duration is nonzero.
|
|
|
|
CrushedAsian255
|
2024-09-02 11:01:58
|
it's currently considered general consensus that AVIF beats JXL at really low bitrates, but is there a definite source on this? I want to add to Wikipedia
|
|
|
MidAutumnMoon
|
2024-09-02 11:07:07
|
I was recently using cjxl to losslessly encode some PNGs (3000x3000-ish). Encoding with effort 10 took half an hour, whereas with effort 9 only took half a minute.
|
|
2024-09-02 11:07:15
|
Is this the norm?
|
|
2024-09-02 11:07:42
|
The command I was using
```bash
command cjxl \
--allow_expert_options \
--effort 9 \
--lossless_jpeg 1 \
--distance 0.0 \
--brotli_effort 11 \
--premultiply 1 \
--modular 1 \
--iterations 100 \
--modular_nb_prev_channels 3 \
--num_threads -1 \
$input $output
```
|
|
|
|
_wb_
|
2024-09-02 11:45:52
|
that's a bigger gap than expected, but it's not impossible. Effort 10 is basically single-threaded and uses substantially more memory, so it can be a lot slower than effort 9
|
|
|
|
CrushedAsian255
|
2024-09-02 11:49:20
|
especially if the image is so big that it starts paging
|
|
2024-09-02 11:49:23
|
that will KILL performance
|
|
|
|
MidAutumnMoon
|
|
_wb_
that's a bigger gap than expected, but it's not impossible. Effort 10 is basically single-threaded and uses substantially more memory, so it can be a lot slower than effort 9
|
|
2024-09-02 11:55:38
|
Yeah, that's what I've observed. Effort 10 uses only one core.
|
|
2024-09-02 11:55:49
|
Half an hour for 30kb saving lol
|
|
|
|
CrushedAsian255
|
2024-09-02 11:57:54
|
if you really want to ~~waste your time~~ achieve best compression ratio, use `--allow_expert_options -e 11`
|
|
|
|
Oleksii Matiash
|
|
MidAutumnMoon
Half an hour for 30kb saving lol
|
|
2024-09-02 11:58:04
|
New meaning for "launch and forget"
|
|
|
|
MidAutumnMoon
|
|
CrushedAsian255
if you really want to ~~waste your time~~ achieve best compression ratio, use `--allow_expert_options -e 11`
|
|
2024-09-02 11:58:47
|
Tried that, -e 11 ate 30GB of memory and I killed it
|
|
|
|
CrushedAsian255
|
2024-09-02 11:58:52
|
lol
|
|
2024-09-02 11:59:21
|
just give it your entire HDD as swap, it's worth it for 19 kB saving, right?
|
|
|
|
MidAutumnMoon
|
2024-09-02 12:00:20
|
That's seconds of saving on load time under bad 3G
|
|
|
|
CrushedAsian255
|
2024-09-02 12:02:26
|
so you can send 4k images to people on dial up 2.7 seconds faster
|
|
|
|
jonnyawsom3
|
|
CrushedAsian255
if you really want to ~~waste your time~~ achieve best compression ratio, use `--allow_expert_options -e 11`
|
|
2024-09-02 12:02:32
|
I was going to tell them to remove --allow_expert_options to avoid accidentally using it
|
|
|
MidAutumnMoon
Tried that, -e 11 ate 30GB of memory and I killed it
|
|
2024-09-02 12:05:58
|
effort 11 essentially runs an effort 10 encode on every thread, running through a list of options and saving the best one
|
|
|
|
MidAutumnMoon
|
|
effort 11 essentially runs an effort 10 encode on every thread, running through a list of options and saving the best one
|
|
2024-09-02 12:07:57
|
I see
|
|
|
|
_wb_
|
2024-09-02 12:14:11
|
I wouldn't use anything slower than default effort tbh, except if you really want to try to squeeze out as much as possible from the current encoder implementation.
|
|
|
|
CrushedAsian255
|
|
_wb_
I wouldn't use anything slower than default effort tbh, except if you really want to try to squeeze out as much as possible from the current encoder implementation.
|
|
2024-09-02 12:15:51
|
i use `-e 8` for lossy, `-e 7 -I 100` for lossless and `-e 8` for JPEG trancoding
|
|
2024-09-02 12:16:02
|
sometime `-e 6` for lossless if im in a rush
|
|
2024-09-02 12:16:09
|
as with lossless you can always retry later
|
|
|
|
jonnyawsom3
|
2024-09-02 12:16:50
|
I generally see a big bump at 7 on lossless, so if I'm going less it might as well be 4 for the speed
|
|
|
|
TheBigBadBoy - 𝙸𝚛
|
2024-09-02 12:21:07
|
I always use `-e 10` <:CatSmile:805382488293244929>
|
|
|
|
CrushedAsian255
|
|
TheBigBadBoy - 𝙸𝚛
I always use `-e 10` <:CatSmile:805382488293244929>
|
|
2024-09-02 12:23:46
|
do you do parallel encodes?
|
|
|
|
TheBigBadBoy - 𝙸𝚛
|
2024-09-02 12:24:57
|
on my laptop yeah
not my phone, not enough RAM <:KekDog:805390049033191445>
|
|
|
|
Oleksii Matiash
|
|
CrushedAsian255
do you do parallel encodes?
|
|
2024-09-02 12:25:02
|
For e 10 it could lead to terrible RAM consumption
|
|
|
|
CrushedAsian255
|
2024-09-02 12:26:04
|
36gb ram [PepeHackerman](https://cdn.discordapp.com/emojis/806632585429909514.gif?size=512&quality=lossless&name=PepeHackerman)
|
|
|
|
Oleksii Matiash
|
2024-09-02 12:27:46
|
I have 64, but for large images e 10 easily asks for 40+, so I'm stopping it before it starts swapping
|
|
|
|
CrushedAsian255
|
2024-09-02 12:28:47
|
how big are these images?
|
|
2024-09-02 12:29:07
|
i guess whatever apple was saying about 8gb being equal to 16gb must be doing something to help
|
|
|
|
TheBigBadBoy - 𝙸𝚛
|
2024-09-02 12:30:56
|
I wouldn't trust anything Apple says <:KekDog:805390049033191445>
|
|
|
|
Oleksii Matiash
|
|
CrushedAsian255
how big are these images?
|
|
2024-09-02 12:31:22
|
I once tried to -e 10 photos from 80 MP camera.. 🥲 Also I'm on Windows, so no Apple magic 😅
|
|
|
|
CrushedAsian255
|
2024-09-02 12:33:16
|
what type of image?
|
|
2024-09-02 12:33:20
|
lossless jpeg transcode?
|
|
2024-09-02 12:33:23
|
or is this pixels to jxl
|
|
|
|
Oleksii Matiash
|
2024-09-02 12:33:29
|
No, native jxl
|
|
2024-09-02 12:34:03
|
Jpeg transcode is ok, -e 10 uses less than 10 GB
|
|
|
|
CrushedAsian255
|
2024-09-02 12:34:29
|
> less than 10 GB
And that’s a small amount of RAM??
|
|
|
|
Oleksii Matiash
|
2024-09-02 12:34:58
|
It is usable on my system to use it 4-threaded, for me it is fully enough
|
|
|
|
jonnyawsom3
|
|
Oleksii Matiash
No, native jxl
|
|
2024-09-02 12:51:13
|
Encoding from camera RAWs?
|
|
|
|
Oleksii Matiash
|
|
Encoding from camera RAWs?
|
|
2024-09-02 12:51:35
|
From 16 bpc tiffs, of course
|
|
|
|
CrushedAsian255
|
2024-09-02 12:52:00
|
I was sent a bunch of JPEGS but they used such a high quality they’re bigger than the corresponding DNG somehow, like wtf did they do?
|
|
|
w
|
2024-09-02 12:52:04
|
i thought cjxl cant do tiff
|
|
|
|
CrushedAsian255
|
2024-09-02 12:52:09
|
It can’t
|
|
|
|
w
|
2024-09-02 12:52:28
|
i also gave up trying to jxl photos
|
|
|
|
Oleksii Matiash
|
|
CrushedAsian255
I was sent a bunch of JPEGS but they used such a high quality they’re bigger than the corresponding DNG somehow, like wtf did they do?
|
|
2024-09-02 12:53:02
|
Raws are 1-channel-per-pixel
|
|
2024-09-02 12:54:00
|
That's why uncompressed dngs from my camera are 160 MB, and 16 bpc tiffs - 480 🤷♂️
|
|
|
|
CrushedAsian255
|
2024-09-02 12:54:22
|
They must have use quality 100
|
|
|
|
Oleksii Matiash
|
|
w
i thought cjxl cant do tiff
|
|
2024-09-02 12:55:57
|
Well, to be precise - from pngs. My raw converter can export only jpeg or tiff, that's why I'm thinking about it as of tiffs
|
|
|
monad
|
|
MidAutumnMoon
I was recently using cjxl to losslessly encode some PNGs (3000x3000-ish). Encoding with effort 10 took half an hour, whereas with effort 9 only took half a minute.
|
|
2024-09-02 03:47:50
|
If you are working with images of photo-like complexity (including 3d renders, digital paintings for example), you can get e9 density in 1/3 the time with d0e7E4g2.
|
|
|
Kampidh
|
2024-09-02 09:26:29
|
trying to encode multilayered animated jxl, along with reference frame for the bg
though if it's transcoded with cjxl will coalesce the layers.. =p
|
|
|
|
RaveSteel
|
|
Kampidh
trying to encode multilayered animated jxl, along with reference frame for the bg
though if it's transcoded with cjxl will coalesce the layers.. =p
|
|
2024-09-02 09:29:15
|
Which program did you use to encode this?
|
|
|
|
Kampidh
|
|
RaveSteel
Which program did you use to encode this?
|
|
2024-09-02 09:32:50
|
self written, just for playing around with libjxl API
|
|
|
|
RaveSteel
|
2024-09-02 09:33:59
|
Nice. The input has to be PNGs?
|
|
|
|
Kampidh
|
2024-09-02 09:36:28
|
it is (for now), while technically it can be whatever QImage can open
|
|
|
|
RaveSteel
|
2024-09-02 09:37:26
|
Are you planning on putting it on GitHub?
|
|
|
|
Kampidh
|
2024-09-02 09:41:09
|
not yet though, since it's a quick project and it's still a whole block of unstable mess xD
|
|
|
|
RaveSteel
|
2024-09-02 09:41:58
|
Fair point
|
|
|
|
DNFrozen
|
2024-09-02 09:55:47
|
and does it support effort 11?
|
|
2024-09-02 10:46:00
|
🤷♂️
|
|
|
|
jonnyawsom3
|
2024-09-02 10:47:15
|
Huh, for some reason a 5800 surprises me on a system with 128 GB of RAM
|
|
|
|
DNFrozen
|
2024-09-02 10:48:09
|
the ram prices were the reason i choose am4 and not am5
|
|
|
|
RaveSteel
|
2024-09-02 10:48:38
|
ddr5 really was expensive for a year or two
|
|
2024-09-02 10:49:06
|
And then the boot times were atrocious if you enabled EXPO
|
|
2024-09-02 10:49:21
|
I had at times boot times of 2-3 minutes
|
|
|
|
DNFrozen
|
2024-09-02 10:49:27
|
i build the pc in december 2023 (my pc wasa quite old and was still on win 8.1)
|
|
|
|
jonnyawsom3
|
2024-09-02 10:50:11
|
I'm sat on a ryzen 1700 and a dying 1070TI, along with 16 of RAM that constantly goes to swapfile
|
|
2024-09-02 10:51:13
|
Issue is figuring out what to replace first, although it's looking like the GPU since I can't use the computer at all if I have no output
|
|
|
|
RaveSteel
|
2024-09-02 10:51:32
|
In what way is the GPU dying?
|
|
|
|
jonnyawsom3
|
2024-09-02 10:52:42
|
I mentioned it before but the fan bearing makes it vibrate the computer apart whenever the speed changes. Disconnected my SSD while in use and seems to be a flaw in the design since replacement fans just broke in the same way
|
|
|
|
DNFrozen
|
2024-09-02 10:52:50
|
here the pic that sums up my old pc
|
|
2024-09-02 10:52:51
|
https://cdn.discordapp.com/attachments/1178674227159773315/1178674285309591693/IMG_20231127_132630.jpg?ex=66d70178&is=66d5aff8&hm=10a84e0c698e8ef349a3bbe6d23418125d0c56b064217f1a44ceaa1b12ef08a1&
|
|
|
|
RaveSteel
|
|
I mentioned it before but the fan bearing makes it vibrate the computer apart whenever the speed changes. Disconnected my SSD while in use and seems to be a flaw in the design since replacement fans just broke in the same way
|
|
2024-09-02 10:53:03
|
Ah yes, I remember
|
|
|
|
jonnyawsom3
|
2024-09-02 10:53:41
|
I actually met a friend on Satuday who offered me their spare GPU... Turns out it's the exact same model and with the same issues...
|
|
|
DNFrozen
🤷♂️
|
|
2024-09-02 10:54:47
|
I would like to find out if the extra cache makes much of a difference in cjxl, but need someone with a normal 5800 to test against
|
|
|
|
DNFrozen
|
2024-09-02 10:54:57
|
in my current pc i reused the PC case from my first pc and the old radiator/pump
|
|
2024-09-02 10:54:59
|
https://cdn.discordapp.com/attachments/914990461825990656/1183814206768824401/IMG_20231211_175312.jpg?ex=66d73f65&is=66d5ede5&hm=95e31a0d564ae84acc254b60cd21e13a84fae9f3e25714df97297af0cd5903cd&
|
|
|
|
RaveSteel
|
2024-09-02 10:55:17
|
What an abomination 😂
|
|
|
|
jonnyawsom3
|
2024-09-02 10:55:35
|
It's beautiful
|
|
|
|
DNFrozen
|
2024-09-02 10:57:06
|
i have no idea if the cache makes a difference for jxl conversion
|
|
|
|
RaveSteel
|
2024-09-02 10:58:14
|
Have you checked whether the 3d vcache impacts productivity, like blender etc?
|
|
|
|
DNFrozen
|
2024-09-02 10:58:30
|
I've done some undervolting and set the frequency limit to 3.5GHZ to make sure it doesn't boost
now it max out at 50W power consumption
|
|
|
|
RaveSteel
|
2024-09-02 10:58:33
|
if yes, the chances are good that it positively affects encoding as well
|
|
|
|
DNFrozen
|
2024-09-02 10:59:04
|
and 30W in idle
|
|
2024-09-02 11:01:48
|
the .jxl files are effort 9 and the .e11 are effort 11
|
|
|
|
jonnyawsom3
|
|
RaveSteel
Have you checked whether the 3d vcache impacts productivity, like blender etc?
|
|
2024-09-02 11:04:03
|
The x3d chips have lower clocks than the normal ones, which generally lowers productivity tasks by a percent or two, but when something can use the cache the gains can be huge. First example that comes to mind is VRChat, which has a 600% uplift because the game almost always runs out of VRAM and has to fetch the textures from RAM or disk (Best guess is a faster lookup for where the data is being stored as opposed to the time spent actually fetching it)
|
|
|
|
DNFrozen
|
2024-09-02 11:04:28
|
for small thumbnais like 300*300 the saved space is still quite good and it only take like 90-120 seconds
|
|
|
|
RaveSteel
|
|
The x3d chips have lower clocks than the normal ones, which generally lowers productivity tasks by a percent or two, but when something can use the cache the gains can be huge. First example that comes to mind is VRChat, which has a 600% uplift because the game almost always runs out of VRAM and has to fetch the textures from RAM or disk (Best guess is a faster lookup for where the data is being stored as opposed to the time spent actually fetching it)
|
|
2024-09-02 11:05:24
|
Yes, games can profit massively from it, which is why i specified productivy. I seem to remeber that the diff for productivity wasn't to large, but it's been a long while since I read the review for the 5800x3d
|
|
2024-09-02 11:05:50
|
so no idea, basically
|
|
|
|
jonnyawsom3
|
2024-09-02 11:06:06
|
Compression is generally very memory intensive, so I'd assume some kind of uplift
|
|
|
|
DNFrozen
|
2024-09-02 11:06:38
|
yeah the x3d stuff has no advantage in 7zip
|
|
2024-09-02 11:07:27
|
it only works if everything fits in the cache
|
|
2024-09-02 11:08:58
|
I'm glad i don't have one of these degrading intel CPU's
|
|
|
|
RaveSteel
|
|
|
Orum
|
|
Compression is generally very memory intensive, so I'd assume some kind of uplift
|
|
2024-09-02 11:46:09
|
depends on the method and often settings too
|
|
|
|
CrushedAsian255
|
2024-09-02 11:46:59
|
also depends heavily if streaming encode is possible
|
|
|
|
monad
|
|
DNFrozen
for small thumbnais like 300*300 the saved space is still quite good and it only take like 90-120 seconds
|
|
2024-09-03 01:49:18
|
try git head
|
|
|
|
MidAutumnMoon
|
|
monad
If you are working with images of photo-like complexity (including 3d renders, digital paintings for example), you can get e9 density in 1/3 the time with d0e7E4g2.
|
|
2024-09-03 03:06:23
|
wow thanks. this combination works wonderfully
|
|
2024-09-03 03:06:34
|
I'm using these parameters from now on
|
|
|
|
DNFrozen
|
|
monad
try git head
|
|
2024-09-03 06:30:37
|
what is that?
|
|
|
|
monad
|
2024-09-03 06:41:31
|
most recent unreleased version on github
|
|
|
|
username
|
2024-09-03 06:42:13
|
https://artifacts.lucaversari.it/libjxl/libjxl/latest/
|
|
|
|
DNFrozen
|
2024-09-03 06:44:26
|
are there any changes to lossless jpg recompression or when converting png to jxl with effort 11?
and whats the difference between the static and non-static windows versions?
|
|
|
|
Oleksii Matiash
|
|
RaveSteel
I had at times boot times of 2-3 minutes
|
|
2024-09-03 06:44:50
|
2-3 minutes seems to be too much. My pc performs that RAM dark magic in 45 seconds, but honestly I don't care at all. I turn on it when I wake up, and go to morning routines, so it has plenty of time to wake up too. And also this magic can be turned off, like it is set on my sister's pc with 7800X3D
|
|
|
|
monad
|
|
DNFrozen
are there any changes to lossless jpg recompression or when converting png to jxl with effort 11?
and whats the difference between the static and non-static windows versions?
|
|
2024-09-03 06:52:45
|
e11 is like 0.1% denser at 6% the compute (less practical speed gain as cores increase)
|
|
|
|
DNFrozen
|
2024-09-03 07:16:03
|
seems like dark magic but I'll take it
|
|
|
|
RaveSteel
|
|
Oleksii Matiash
2-3 minutes seems to be too much. My pc performs that RAM dark magic in 45 seconds, but honestly I don't care at all. I turn on it when I wake up, and go to morning routines, so it has plenty of time to wake up too. And also this magic can be turned off, like it is set on my sister's pc with 7800X3D
|
|
2024-09-03 07:49:07
|
±2 minutes was normal when AM5 released, it improved a ton since then due to firmware updates, but it really was that bad. You will find many people complaining about boot times in older posts
|
|
|
|
Oleksii Matiash
|
|
RaveSteel
±2 minutes was normal when AM5 released, it improved a ton since then due to firmware updates, but it really was that bad. You will find many people complaining about boot times in older posts
|
|
2024-09-03 07:52:02
|
I know, I upgraded my pc soon after AM5 launch. And I'm too not excited about this lag, but a) it can be disabled b) performance bonuses of AM5 outweights this for me
|
|
|
|
DNFrozen
|
2024-09-03 07:55:05
|
well never trust code from chatgpt 🙈
i asked chatgpt to modify my code so that the cjxl processes would start in a lower priority so that it would affect my PC too much
the changed code did that.... but instead of waiting for process completion it was now trying to convert all images at once so that my only option was hitting the reset button
|
|
|
|
CrushedAsian255
|
2024-09-03 07:55:43
|
oops
|
|
2024-09-03 07:55:50
|
next time at least read what it outputs
|
|
|
|
DNFrozen
|
2024-09-03 07:57:09
|
yeah lesson learned
|
|
|
|
CrushedAsian255
|
2024-09-03 07:57:47
|
what if it accidentally got inputs swapped and deleted your images
|
|
2024-09-03 07:57:59
|
please say you at least ran it on a copy of your images
|
|
|
|
DNFrozen
|
2024-09-03 07:59:15
|
yup it's a copy
|
|
|
|
CrushedAsian255
|
2024-09-03 08:00:22
|
good
|
|
2024-09-03 08:00:38
|
also make sure to verify the JXL outputs in case there is some weird edge-case bug
|
|
|
|
DNFrozen
|
2024-09-03 08:01:27
|
i don't know a good way to verify them
|
|
|
|
CrushedAsian255
|
2024-09-03 08:01:36
|
are these JPEG or PNG source
|
|
2024-09-03 08:01:57
|
if they're JPEG to JXL, then convert all the JXLs back to JPEG and diff them or check xxHash
|
|
2024-09-03 08:02:02
|
(or md5 or whatever)
|
|
2024-09-03 08:02:32
|
if they're PNG, use `compare -metric ae "in.png" "transcoded.jxl"`. it should give 0 if using lossless
|
|
2024-09-03 08:02:36
|
if using lossy, i don't know
|
|
|
|
DNFrozen
|
2024-09-03 08:04:03
|
i'm convertiing jpeg and png files in seperate folders with different parameters
|
|
2024-09-03 08:04:16
|
so i could check them like that
|
|
|
|
CrushedAsian255
|
2024-09-03 08:05:25
|
just make sure to
|
|
2024-09-03 08:05:34
|
and if there's a discrepency file an issue
|
|
|
|
DNFrozen
|
2024-09-03 08:06:50
|
I'll check them when all files are converted (will probably still take weeks for the pngs)
|
|
|
|
Quackdoc
|
2024-09-04 09:31:05
|
I think I asked this before, but can JXL / cjxl differentiate between sRGB transfer and a pure gamma 2.2 transfer?
|
|
|
|
_wb_
|
|
|
Quackdoc
|
2024-09-04 09:59:03
|
how can we specify that in jxl? I know it's something like ` -x color_space=RGB_D65_SRG_Per_SRG` would it just be ` -x color_space=RGB_D65_SRG_Per_G22`?
|
|
2024-09-04 09:59:14
|
actually, is there a table or something of options?
|
|
|
|
monad
|
|
Quackdoc
actually, is there a table or something of options?
|
|
2024-09-04 10:50:17
|
<https://github.com/libjxl/libjxl/blob/main/lib/extras/dec/color_description.cc>?
|
|
|
|
Quackdoc
|
2024-09-04 10:57:27
|
hmmm, a little confused on how to invoke this still, I think it's supposed to be `-x color_space=RGB_D65_SRG_Per_g2.2` ?
|
|
2024-09-04 11:09:41
|
doesn't seem to have worked
|
|
2024-09-04 11:14:05
|
oh im an idiot never mind
|
|
2024-09-04 11:17:19
|
ok so I ought to be using `-x color_space=RGB_D65_SRG_Per_g0.45454545454` not the most ergonomic thing in the world, but usable
|
|
|
|
Traneptora
|
|
Quackdoc
ok so I ought to be using `-x color_space=RGB_D65_SRG_Per_g0.45454545454` not the most ergonomic thing in the world, but usable
|
|
2024-09-05 10:19:14
|
my question is why do you want gamma45 instead of sRGB
|
|
2024-09-05 10:19:26
|
usually you want actual sRGB
|
|
2024-09-05 10:21:46
|
<@245794734788837387> https://discord.com/channels/794206087879852103/848189884614705192/1280160533924024340
Feel free to do it
|
|
|
|
Quackdoc
|
|
Traneptora
my question is why do you want gamma45 instead of sRGB
|
|
2024-09-05 10:26:29
|
A lot of content is mistakenly actually graded on/for gamma 2.2 and improperly tagged. so overriding the tags is a better time to just set the right tags instead of undo the gamma 2.2 and reapply the sRGB transfer.
also it's a lot of times simply better to properly grade for 2.2 so long as it's tagged right, People with properly color managed setups will usually can display it fine, and when it comes to actually grading, the simple pure transfer I found is better for properly getting a nice consistent luminance gradient as within the typical SDR ranges, sRGB can look a little odd when not on CRT
|
|
|
|
Traneptora
|
|
Quackdoc
A lot of content is mistakenly actually graded on/for gamma 2.2 and improperly tagged. so overriding the tags is a better time to just set the right tags instead of undo the gamma 2.2 and reapply the sRGB transfer.
also it's a lot of times simply better to properly grade for 2.2 so long as it's tagged right, People with properly color managed setups will usually can display it fine, and when it comes to actually grading, the simple pure transfer I found is better for properly getting a nice consistent luminance gradient as within the typical SDR ranges, sRGB can look a little odd when not on CRT
|
|
2024-09-05 10:27:41
|
if it's lossy there's no particular reason to decode to gamma22 because XYB is linear already
|
|
|
|
CrushedAsian255
|
2024-09-05 10:27:42
|
True sRGB is that one with the slightly weird transfer right
|
|
2024-09-05 10:27:43
|
?
|
|
|
|
Traneptora
|
2024-09-05 10:28:07
|
if you request from an XYB image you're basically asking libjxl to apply the forward TRC function
|
|
|
|
Quackdoc
|
|
Traneptora
if it's lossy there's no particular reason to decode to gamma22 because XYB is linear already
|
|
2024-09-05 10:28:55
|
I'm talking setting it on input not on output.
|
|
2024-09-05 10:30:24
|
the content in the end still has the transfer function applied to the image when it's sent to the image. so that needs to be dealt with, it could be nice to just linearize the image first, but im not sure how the optimal workflow for a simple tool would handle that
|
|
|
CrushedAsian255
True sRGB is that one with the slightly weird transfer right
|
|
2024-09-05 10:34:34
|
yeah, it's close to a gamma 2.2 but not quite, as it was designed for CRTs, and needed some additional handling, in the end sRGB was designed for ~80nits
|
|
|
|
Traneptora
|
|
Quackdoc
I'm talking setting it on input not on output.
|
|
2024-09-06 02:54:31
|
yea, but if it's on input it'll just use the tagged transfer of the PNG, right?
|
|
2024-09-06 02:55:53
|
You can always overwrite the tag on the PNG using `umbrielpng` (the png filtering program I wrote)
|
|
2024-09-06 02:57:05
|
```
umbrielpng --fix-in-place --cicp-prim=bt709 --cicp-trc=gamma22 input.png
```
|
|
2024-09-06 02:57:38
|
gamma2.2 is 4, by cICP or H.273
|
|
2024-09-06 02:57:59
|
if the input is actually gamma22 and not sRGB that is
|
|
2024-09-06 02:58:16
|
if it's actually sRGB you don't want to do this cause it's just going to linearize it anyway when it converts it to XYB
|
|
|
|
Quackdoc
|
|
Traneptora
yea, but if it's on input it'll just use the tagged transfer of the PNG, right?
|
|
2024-09-06 02:59:45
|
I'm not sure, I didn't test. I was hoping I could do it directly with cjxl. I am aware of cicp and stuff
|
|
|
|
Traneptora
|
2024-09-06 03:00:02
|
I know `cjxl` reads the PNG's `cICP` tag
|
|
|
|
Quackdoc
|
|
Traneptora
if it's actually sRGB you don't want to do this cause it's just going to linearize it anyway when it converts it to XYB
|
|
2024-09-06 03:00:31
|
this is explicitly the reason why i want to do this, as applying the inverse sRGB transform on content encoded/designed with a pure 2.2 transfer can really bung things up (and ofc vice versa, looking at you KDE)
|
|
|
|
Traneptora
|
|
Quackdoc
this is explicitly the reason why i want to do this, as applying the inverse sRGB transform on content encoded/designed with a pure 2.2 transfer can really bung things up (and ofc vice versa, looking at you KDE)
|
|
2024-09-06 03:06:06
|
I see, in that case I'd overwrite the tagged content on the PNG
|
|
2024-09-06 03:06:15
|
using umbrielpng or a similar tool
|
|
|
|
Quackdoc
|
2024-09-06 03:08:02
|
yeah, my goal was for this to be a feature in a conversion tool the mockup here.
|
|
|
|
Traneptora
|
2024-09-06 03:08:27
|
if you're doing a GUI, then you can link directly to libjxl
|
|
2024-09-06 03:08:35
|
and libjxl lets you tell it what the input is tagged as
|
|
2024-09-06 03:08:43
|
since libjxl itself doesn't handle PNG reading
|
|
|
|
Quackdoc
|
2024-09-06 03:09:23
|
that's possible, but I was just going to run this as a command runner [av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&quality=lossless&name=av1_dogelol)
|
|
2024-09-06 03:09:49
|
not like it matters to much, I plan on making multiple backends for this since it's more about testing the slint ui framework then it is about making a useful tool
|
|
|
|
CrushedAsian255
|
2024-09-06 04:31:32
|
just wondering, what happened to FLIF/FUIF/Pik? Are they all discontinued?
|
|
|
|
Orum
|
2024-09-06 04:33:40
|
they exist but there's little interest in them
|
|
2024-09-06 04:33:53
|
FLIF/FUIK/Pik more or less live on in JXL though
|
|
|
|
Traneptora
|
2024-09-06 04:33:58
|
I believe they are "discontinued" in the sense that JXL supercedes all of them
|
|
|
|
CrushedAsian255
|
2024-09-06 04:34:07
|
i found a flif spec
https://flif.info/spec.html
|
|
|
|
Traneptora
|
2024-09-06 04:34:36
|
yea, FLIF was Jon's pet project that became FUIF, which is basically FLIF but with Squeeze and Lossy
|
|
2024-09-06 04:34:51
|
FUIF essentially became Modular while Pik became VarDCT
|
|
|
|
CrushedAsian255
|
|
Traneptora
I believe they are "discontinued" in the sense that JXL supercedes all of them
|
|
2024-09-06 04:34:54
|
"discontinued" so no one will try to get them adopted for anything?
|
|
|
|
Traneptora
|
2024-09-06 04:35:12
|
there's no particular reason to use any of them since all of the technologies invented for them are in JXL except the ones that were intentionally left out
|
|
2024-09-06 04:35:15
|
JXL is the best of it all
|
|
2024-09-06 04:35:36
|
basically, the JPEG committee called for proposals to make the next JPEG format
|
|
2024-09-06 04:35:47
|
Cloudinary submitted FUIF and Google submitted Pik
|
|
2024-09-06 04:35:52
|
both were combined to become JXL
|
|
|
|
CrushedAsian255
|
|
Traneptora
JXL is the best of it all
|
|
2024-09-06 04:35:55
|
so theres no point using them?
|
|
|
|
Orum
|
2024-09-06 04:36:27
|
it's like asking "why not use jpeg?"
|
|
2024-09-06 04:36:38
|
like, sure, it still exists, but it's old
|
|
2024-09-06 04:36:59
|
but at least jpeg has wide support, while the same can't be said for FLIF/FUIF/Pik
|
|
|
|
Traneptora
|
|
CrushedAsian255
so theres no point using them?
|
|
2024-09-06 04:37:25
|
pretty much
|
|
|
|
CrushedAsian255
|
2024-09-06 04:37:26
|
so it's like why use MP2 when MP3 exists?
|
|
|
|
Orum
|
2024-09-06 04:37:43
|
or mp3 or vorbis when opus exists
|
|
|
|
Traneptora
|
2024-09-06 04:37:53
|
well no, mp3 has more widerange support on legacy devices
|
|
2024-09-06 04:37:57
|
but FLIF never got that
|
|
|
|
CrushedAsian255
|
2024-09-06 04:37:57
|
you can use the support argument for mp3
|
|
2024-09-06 04:38:07
|
so flif was really just a prototype for jxl
|
|
2024-09-06 04:38:08
|
?
|
|
|
|
Orum
|
2024-09-06 04:38:12
|
yeah, basically
|
|
2024-09-06 04:38:26
|
or at least for part of JXL
|
|
|
|
Traneptora
|
|
CrushedAsian255
so flif was really just a prototype for jxl
|
|
2024-09-06 04:43:28
|
basically yea
|
|
|
|
Tirr
|
2024-09-06 04:57:03
|
I remember the moment when FLIF came out, I thought it's cool, especially meta-adaptiveness and progressiveness
|
|
2024-09-06 04:57:29
|
now all of them live in JXL, with reasonable computational cost
|
|
|
|
CrushedAsian255
|
|
Tirr
I remember the moment when FLIF came out, I thought it's cool, especially meta-adaptiveness and progressiveness
|
|
2024-09-06 05:08:01
|
are MA trees MANIAC v2?
|
|
|
|
_wb_
|
2024-09-06 05:34:04
|
Yep
|
|
2024-09-06 05:36:18
|
I guess MA trees + hybriduint ANS together is MANIAC v2
|
|
|
|
jonnyawsom3
|
|
CrushedAsian255
so theres no point using them?
|
|
2024-09-06 05:36:32
|
Well, if you have extremely specific use cases then FLIF/FUIF can be best for progressive lossless in terms of filesize currently, which bloats JXL
|
|
|
|
_wb_
|
2024-09-06 05:39:27
|
You'll always have very specific images where even GIF is best, or PNG. For FLIF and lossless WebP there will be more examples. In lossless compression it's all about overall performance on a representative corpus.
|
|
|
|
CrushedAsian255
|
2024-09-06 05:40:22
|
Cause technically if you want you can make a seperate format for each image you want to store that stores exact image as an empty file otherwise works exactly like PPM
|
|
2024-09-06 05:40:39
|
I know that is a stretch but still
|
|
|
|
jonnyawsom3
|
2024-09-06 05:41:55
|
The evolution of Stalin sort, Stalin storage.
If the image doesn't match, just output nothing
|
|
|
|
CrushedAsian255
|
2024-09-06 05:43:10
|
New format: .stalin
Only valid bitstream is an empty file, who decodes to an image of Joseph Stalin. It does not matter which image.
All other bitstream are invalid and will be sent to the gulag
|
|
|
|
_wb_
|
2024-09-06 05:45:38
|
The progression in FLIF came from its interlaced sample visiting order, which is nice but it's not great for memory locality, and it effectively only allows nearest neighbor subsampled previews. E.g. a checkerboard pattern at the pixel level (alternating black and white pixels) will look completely black in all the previews.
Squeeze is better and produces a gray preview for a checkerboard pattern, but it comes at the cost of introducing new colors which can harm compression, especially for nonphoto images.
|
|
|
|
CrushedAsian255
|
2024-09-06 05:46:46
|
So FLIF is like only sending the left audio channel where Squeeze is like sending a mono downmix?
|
|
|
|
_wb_
|
2024-09-06 05:47:24
|
Yeah basically
|
|
|
|
CrushedAsian255
|
2024-09-06 05:47:28
|
It works well for stereo but will break if you have completely different data on left and right channel (like 2 mono tracks in different language)
|
|
|
_wb_
The progression in FLIF came from its interlaced sample visiting order, which is nice but it's not great for memory locality, and it effectively only allows nearest neighbor subsampled previews. E.g. a checkerboard pattern at the pixel level (alternating black and white pixels) will look completely black in all the previews.
Squeeze is better and produces a gray preview for a checkerboard pattern, but it comes at the cost of introducing new colors which can harm compression, especially for nonphoto images.
|
|
2024-09-06 05:48:07
|
What makes this any better than Adam7 though?
|
|
|
|
_wb_
|
2024-09-06 05:49:34
|
FLIF did Adam-inf (not just 7 steps), and it did it in a planar way on the YCoCg image in an order that gives you luma faster than chroma, giving better previews faster.
|
|
2024-09-06 05:50:29
|
Also in FLIF, the prediction uses info from previous interlacing levels, while in PNG each interlacing level is just coded independently.
|
|
|
|
CrushedAsian255
|
2024-09-06 05:50:51
|
Is the first scan literally 1 pixel?
|
|
|
|
_wb_
|
2024-09-06 05:51:46
|
That's what makes it compress so much better: in the final scan (the one that covers half the pixels) you can predict a pixel from 7 out of the 8 neighbors, which is pretty cool.
|
|
|
|
CrushedAsian255
|
2024-09-06 05:52:36
|
That’s really cool
|
|
2024-09-06 05:53:04
|
I kinda want to write a FLIF decoder as like a “warm up” before working on a JXL decoder
|
|
|
|
_wb_
|
|
CrushedAsian255
Is the first scan literally 1 pixel?
|
|
2024-09-06 05:53:30
|
I don't remember what the exact criterion was FLIF used, it could be. In JXL it can be signaled how many Squeeze steps to do, the default is to keep squeezing until you get a pass that fits in 8x8
|
|
|
|
CrushedAsian255
|
2024-09-06 05:53:35
|
(No this hypothetical future decoder is not a production ready thing, it’s a personal project to learn more about formats)
|
|
|
_wb_
I don't remember what the exact criterion was FLIF used, it could be. In JXL it can be signaled how many Squeeze steps to do, the default is to keep squeezing until you get a pass that fits in 8x8
|
|
2024-09-06 05:53:50
|
Why specifically 8x8?
|
|
2024-09-06 05:53:58
|
Is Modular also in 8x8 blocks?
|
|
2024-09-06 05:54:04
|
I thought Modular was in larger groups?
|
|
|
|
_wb_
|
2024-09-06 05:55:14
|
No, that is just an arbitrary stop condition. There is little point to go all the way to 1x1.
|
|
|
|
CrushedAsian255
|
2024-09-06 05:56:33
|
How is it ordered though? Is each Squeeze step its own Modular bitstream?
|
|
|
|
_wb_
|
|
CrushedAsian255
I kinda want to write a FLIF decoder as like a “warm up” before working on a JXL decoder
|
|
2024-09-06 05:56:39
|
Beware: the flif "spec" is not complete and not detailed enough to do an actual implementation from just the spec.
|
|
|
|
CrushedAsian255
|
|
_wb_
Beware: the flif "spec" is not complete and not detailed enough to do an actual implementation from just the spec.
|
|
2024-09-06 05:57:00
|
Is it more of an RFC?
|
|
|
|
Tirr
|
2024-09-06 06:04:08
|
Modular sub-bitstream has a notion of "channels", which can be added by transforms like Palette and Squeeze. Squeeze transform take some channels and produce twice more channels, repeated as needed (repetitions and parameters can be explicitly signalled or implicitly generated from frame dimension). Each Squeeze transform increases horizontal or vertical subsampling factor of the channels by one.
After then, Modular channels are split by its subsampling factor, so that channels with higher subsampling decodes earlier.
|
|
|
|
CrushedAsian255
|
2024-09-06 06:09:22
|
So is this the “c” number from Jxl-art?
|
|
|
|
Tirr
|
2024-09-06 06:10:03
|
I'm not familiar with jxl_from_tree syntax 😅
|
|
|
|
CrushedAsian255
|
2024-09-06 06:10:23
|
I haven’t gotten to modular decoding yet, so I don’t really get it
|
|
2024-09-06 06:10:44
|
I’m still only barely reading the image header
|
|
|
|
_wb_
|
|
CrushedAsian255
So is this the “c” number from Jxl-art?
|
|
2024-09-06 06:21:36
|
Yes. Without any transforms, c=0 is R, 1 is G, 2 is B. But transforms can add or remove channels.
|
|
|
CrushedAsian255
Is it more of an RFC?
|
|
2024-09-06 06:22:45
|
RFCs are real specs. The document I made for flif is just a draft, something that could eventually have become a spec but it is unfinished and probably never will be finished.
|
|
|
|
CrushedAsian255
|
|
_wb_
RFCs are real specs. The document I made for flif is just a draft, something that could eventually have become a spec but it is unfinished and probably never will be finished.
|
|
2024-09-06 06:25:20
|
are RFCs like release candidates?
|
|