JPEG XL

That make sense, I remember Butteraugli design intent is for this quality target(d0.5~d3).

Also, this image is kind of a cheat.

It activates some of the screen content tools that aomenc has access too.

Well av1 has palette blocks, which are probably useful for anime. Jxl doesn't have that, but it does have patches which can be used as palette blocks.

one thing I've noticed from implementing jxsml is that you can also perform Palette transformation over already transformed Palette metachannels as well

Yes, no idea if that is useful for anything

I think it might be useful if you have e.g. a 10-bit image where each channel only actually uses 800 different values, and those are the same values for R,G, and B.

Then doing three single-channel palettes is useful to bring the range of the channels to 0-799 instead of 0-1023

Ah wait no palette on palette is not useful for that

For that case, doing an RCT on those 3 identical palette metachannels would be useful, so it only has to be encoded once and the two others become just all-zeroes

If you're just exporting for the web or to share with others, no. If you are saving a layered file for storage and possibly future editing, the color values should be preserved, transparent or not.

Yes, if you intend to do further editing of the alpha channel, then of course all values should be preserved. But then you probably also want to use lossless, not lossy.

https://www.phoronix.com/scan.php?page=news_item&px=OpenJPEG-2.5-Brings-HTJ2K

What's the best tool to use to compress PNGs as much as possible? Without lossy transformations.

I personally run a combination of optipng and zopflipng

optipng will do things like colorspace reduction, potentially to palettes

and zopflipng compresses the idat chunk with zlib

zopflipng cannot multithread but that's not an issue if you're doing pngs in bulk since you can just use GNU Parallel to process-based parallelize

mm

any settings recommendations?

um, well I personally don't use zopflipng, I use `pigz` which is a multithreaded gzip that supports -11 for zopfli

but.................

I recommend ECT. https://github.com/fhanau/Efficient-Compression-Tool/releases/tag/v0.8.3

I wrote my own shell script that reads a PNG from stdin and writes a PNG to stdout and recompresses the idat chunk and fdat chunks with pigz

so it can thread

it's *horrible*

https://0x0.st/oaS7.sh

Isn't ECT more efficient than optipng+zopflipng?

I don't know, is it?

Think so

ECT doesn't appear to be packaged on my distro anyway, and I'm too lazy to get it on my own >.>

Guess we could try what's more efficient with this image <:KekDog:884736660376535040>

AVIF will the best for strong compression

And this is really the best solution for the Internet

most images on the internet are not low quality

a?

go will be

Most people watch not the original picture, only watch the discord preview, jpeg, or in exceptional cases wepb

https://media.discordapp.net/attachments/805176455658733570/977234079185506374/Resize.png?width=400&height=225 this

This has nothing to do with the image though <:KekDog:884736660376535040>

That's just a Discord preview. If someone were to use the image I uploaded for an efficiency comparison between zopflipng+optipng then they wouldn't open the console to get the Discord preview :p

I don't really think so. Current "web quality" is around d1.5 to d2.5. Why would people be happy with lower fidelity than what they use now?

ECT + Pingo if you're on Windows and only care about 8-bit. Otherwise, ECT.

```git clone https://github.com/fhanau/Efficient-Compression-Tool --recursive cd Efficient-Compression-Tool && mkdir build && cd build cmake ../src -DECT_MULTITHREADING=ON -DECT_FOLDER_SUPPORT=ON -DCMAKE_CXX_FLAGS="-march=native" -DCMAKE_C_FLAGS="-march=native" -DCMAKE_BUILD_TYPE=Release && make -j 16 sudo make install``` Now the problem would be the dependencies <:kekw:808717074305122316>

well

this is the argument

I did not understand how you made a quality assessment use D for all image

this is d25 for JXL only but for AVIF "d" is useless the same kilobytes

avif will be exactly in this quality or even better - for this content, but 5 more times low size

it's hard to imagine the amount of content that can be involved, it's literally all the art on the Internet

I want to say that the problem is exactly the same as with jpeg 30 years ago

I wanna try this, so could you send me the crc32.py script it calls as well?

https://0x0.st/oaQE.py

ty

you can also use `crc32` from perl

should be at `/usr/bin/vendor_perl/crc32`

interesting color map

looks like an interpretation of Cr and Cb

yep

see all jpeg moment

Did you see my jxl file for that image?

no

show , am I missing something?

https://encode.su/threads/3397-JPEG-XL-vs-AVIF?p=74395&viewfull=1#post74395

and you exactly used varDCT ?

No that is modular

HA

For images like that, modular is more suitable

the problem

oh the big problem

See, the problem is that when JXL fails, it fails a bit.

When a video based image format fails, it tends to fail horribly.

What's the big problem? We have both modes exactly to cover different kinds of images.

It would be nice if we can make an encoder that more cleverly uses both modes, and I am working on that

But bitstream-wise, I don't see the problem...

In any case, for such images, vector formats should be used whenever possible. Rasterization is already an unnecessarily lossy operation imo.

Alright, I did optipng + thebombzen's script, and got the size down from 92'800'591 bytes to 77'606'529 🥳

funny that just yesterday I was compressing 174 thousand images into a limited palette of 32 colors for png

lol, that's one massive PNG

yeah lol

8000x8000 pixels

Probably would be a good candidate for patches, especially if the program generating the image could take advantage of jxl features directly.

Try WebP 😛

You'll see.

currently `cjxl -q 100 -e 9`'ing it :p

kill the cpu

use avif -s 0

AVIF's lossless mode is not that amazing though.

It struggles to beat webp.

you lost the color

can you Compress even harder?

this my 20 kb avif

doesn't it even struggle to beat png lol

the exceptional maximum that I can do without losing colors

but...

If you really need that image compressed a lot, I suggest optimizing the svg further and doing brotli on it

I just did lossless jxl on a pngquanted version of the image, I am sure better things than that can be done, but it's a bit of an academic exercise imo

as soon as I take any anime art using a shadow or a gradient, Everything will break, except for AVIF

Yeah delta palette is probably a better idea, it has no problems with gradients

and yet jpeg xl works better with guardians, even at low compressions

i mean giant solid gradients like 1000 pixels in size

15 minutes passed, of course, not bad :)

I got it down to 39KB with a bit of SVG optimisation.

130 kb wtf

I probably need to start doing manual tricks to get better than that.

oh god lossless is not a lossless

wait

Avif does not contain an algorithm for lossless compression?

It does.

should use `-d 0` instead of `-q 100`

it's not different but ideally you use distance instead of quality

a good habit to get into

it means i'm doing something wrong

wtf its not the lossless

just XOR image

probably need `--max 0 --maxalpha 0`

cavif has a seperate `--lossless` option.

oh

avifenc has `--lossless` yea, and if you override any settings that make it not lossless then it will warn you

lol

same

but -min 0 lossless is not true lossless

jxl fidelity is unmatched

👉 not for anime

`cjxl -q 100 -e 9` took it down to 71'456'496 bytes

noice

you didn't set the alpha quality setting

because I can it

the ability to compress the alpha channel at Another level is a big advantage

and....154 kb

what

the PNG is 60 kb

what the 🤡

jxl has no impovement

same quality as befoe

as thee months ago

ok...my pc too, already 5 years

and smartphone...

it seems this is the end

av2 will never come out

but... what the hell is going on with lossless avif

they really, no joke, create an exact copy of the image based on their algorithms FOR LOSSY COMPRESSION

just like jpeg without round the DCT coefficients

**in this case, the situation in which there will be color Paletre - instant defeat**

Jpeg XL was much smarter in this sense and just had a separate algorithm for lossless compression, just like webp

and yet no one can beat it

but if you use a palette, then it is possible, therefore, it is better not to give such a chance and take a more complex image

and even with noise

there is a parameter in avif, but I'm not entirely sure that it does something useflu enable-chroma-deltaq=B : Enable delta quantization in chroma planes (0: disable (default), 1: enable)

is you community in ussian language

on telegram

yea

do not worry, i won't enter

is about hacking?

no, it's just what i like

sometimes these are very distant things like codecs or something technical

do you have study

maybe you should add another link

no

no university

are you young

I dropped out of university and work

I also break the law by not joining the army

they will come for me soon, and my life was not long

sorry you don’t understand, but in this picture I created a comprehensive instruction for 3 video codecs at once

and I also made a minecraft font based on game files, although I used some kind of Japanese algorithm on github with Japanese descriptions and instructions in Japanese

I use it every day everywhere, I stopped wearing glasses and I see text only through this font, because it occupies a maximum of useful area and is discrete

That is actually a pretty cool font.

this version has all unicode characters for all languages, even china and japanese character set

yez

it turned out that avif lossless for vector images is not good at all,

101638 bytes

in the CJXL should do this: -t is --num_threads

Howdy folks, I'm looking for an image format that can compress (losslessly) the h*ck out of my hundreds of GBs of 3D stereo pairs like these PNGs Someone suggested HEIF which supports that <https://nokiatech.github.io/heif/examples.html> but JPEG XL might be able to achieve even lower filesizes. Should I go with HEIF, JPEG XL or does anyone have any other suggestions? 👀

why are this is pictures?

do not use HEIF it is already outdated, there is either JPEG XL or AVIF Since you have an image of a non-realistic plan here, I think avif will suit you better, but it all depends on how many such pictures you have and how long you are willing to wait (processor power)

<@740013717114192023> but better tell how many times from the compressed PNG you would like to compress these images?

Current libjxl will not do this, but for such images, you could encode one half and then encode only the diff for the other half

how would you do that?

Encode one frame that is just one half, then a next frame that uses Patches to subtract one half from the other

because if you want to compress about ~2 times the original, then it's best to use JPEG XR which handles a lot better with a little rounding of the coefficients, better than jpeg and jpeg XL. And at the same time it has a very high speed, I would recommend XnConvert for multithreaded compression, jpeg XR "with all block filters" at quality 95 <@740013717114192023>

oh yes... a stereo pair, for sure, can get confused and write an algorithm that will translate each pair of frames into an AV1 video, but this is very confusing

wait, is this channel for video? I'm just trying to compress side by side images, not sure how to encode frames separately

compression cannot work in such a way that it affects part of the image on a completely different pixel gap away

I know there's a video codec (MVC, used in Blu-ray 3D movies) that encodes the second view taking into consideration the difference to the other or something like that so basically I'm trying to do that, but for individual stereo pair images

wait a second, I'll see how compress 3d videos on torrents

Jxl can do zero-duration frames to make a composite still image

MVCs are common in BD3D rips with MakeMKV

so I take it that's doable with cjxl.exe, right? what commands would I need to do this?

Half OverUnder

AVC

yeah that's definitely re-encoded

just avc

Nah there is currently no cjxl option to split up a stereographic image, it's just something that could in principle be done

completely standard AVC no one takes into account the compression of the difference between the halves of a stereo pair

A proper BD3D rip should look something like this in MediaInfo (notice the `Stereo` flag in Format profile) ```Video ID : 1 ID in the original source m : 4113 (0x1011) Format : AVC Format/Info : Advanced Video Codec Format profile : Stereo High@L4.1 / High@L4.1 MultiView_Count : 2 MultiView_Layout : Both Eyes laced in one block (left eye first) Format settings : CABAC / 3 Ref Frames Format settings, CABAC : Yes Format settings, Reference : 3 frames Codec ID : V_MPEG4/ISO/AVC Duration : 1 h 33 min Bit rate mode : Variable Bit rate : 30.8 Mb/s Maximum bit rate : 30.0 Mb/s Width : 1 920 pixels Height : 1 080 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 23.976 (24000/1001) FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.620 Stream size : 20.2 GiB (75%) Language : English Default : No Forced : No Original source medium : Blu-ray```

Stereo flag? I do not see

5th line in my paste

> Format profile : **Stereo** High@L4.1 / High@L4.1

ho

`left eye first` is another giveaway that the video is stereo 3D

so with what we currently have now. what would you recommend?

funny that it is written with the slash

well

avif can do that too. of course, only then it will not differ much from the AV1 video

here are the results with some converters so I just wanted to keep looking for alternatives

<:AV1:805851461774475316>

GIMP HEIC seems to do a better job than JXL (faster and slightly lower filesize, perhaps because it might be automatically detecting it as a stereo pair and using differences) but I'm not quite sure how lossless it is. I put it on top of the source PNG in GIMP with blending mode set to difference and it was all black, which usually means they're identical/lossless but I wanna make sure.

neat, where is this from if you dont mind me asking?

compiled

ahh, neat. just tried it and it's pretty dang impressive, but I'm also afraid it might be lossy

yeah the default's lossy so I'll check out the commands/help

Its --lossless for avifenc

<@740013717114192023>

Oh that's right. already downloaded it and gonna try it next 👌

like this, right?

yes but q 95

and specify the folder where

dang, lossless quality comes at a high price

and remove the ask parameter, without it the multithread will not work

i say use 95

its maximum lossless

for u

you mean like visually lossless?

Yeah, there is a lot to be said for very near lossless.

but by the way, there is another codec for you

Like cjxl -d 0.3 kind of level.

<@740013717114192023>given the plane of the content, you can try WEBP at 100 quality (lossy)

the reason I want lossless is cuz even slight differences are a lot more noticeable when viewing 3D images just like crossing your eyes makes find the difference games very obvious, cuz compression artifacts in one eye will probably not match other

iWEBP at 100 quality (lossy) will slightly less than jpeg XR 95

what app should I use to convert? XnConvert?

yes

<@740013717114192023>have you already made all the important settings? send another screenshot

oh yes, webp may not be suitable because YUV420

Yeah, I'm just trying other lossless formats

so I'm just looking for the smallest lossless method with reasonable encoding/decoding speed

then I think jpeg xr 95 is the best option, more test for this option

i gotta go but I'll keep trying more formats. XnConvert seems like a perfect tool for what I need. guess I just need to find the right format and settings 👌

probably rimworld from the filename

Still a cursed name tbh <:KekDog:884736660376535040>

Rimworld, yes

Since it's composed entirely of sprites, i wonder how much smaller it could be with the proper application of patches

ECT got it down to 77,572,980 bytes <:KekDog:884736660376535040>

Don't know what parameters though, because running 4 instances with different parameters and it's still going, so might become smaller XD

77,315,000 <:thinkies:854271204411572236>

Hm, 77,175,971

Use lossless WebP if you want decent encoding speed and fast decoding, and don't mind being limited to 8 bpc. JXL at something like -s 3 might also be a nice alternative but the decode speed of lossless/modular content was a bit slow last time I checked. Use near-lossless WebP (start with 60) if you want something that is practically indistinguishable from the original, even when zooming in. Use lossy (VarDCT) JXL with low distance if you want mostly transparent results with low file sizes.

<@456226577798135808> <@226977230121598977> I kept trying more formats and here are my results. q=quality, c=compression level

the smallest lossless results is avif, but compression time is 4 mins 😬

also the 3D image viewers I have don't support uncommon formats like avif, heic, jxl, etc, but webp is supported so it does seem like the best option

compression is absolutely lossless in this case is meaningless

well, if there is only webp support then yes, use q100 lossy with algorithm 6

but the yuv420

not quite sure what you mean, but my priorities are small filesize>lossless quality>decoding time>encoding time

these are my options atm btw

<:PepeOK:805388754545934396>

it's probably not lossless RGB then, AVIF is rather bad at that and surely using YUV in your case. I would keep my distance to AVIF, HEIC and lossy WebP unless you are targeting very low bit rates.

then you should go with (near) lossless WebP but I recommend using cwebp directly: https://storage.googleapis.com/downloads.webmproject.org/releases/webp/index.html

then we've got a winner webp lossless compression level 5 just takes 5 seconds, while level 6 takes 33 seconds yet the KBs are the same <:WhatThe:806133036059197491>

it might just try more stuff and then finds out that it isn't any more useful for that particular image.

yeah, that's why I was hoping for a format that was aware that it's a stereo pair image to encode redundant data efficiently

I'm confident that this won't be effectively possble when targeting true lossless results.

as for lossy formats, even webp at 100 quality has significant quality loss, especially at high contrast edges. yuck

yes, it's way better to use something like `-near_lossless 60` with cwebp

neat, I'll give it a shot 👌

Webp q100 is still 4:2:0 and tv-range ycbcr, so that is roughly 7.5-bit G and 6.5-bit R and B with 75% of the R and B missing.

Nope. Lossless AV1 is lossless.

Just activate the lossless flag, and it's fine.

But wasn't that only added "recently"? I would guess that XnConvert could be using an older version for AVIF. Otherwise I would be pretty impressed if there existed an image where it beats WebP as well as JXL for lossless.

It's not recent(it's been about 1-1.5 years since it's lossless), but I wouldn't trust XnConvert for good conversion at all in this context.

Not impossible on a single image, but somewhat surprising, yes

Ideally, you'd check with avifenc/avifdec if the image is lossless or not.

If it can use intra block copy effectively, while jxl is not trying such a thing atm, it could be that it is more effective for stereographic images...

`avifenc -l test.png test.avif`uses YUV444

That 4,980kb avif was encoded with avifenc using the lossless flag too, and here's the mediainfo ```Image ID : 1 Format : av01 Codec ID : av01 Width : 3 840 pixels Height : 1 080 pixels Color space : RGB Bit depth : 8 bits Stream size : 4.86 MiB (100%) Color primaries : BT.709 Transfer characteristics : sRGB/sYCC Matrix coefficients : Identity Color range : Full Codec configuration box : av1C```

sYCC, so it's not lossless as expected

There's something wrong.

Something changed.

I need to check with aomenc.

Was it only possible to get lossless if you took a single AV1 frame and put that into an AVIF container?

No, I could encode AV1 lossless in the past no problem, even RGB...

Quite weird honestly.

Identity matrix could mean it is not YCC

Can you decode and run `compare -metric pae orig.png decoded.png null:` ?

I wouldn't be surprised if avif wins on images that have duplicated regions if it can use intra block copy

It's obviously not lossless. Anything other than 0.00 in `butteraugli_main` means it's not lossless. That's why it's strange: something changed.

```butteraugli_main /home/bluezakm/Videos/2048x1320_david-marcu-441_png_original.png /home/bluezakm/Videos/2048x1320_david-marcu-441_png_original-avif.png 2.8017091751 3-norm: 1.348538```

intra won't be effective for lossless. I saw that when testing lossless x264 vs individual fjxl frames.

I posted a new reply to the issue: https://github.com/AOMediaCodec/libavif/issues/919

Yeah usually not, there might be exceptions though. In principle both av1 and jxl can exploit the redundancy between left and right half of stereographic images, but I know libjxl isn't doing that (yet).

60 really does seem like the magic number I see no pixel differences compared to the original, but as soon as I drop to 59, yikes

61-100 also yield no visible pixel differences as expected, and so does avifenc- lossless

Here's another case of counter-productive max compression level (-z command) ``` 1-NEARLOSSLESS60z1.webp 3.111s 3381720 bytes (6.52 bpp) 1-NEARLOSSLESS60z2.webp 3.231s 3379364 bytes (6.52 bpp) 1-NEARLOSSLESS60z3.webp 4.337s 3143490 bytes (6.06 bpp) 1-NEARLOSSLESS60z4.webp 4.259s 3143490 bytes (6.06 bpp) 1-NEARLOSSLESS60z5.webp 4.460s 3108410 bytes (6.00 bpp) 1-NEARLOSSLESS60z6.webp 6.261s 3106254 bytes (5.99 bpp) 1-NEARLOSSLESS60z7.webp 7.334s 3107114 bytes (5.99 bpp) 1-NEARLOSSLESS60z8.webp 8.000s 3047462 bytes (5.88 bpp) 1-NEARLOSSLESS60z9.webp 41.358s 3047034 bytes (5.88 bpp) ``` level 9 takes over 40s while level 6 (i think it's the default, so no wonder) takes just ~6 seconds and the filesize is the smallest so I guess I'll just use -z 6/keep the default.

Well 8/9 are a bit smaller, here it looks 8 is a good idea

oh h*ck you're right. that was my first impression but then brainfart

https://giphy.com/gifs/ohdY5OaQmUmVW

I also made the same brainfart, looking too much at last digits lol

numbers. they're hard, man

https://encode.su/threads/3397-JPEG-XL-vs-AVIF?p=74423#post74423

your program might not show differences below a certain limit. IIRC from what <@532010383041363969> told, then 80 should allow +-1 Bit change, 60 +-2, 40 +-4. But no idea what happens in between for 59.

👍

❓

with my setup, it should exaggerate even slightest pixel differences but yeah there's probably a more objective method to compare pixels more accurately, but my guess that this is at least literally indistinguishable even if it might not be mathematically lossless. I just wanna avoid any visible pixel differences cuz they're way more noticeable in 3D cuz eyes/brain are pretty sensitive to left-right differences (like for depth perception)

What about Aliasing? Shouldn't this be a way bigger visual concern than slighly lossy compression?

<@740013717114192023>ok just use classic jpeg with yuv444 and Q 100

Why would you ever do that?

That's almost guaranteed to be larger than lossless while not actually being lossless

<@456226577798135808> not sure I follow. isn't aliasing just sharp, blocky edges in raw rendered frames? not sure how it's relevant in lossless compression. if the conversion somehow messes with the edges, the difference will be visible, otherwise it will be just a black frame AKA all pixels should at least in theory be identical <@226977230121598977> do any of these set yuv444?

yes 1x1x1 is yuv444

and optimaze huffman

Well, that's also an option.

same as jpeg2000 at max quality

thing is that it is fast

Fast png is faster, fully lossless, and probably denser

Or fjxl

I'm actually getting a much smaller filesize compared to the original 9MB PNG and it also seems lossless (no pixel differences) and it really is pretty fast (sub-second encoding time) and smaller compared to this PNG re-compressed to be able to post on discord (took 3 seconds)

what do people here use to compare truly lossless conversions to the original?

compare -metric pae

jpeg2000 have Wavelet transform - interesting for your case

Not really I think. DWT is quite bad for non-photo, maybe even worse than DCT...

I mean the game's aliasing patterns will be slightly different between the left and right image. I can't imagine high quality lossy compression to be as noticeable with 3D as this aliasing "issue". Btw, your Nickelodeon game seems to have problems with the bloom implementation. It's too far left for the left eye and too far right for the right eye (very awkward on e.g. the brown ball above player 2).

ahh I see what you mean. 3D objects don't (and shouldn't) be exactly the same for both eyes, unless they're flat/2D (e.g. HUD). The separation between left and right will inherently make objects look slightly different, but most of the time it has to be cohesive. (e.g. a flat-colored cube will look like the same shape for both eyes, but water will appear to have different shapes because of reflection/refraction.) I've never noticed anti/aliasing making a noticeable difference in 3D, but I absolutely regret taking some of my early 3D screenshots in JPG format cuz the compression artifacts are exaggerated in 3D, like seeing through dirty glasses.

As for that particular screenshot, this game isn't a great example of proper 3D. It's just a recent one I found to have multiple qualities I thought would be important for re-encoding (high contrast edges, text, many colors, grass, gradients, etc)

but you're right, it's fairly rare but there's definitely something weird going on with the brown yarn ball strings AA/haze/bloom. Some games have broken effects in 3D so I'll see if someone in the 3DV community can look into it 👌

<@794205442175402004> which tool is that?

Imagemagick

`compare -metric pae orig.png compressed.png diffmap.png`

Or `null:` as the last argument if you don't need to have a diffmap

yikes, now it looks like a Jackson Pollock lmao

Can adjust tollerance for diffmap with -fuzz

-fuzz 3 or whatever

But just the number you get is what tells you how much the peak error is

Should be 0 when really lossless

Default imagemagick is 16-bit internally, so an off-by-one in 8-bit will be something like a pae of 256

gotcha. with the jpg I get `1028 (0.0156863)`, but with the PNG re-compressed file I get `0 (0)`

So that's up to off-by-4 in 8-bit values

Which makes sense, just YCbCr will already do something like off-by-3

You can do `-verbose` (before the `-metric`) to see it per channel

Probably will be mostly R and B that have errors

Can also use `mae` instead of `pae` to see mean error instead of peak

Or rmse/psnr

oof, guess webp near_lossless 60 was lossy too (should've expected it from the flag name lol)

`cwebp -lossless -z 8` is truly lossless (0) so guess I might just go with it

but gaining 20-30% density by changing pixels less than 1/256th is still pretty impressive. I don't think any modern (lossy) format can compete with that.

here's the general impression I've gotten from the formats I've tried

Oh wait, JXR is actually lossless. I'll be updating the chart here <https://airtable.com/shryqQXwTwpNAakt1>

I'm still annoyed HEIF/HEIC is supposed to be lossless according to the XnConvert quality slider, but even all the examples here are lossy according to compare <https://avi.alkalay.net/2018/08/heic-lossless-images.html>

I think compatibility is a very large point in webp's favour. (Almost to the point of going for PNG instead, but that depends on your exact usecase.)

I just wanna shrink my 3D screenshot collection (mostly PNS/PNG and some JPS/JPG) in size, while keeping the same quality in case I need to re-convert to a more efficient format in the future without generation loss

that's why I wanna make sure before I commit to letting my CPU burn overnight <:kekw:808717074305122316>

In that case webp lossless is probably the best option, as the tooling exists for you to be able to use it currently. JXL isn't quite there yet with the tooling, though I would be tempted to use it on the JPG files due to the bitexact recompression.

Ohh that's right. Doesn't make a lot of sense to convert lossy to lossless guess I'll just convert PNGs to WebP lossless and JPGs to JXL

<@176735987907559425> does cjxl do bitexact recompression by default when the source is JPG?

for non-progressive typical JPEG files, yes to my knowledge (you can override it by `-j`)

Yep, it is the default so long as a distance or quality isn't specified (unless that is -d 0 or -q 100). It should say "JPEG, lossless transcode" in the output.

Also for progressive ones. The jxl will not mirror the jpeg scan script, but it will encode the same data and can reconstruct the file just fine.

Look at it this way: a typical 8-bit photo can be losslessly compressed at 12 bpp. A lot of that goes to the noise in the lsb. If you allow the lsb of every channel to change, that's basically 3 bpp worth of incompressible bits won.

Actually allowing an off-by-one in either direction means you can quantize RGB by a factor of 3 (rounding to the nearest multiple of 3, which is never more than 1 away), so you can get rid of a bit more than one bit of information

It also means you reduce 16 million colors to 16/27 million colors...

https://media.giphy.com/media/KxhIhXaAmjOVy/giphy-downsized-large.gif

Doing the quantization not directly on the RGB values, but instead on predictor residuals, will have the same compression advantage but produces nicer results, at least when using an averaging type of predictor. That is what lossy flif does, and what you can also do to do lossy png24 (using the AvgN+W predictor in png). I don't know if that's what near-lossless webp does too? <@532010383041363969>

In jxl we could also do predictor residual quantization. Probably not as effective as delta palette, but still could be worth doing since it has strong error bounds...

I collect some webp near-lossless detail from Jyrki Alakuijala 🙂

<@794205442175402004> Is it possible that there's a bug with ssimulacra_main and butteraugli_main regarding checking some lossless stuff? https://github.com/AOMediaCodec/libavif/issues/919#issuecomment-1135163700 On his end, these images seem to be lossless: https://slow.pics/c/AlqqgdC3

Even PSNR agrees.

`ffmpeg -i 2048x1320_david-marcu-441_png_original-s0-avif.avif -i /home/bluezakm/Pictures/unknown/2048x1320_david-marcu-441_png_original.png -lavfi psnr=stats_file=psnr_logfile.txt -f null -` `[Parsed_psnr_0 @ 0x441d5c0] PSNR r:inf g:inf b:inf average:inf min:inf max:inf`

something I've wondered is what if you take an 8-bit RGB and compress it *lossily* in 16-bit with a high enough quality that you can bitexactly reproduce the original image. is this a thing that is feasible?

that is, you take each value and multiply it by 257, and then lossily compress it, and then afterward you divide each value by 257 (rounding nearest)

could this work, and would it be more effective than just lossless modular

assuming standard input PNG conditions, i.e. sRGB, 8-bit RGB (no alpha)

if you set the distance low enough I wonder if you could achieve this

Does the png file have a color profile? Most tools will just compare rgb values without taking color profiles into account. Ssimulacra_main and butteraugli do take color profile into account, so if the rgb values are identical but the primaries are different, they will show some nonzero value

That might just be it 😅

Could be feasible but I doubt it would be more effective. Also if the goal is to end up doing 8-bit rgb losslessly, it's better to not do XYB since that will not uniformly spread the error (you'll get more errors in the blues, etc)

what's the option to use original profile with `cjxl`

Wait, what the... I'm also getting similar results on my end. I converted PNGs to lossless WebP and also see a difference in contrast even tho compare reports all 0s

forgot to rename it, PNS=PNG

looks like a viewer presentation issue, if the pixel data is the same

like one of them is respecting ICC and one isn't, for example

I'm using the same viewer app for both files*. or do you mean there's something missing in the converted file?

I mean the viewer might be respecting metadata for PNG and not for WebP, for vice versa

either way if the pixel data is the same and the same viewer doesn't look the same for the different files, then it has to be a metadata issue

since there's nothing else it could be

makes sense. that explains why they look identical in Firefox/Discord

guess it was a different issue, cuz I can still see the difference in those on the browser

h*ck sorry I left the mention on

anyway guess I'll open an issue for the app then

Cjxl does that by default if the input is png or jpeg and you're doing lossless. If you do lossy, it will investigate the input profile, and if it's 'close enough' to an enum space, it will signal an enum space instead of encoding an icc profile.

no, I meant, how do I tell it to disable XYB encode for lossy input

like is there an option

Yeah `compare` ignores color profile, if you use cwebp it will silently strip the color profile, which is pretty bad behavior but it is what it is (also dwebp will silently strip the color profile in the decoded output)

Ah, -c 1 should do that, but no idea if that actually works in vardct mode

do you mean `-C 1` for YCoCg?

No

Lowercase -c

cause it's undocumented

VarDCT cannot use RCTs since those are modular transforms

I did `--help --verbose` and it still wasn't there

one more --verbose

oh didn't realize I could do `--help -v -v`

ah, there it is

Lowercase -c 0 means XYB, -c 1 is RGB, -c 2 means the input you are passing it should be interpreted as YCbCr instead of RGB

so `-c 2` means I'm passing it YUV 4:4:4 input using the BT470BG matrix

Uh

(the standard Jpeg matrix)

Yeah

The jpeg matrix, full range

thanks

You can use that to do lossless yuv444 with modular

is there any particular reason to use that matrix and not RGB for input that isn't supposed to be legacy jpeg compatible

cause the problem is if you have yuv444 with a different matrix, like the BT709 matrix, then the colors will be wrong

Yeah I guess we should have made the ycbcr matrix a header field that just has jpeg as a default value or something

also how does JXL losslessly recompress 4:2:0 jpegs? do you compress than as 4:4:4 and scale the Cb and Cr planes up by a factor of 2?

In the YCbCr case, we allow each channel to be subsampled 2x in either direction, there are basically 6 header bits for that

could use the extensions field to signify a different matrix probably

that would allow it to be compatible

but I figure that is probably a low priority at this point

PNG has a really intelligent clause in the spec that says decoders are required to ignore unknown chunks and not signify an error

which guarantees forward compatibility

could make JXL work similarly where if an extension field is set to something nonzero and a decoder doesn't know what it means, just ignore it

Yes, I guess an extension would be sensible. Old decoders would fall back to using the wrong matrix and give wrong colors, but that's probably not the end of the world since this use case would probably allow requiring everyone involved to update their decoder

Yes, extensions are defined like that. Decoders will ignore them.

Well, header extensions at least.

also, since garbage at the end of a codestream is considered legal, you could always add extra chunks after the codestream and older decoders will just ignore it

or something of that form

if it belongs in the codestream and not in the container that is

We have a mechanism where extensions signal their length so old decoders know how many bits to skip

... hm? I didn't encounter this when reading the header draft

So we can add extensions and have the extra info right in the header

the extensions field itself is a U64() to determine which extensions are present iirc

and for each bit set, in order, you have another U64(), which is the payload, right?

unless it's changed since that draft

I didn't see anything that could be a payload except the U64()

Well the other U64() per extension is not the payload, it is the _size_ of that payload

oh, that's... not clear.

so that U64() is a size field in bits, followed by the actual payload?

So a decoder needs to read them and skip each one it doesn't know

I think it's first all the sizes, then all the payloads, iirc

it would have to read it anyway since U64() is variable length, but I didn't realize it had to skip that many bits

At the moment it will be zero bits to skip since nothing is writing extensions, but yes, when something does write extensions, you have to skip the sum of those sizes

... that is not stated in the spec draft

That last sentence is maybe not explicit enough

Yea, that implies to me that the extension_bits is the payload, not the size of the payload

The "starting after extensions_bits has been read" should help, but probably we should just add a row there of type u(extension_bits[i]) and with name extension_i_payload, or something

yea, that would make it very explicit

Problemo solvedo https://github.com/gkv311/sview/issues/107#issuecomment-1136467761

Thank h*ck for actively maintained FOSS 🙏

I used `-mt` for multi-threading, but shouldn't it use more CPU? seems to peak at ~13% <:DogWhat:806133035786829875>

webp<:Thonk:805904896879493180> multi-thread<:WhatThe:806133036059197491> how

probably only parts of it are actually multi-threaded, e.g. things like color transforms are easy to parallelize

Even with the best 90s computers it would take so long it's unrealistic and nobody could play it back™️

I remember the time when mp3 was just invented and people with old computers had trouble playing them

I recall a similar experience with jpeg(1)

Lol. I wasn't even born yet when mp3 became a thing XD

Lol, even AAC is older than me <:KekDog:884736660376535040>

mp3 is a 30 year old codec iirc

mp3 is like the audio equivalent of legacy jpeg

universal support but very primitive

JXL to legacy jpeg to me is like Opus to mp3

CPU also seems to be going up and down and the process seems to be restarted. so if it allowed multiple parallel processes, it would be a lot faster, at least in theory

<:HaDog:805390049033191445>

128x64 why not

I know that VP9 is parallelized only for 2 threads. and further only after frame splitting

this is an extreme explicit association The saddest thing is that no one makes a better codec than opus. it's sad we've reached the end

I mean, opus is lossy and last I checked it doesnt support 44.1khz so there's still room for improvement

not supporting 44.1 kHz was an intentional decision

because it's lossy

essentially all hardware samples at 48 kHz except some hardware that specifically plays back CDs

so you're going to have to resample 44.1 kHz to 48 kHz *anyway* on playback

this forwards the resampling to the encoder side which makes more sense

you don't need 44.1 for lossy

and also, there is SOX - the last word in the world of respemlers

Basically, it's not an issue 😄

don't forget the Nyquist-Shannon Sampling Theorem

yeah, I get that, but it still seems kinda arbitrary that they chose to leave out 44.1khz while 8khz is still supported like, it wouldn't make a lot of sense to convert 8khz to 48khz, increasing filesize for no good reason. is it because of multiples?

because 44.1 kHz and 48 kHz are both fullband

fullband being able to represent frequencies in the 0-20 kHz range

anything higher than 20 kHz is irrelevant

so 44.1 kHz and 48 kHz are both fullband

as far as I'm aware opus has a hard lowpass at 20 kHz anyway

8 kHz on the other hand is not fullband, it's narrowband, and it's used for a different purpose

ahh, I see. makes sense

still, as much as I like opus, I wouldn't say it's the "best" codec out there. like if the point is to archive audio bit-perfectly/non-destructively, it's probably better to go with a lossless format

it's the best lossy audio codec out there

but you are correct that flac is better suited for archival