JPEG XL

Obviously also the early display and storage technology was quite low fidelity anyway, so there was no point in broadcasting a high-fidelity signal that would then be ruined anyway.

I have a suspicion that the RGB to YUV formulas were tuned so the RGB color bar test would show as a nice grayscale steps on a black and white screen.

I think there is no correct way how to extract luma from a color picture. Even different camera technologies have different sensitivity to different wavelengths.

professional cameras have specifications that detail the photo sensitivity of the sensors

well in most cases

There is a 'correct' way to extract the luma from a color image, based on the (average) response curves of the cone cells in the HVS. But to actually turn a color image into a grayscale one, you might want to do something different — e.g. take into account that we can see contrasts based on chromacity too, and try to not lose that information (e.g. so you don't just remove red text on a green background when both have a similar luma), or take into account that L,M,S cones are distributed differently in the fovea, causing S to be more relevant for its wide/peripheral/low-frequency contribution and L,M more for their high-frequency contribution (that's what inspired XYB to define luma using only L and M cone responses).

color->grayscale has been the source of many a headache for me lol

is something weird with the metadata in this png? ffmpeg says `rgba(pc, gbr/unknown/unknown)` (is there alpha or not?), and cjpegli (but only in xyb mode) creates a jpeg that is stretched horizontally and with messed up colors (as if it interpreted rgba data as rgb)

there is for sure an alpha channel

there is alpha, and it is trivial (all opaque)

https://github.com/libjxl/libjxl/issues/2671

aha, didn't find that, thanks

There were several kinds of black and white cameras in use, either film and electronic sensors, vacuum tubes and solid state.

oxipng removes unused alpha channels

All PNGs should be passed through at least fast optimizer to minimize the used color type because so many programs produce bloated PNGs.

For end delivery, definitely...

With GIMP you could mean to produce zero-compression PNGs

The size simply consists of pixels * 3 or 4 with a little information

That's just setting the zlib level to zero, right? You can do that, although I don't know if it has any use other than curiosity.

Sure a simple setting of compression level

Using 0 would result in that size slightly larger than pixels * 3 or 4

Oh I'm not sure about that 4 as I haven't tested with Alpha channel yet

The file must be slightly larger than the raw pixel array. The file structure takes some space as well.

Especially in PNG with the CRC checks with every chunk

With level 0 the one without background even grows 33% larger

And... nothing

You can produce a totally bloated PNG without putting in anything

It becomes 8 KB after optimisation

Grayscale and paletted PNG formats are nearly identical, the format of IDAT chunks is the same. Also I think 4bit and 2bit grayscale formats are there mostly for completeness as these are not used very much. Their paletted counterparts do show up occasionally.

If you use Safari, do you see glitches on the animated WebP? https://jpegxl.info/test-page/

Yes, it leaves traces of not properly cleared pixels.

(also the animated jxl is not animated but a still image)

That JXL works fine on a few Gecko-based browsers like Pale Moon

Not directly relevant, since I'm not relying, but I've found they are useful for getting "directions". They cover a huge range of topics and can help you tell where to look. In the past I asked a question on stack exchange: https://math.stackexchange.com/questions/4710687/optimizing-cookie-clicker-garden-mutation-layout With chatgpt I can get an answer quicker and if it's wrong, no biggie

interesting SVT-av1 artifacts on full static frame

<:AV1:805851461774475316>

Why is the sample animation identical to the ones on jpegxl.info? https://w3c.github.io/PNG-spec/Implementation_Report_3e/

The witness! I love this game!

🫶

It's this one: https://commons.m.wikimedia.org/wiki/File:24-cell-orig.gif

Except an apng version of it, with full alpha, which you cannot do in gif.

Yeah the one on Wikimedia Commons isn't with Alpha

Makes me want to re-render a version in Blender

I can't really understand this 24-cell however

I have no intention to

my brain is busy enough with that sleeping beauty paradox

I'm starting to think that the paradox is that _you_ get less sleep because of it

but my blog post is taking shape so hopefully not for much longer

(I have already convinced myself that the argument for 1/3 is bunk, but I want to explain why clearly enough for it to truly be the _coup de grâce_)

Did you tried to simulate the issue with a small code, run it thousands times and get your answer?

I’m not sure it would solve anything; it’s relatively easy to come up with code that returns either answer (but what’s computed by the “straightforward” way to code the 1/3 answer arguably isn’t a probability)

(there is a less straightforward way that does compute a probability, but a different one than the one in the problem)

svg2 draft was modified for the last time in 2018, do you think it's dead ? https://www.w3.org/TR/SVG2/

oh ! no ! my bad, it went to something else and I didn't noticed : https://svgwg.org/svg-next/

in a nutshell, what are some major differences with SVG 1?

Still trying to understand

https://en.wikipedia.org/wiki/SVG#Version_2

So it's been at the Candidate Recommendation stage for almost 8 years

Let's see PNG spec 3rd edition or SVG 2 would reach the W3C recommendation stage earlier

I still think we need a `.svgb` as a svgz successor with brotli compression

In fact I do serve my static svg as pre-compressed svg.br files for embedding in web pages and it works marvelously

did i do this right? i just installed mozjpeg

``` cjpeg -quality 0 -baseline -verbose bike.png > out.jpeg ```

47kilobytes

why does the jpeg file itself have so many zeroes in it though

zipping the jpeg makes it 30kb

You can use an optimizer like ECT on it for very fast optimization which makes it 10.78KB (23.2%) smaller

so much palette whaou <:KekDog:805390049033191445>

smh I expected you to optimize that file during hours <:KekDog:805390049033191445>

I'm afk now, but started jpegultrascan <:KekDog:884736660376535040>

Might be done by now

btw the update crashing on your machine, is it for all your inputs?

Yup

Tried it with 10 images and they all caused error spam

Heh, 1 and 13 were the same size <:KekDog:884736660376535040>

looks like the graphics of a DOS game

quite expected for a small picture like that <:KekDog:805390049033191445>

It's funny how it's often the case

I rarely have cases where 13 is smaller than 3 XD

I usually just default to using 1 when I optimize many images and 3 when I optimize a few

did you already tried `-b 0` ?

Never tried that <:KekDog:884736660376535040>

Oh i thought mozjpeg was already doing that optimization considering that’s what they claim on their page + they have an optimize flag

yep, and ECT uses mozjpeg's `jpegtran`, but it does a little more of work to ooptimize the file better

I don't remember what was the extra optimization steps (better Huffman coding ?), but ECT compresses in most cases better than mozjpeg

Gotcha, thanks! Ive been curious to see what a few decades of improvement has accomplished for jpeg

I t seems that encoding JPEG as progressive does pretty good optimization by default.

You probably want to remove -baseline from the command

Because of the message above

Ohhhhh

I mustve misinterpreted the description of baseline

I assumed progressive jpeg creates larger files

Because usually things that accommodate for something like progressive transmission end up sacrificing something in return

I've noticed a resurgence lately of games using quantization on the lighting to achieve similar effects, sometimes at lower render resolution too

In jpeg, this apparently works the other way.

Yeah, pretty interesting

In this case because progressive is *slightly* newer (I think), it can be smarter with the encoding

Also... I assume you want the low quality, right?

iirc the data in progressive mode is laid out in such a way where it can be compressed better with the downside being it takes more effort to decode and also won't work in low memory situations since space for the whole image has to be allocated when decoding a progressive JPEG

Yeah i wanted to see how much better mozjpeg’s worst quality is than whatever GIMP uses for jpeg

Jpegli shines when -q is at least 75

JPEG images below quality 75 are generally crap however

Traditional jpeg encoders are kinda dumb, they just cut off preset amount of data, so there was certainly room for improvement.

<@853026420792360980> still don't fully understand what avtransport is, it's used in ffmpeg as some internal protocol, but can also be a streaming protocol and even has some separate container like e.g. nut (and this container is better/more advanced than nut?) but, still not fully used with all features because not standardized?

it's a streamable container and protocol

also, is avtransport container not usable in ffmpeg yet? just through avcat?

Is there any requirement that luma and chroma channels should use separate quant tables in jpeg? I've come across some jpegs where the quant tables are two identical copies.

The jpeg optimizers could reduce them into one. I tried to manually edit the star-of-frame so all channels used the quant table zero and everything seems to work alright.

ECT removed the now unused quant table as well.

No, there is no such requirement.

So having two identical quant tables in the file is unnecessary and the optimizers just don't handle this particular case. Some encoders also put quant tables in separate DQT markers. That's unnecessary as well because one DQT may include multiple tables. ECT will combine multiple DQTs into one.

Yes, I guess some optimizers don't care about such micro-optimizations, but you're right.

yep. I made my ffmpeg svt-av1 fork which has overboost and my own feature function. This is all I'm doing because of the developer's fix, the shit https://gitlab.com/AOMediaCodec/SVT-AV1/-/commit/128712bf0aa83e4f680352b87c4b75ac3419364d

aq 5-9 for overboost and for my func 9-19

<:PepeOK:805388754545934396> Unfortunately, that function i did looks worse. therefore, I will leave only the overboost (variance boost = 5-9) of the original function

I remember seeing the PR in OBS a few months ago, guess I forgot to follow it because it got merged a while ago https://github.com/derrod/obs-roi-ui

<:Poggers:805392625934663710> <https://encode.su/threads/4272-New-approach-to-audio-compression> > **New approach to audio compression?** > We are brewing something new: > <https://github.com/google/zimtohrli> is the "butteraugli" for audio > > <https://github.com/google/tabuli> shows the possibility of modeling audio with a sample-to-sample sliding FT, where higher frequencies have wider bands and shorter windows. > > <https://github.com/google/ringli> for our prototype audio compressor. I think it is one of the few that doesn't use the MDCT and aims at higher quality than traditional codecs. In our internal testing we see ~2.3x more compression near the psychoacoustically lossless domain.

I'm playing with cjpegli. I've noticed two unusual things. cjpegli uses three quantization tables (jpegs usually have one for luma and one for chroma) and it doesn't write JFIF marker (the main purpose of JFIF is to ensure the color jpeg is YCbCr).

Probably leftover from when it subsampled the blue in XYB too

This? https://github.com/libjxl/libjxl/issues/3512

I've used the default settings so I suppose it's YCbCr.

I've came across APP14 marker in RGB encoded jpegs, however cjpegli doesn't seem to support RGB encoding.

well, I think you could use it: ``` $ cjpegli -v -v -v -v -h -x key=value, --dec-hints=key=value color_space indicates the ColorEncoding, see Description(); icc_pathname refers to a binary file containing an ICC profile. ```But never understood where is their `Description()` Also, when you use `cjpegli --xyb`, by default it will use the RGB colorspace in addition to the XYB ICC

Sorry, my post in not very clear. I meant RGB encoding, where the color transformation is skipped and the RGB channels are encoded directly.

I see, XYB requires ICC otherwise the image will look wrong. And the third channel is indeed subsampled for some reason.

In YCbCr encoding, the chroma subsampling is turned off by default.

Ah I see, the channels in XYB encoded jpeg are tagged RGB, so the decoder will not do the YCbCr-RGB conversion and the ICC presumably does the conversion instead? Sorry I don't really understand the XYB concept but this makes sense.

Yes, that's how it works. But I thought we stopped subsampling B because that creates jpeg files that cannot be recompressed with jxl? Or did that never happen?

The last build is still subsampling B but it can be turned off manually.

Conversion of greyscale images by major codecs is more bizarre than my expectations. I'll test further

I had issues when a supposedly grayscale PNG used a palette of 256 shades of gray. Such png may be treated as RGB color image.

Shouldn't it be considered a bug that jpegli produces XYB jpegs with chroma subsampling? Worse that it does it by default?

Agreed. After all, the default is supposed to be "visually lossless" and it doesn't make sense to subsample just one component anyway.

in XYB, it kind of does

But it apparently breaks compatibility, jpegs with one subsampled channel cannot be transcoded to JXL. cjpegli uses a separate quantization table fo each channel so tweaking the quantization table for the third channel might be a better solution than the subsampling.

Even ordinary greyscale and screentone greyscale can be very different among codecs

In terms of file size, supposed to be lossless Greyscale: QOI > PNG > HEIC > AVIF > WebP > JXL Greyscale + Alpha: QOI > HEIC > JXL > PNG > AVIF > WebP Screentone: HEIC > QOI > JXL > AVIF > WebP > PNG Screentone + Alpha: HEIC > QOI > AVIF > JXL > PNG > WebP

Only JXL preserves the Gray colour space from PNG

Interesting for screentone, what are your source images?

Hidden for potential anthropomorphobia

They were initially converted via Clip Studio Paint Ex

It's a long time technique for printed comics

Thanks! Very very small grain but it is the first time I see some which are not from scanned contents 😅

At the same time, dots here are sharper than traditional manga screentones

I simply used the default settings, number of screen frequency=60 and angle=45

Possibly printed results could be different

It is some screentone you generate from a greyscale image?

I didn't know anthropomorphobia was even a thing...

No they were directly converted from the original true-colour image

QOI doesn't support grayscale. WebP lossless encodes everything as RGBA but does pretty good job compressing it.

h265 should support grayscale in theory

AV1 as well

Screen tone is interesting. I expected it would be 1 bit per pixel.

It *used* to be fixed actually, not sure when it was reverted https://github.com/libjxl/libjxl/issues/2284

With different densities and at least three grey levels, screentone can be filled everywhere

There used to be jobs for cropping and pasting screentone to comics in the past

And WebP dumps the original DPI

The preview panel of Finder on macOS can't handle screentone at all, even PNG

I even can't be sure about if JXL really has issues with that

It performs well only with ordinary greyscale but without Alpha

Predefined effort settings in JXL have mixed results.

I used only `cjxl -d 0` for them

This seems to be an issue with downscaling filter.

It even shows an animation initially

it's quite challenging to avoid aliasing/Moiré when downscaling such images

If I open it in Chrome (well, Thorium) and go through the various zoom levels, this is what I get at one point

Yeah it's even difficult to get a proper view

This style looks best if downscaled by half, quarter etc.

25% 50% 75% 100% the best

Yes downscaling is a big issue

Speaking of DPI, JXL doesn't seem to preserve DPI information either. DPI is mostly important for printing so if the images are intended for printing, the DPI information should be preserved.

if you put it in Exif or XMP, it will be preserved

It's hard to determine when DPI info is actually important because so many pictures have DPI tags filled with nonsense just for the sake of it.

I got the screentone JXL down to 590 KB with `-d 0 -e 10 -g 3 -E 3 -I 100` but still worse than PNG and took 4 minutes

Still a significant saving from around 2 MB

So you even added three modular mode options

Without `-e 10` the file size is even larger

The same `-d 0 -e 10 -g 3 -E 3 -I 100` too 155 seconds for me

`-e 9` still does fairly well and faster

About 100 KB larger but it took 39 seconds

this a human eye?

yes, the cones

specifically, JXL does not support RGB JPEGs with chroma subsampling, only YUV

and XYB is for some reason stored as an RGB JPEG

XYB effectively acts as its own “YCbCr-like” transform; you’d arguably not want a further such matrix transform on top

That's what I thought. RGB encoded jpegs don't use chroma subsampling at all, it would look worse than YCbCr.

Would it have to be on top?

The "RGB encoding" of XYZ is a workaround, so no "RGB<>YCbCr" conversion takes place.

The jpeg decoder doesn't have to know anything about XYB and the color management does the conversion thanks to the included ICC profile.

is this true? I thought `jpeg_upsampling` needed ycbcr but not `upsampling` or `ec_upsampling`

I suppose `upsampling` applies to all three color channels at once.

ah I see

Yeah, it's true. RGB JPEGs are very uncommon and even more uncommon for them to have chroma subsampling

since it makes no sense normally

Unless it's not actually RGB

Any combination of subsampling can be enforced in jpeg, even subsampled luma rather than chroma, looks pretty funny.

Speaking of screentone, it's interesting to see that a huge part of the patterns used by mangakas are far from a simple halftones, used as a printing process. Screentone were used as a way to fill large areas very fast, but not always with "halftoned uniform grey" https://b4comics.com/collections/screentone-trame-traditionnelle-manga?ls=en&cache=false clouds/skies/backgrounds and motion lines are very frequent also

Softwares can now replace them completely but encoders and viewers can't deal with well

Repeating patterns might be an opportunity for using patches in JXL.

I would like some explanation about why JXL doesn't perform well on such image

And why libheif makes a clone

Lossless JXL usually works great, however it doesn't like particular types of images and requires tuning compression parameters to achieve good compression. Lossless WebP gives decent results consistently and the slow setting is still pretty fast. The HEIF family is buggy and unreliable, often decodes incorrectly. I don't like it.

It would be interesting to see the sourc code of Pngout. It has some interesting quirks. Especially Pngout compression is influenced by the encoder that encoded the original png so there may be different results. I was looking for an open source alternative, oxipng looks very promising. Thanks to the zopfli compression it usually gives better results that Pngout. However in some cases, especially flat-colored artwork, Pngout performs better compression.

Oxipng supports zopfli and libdeflate. Curiously, libdeflate is sometimes better than zopfli.

<https://github.com/fhanau/Efficient-Compression-Tool>

This works great for jpegs. I've tried it on few pngs already optimized with oxipng and couldn't optimize them more. They both use zopfli so I suppose there's not much difference.

And I like more options available in oxipng.

oxipng has much less optimization strategies, ect still has the most efficient and comprehensive own and implemented from other optimizers methods and -9 is not the maximum optimization, there are some hidden options (but very slow), though that doesn't mean that certain png's are possible to optimize further after some optimizers

ECT is, as it says, efficient. Much lower memory usage and time spent compared to oxipng for equal or higher compression

ect can also be very slow with some settings and spend days, weeks, months on a single png <:KekDog:805390049033191445>

I tried a dozen of optimizers, and ECT is the one that can reach the smallest files (given enough time) for any Deflate data (GZIP, ZIP, PNG)

the only limitation I saw is the maximal filesize, which is 1.2GB iirc but then just need to use another optimizer, like this one: https://github.com/MrKrzYch00/zopfli (a mod of zopfli) Running `./zopfli --help` prints```Floating point arithmetic precision: 64-bit Maximum supported input file size: 17592186044415MB.```

but it is beaten quite easily by ECT

I imagine it would be relatively easy to split the data and run ECT seperately, then combine the results... Somehow....

I've tried only ect -9 so far

well `ect -9 --mt-deflate` gives almost always smaller output, even for small files and it is faster <:KekDog:805390049033191445>

I will try the advanced options.

In jpeg optimization in ECT, I discovered a possible improvement. Sometimes I come across jpegs, where luma and chroma use identical quantization tables, so there are two identical copies stored in the file. ECT could compare quantization tables in the file and reduce the duplicates.

I can manually edit the start-of-frame header so all channels use one copy of the quantization table and ECT will remove the unused copy. Everything seems to work without issues.

Currently having to 7z all my raw files because for some reason the 'lossless' DNG converter changes the decoded output and always embeds a preview even with the option disabled

At least it's almost done....

I must be missing something obvious but how does 7z-ing them solve the problem?

I assume the point of using the lossless converter was to save space

ah, somehow I assumed that the source raws were not DNGs themselves

Yeah, it's 529 uncompressed DNG raws that total around 12 GB, I know my phone can read lossless compressed DNGs too, but whenever I try to convert them the colors are shifted and a 200 pixel wide jpeg preview is embedded, so 7z it was...

Mostly done for redundancy, since I have them on my PC with filesystem compression too, but having them on the phone locally too will be useful if I need them

How does 7z compare to Brotli?

more of a question of "How does LZMA2 compare to Brotli?" since 7z is just a container

funny thing is since 7z is just a container you can use Brotil in a .7z and people have

Original file was 22.5 MB Brotli got 9.24 MB at max compression in 68 seconds 7z is 8.78 MB in 10 seconds (Single file, so not LZMA2's strongsuit either) JXL is 7.42 MB in 3 seconds through the Adobe DNG converter (Although effort settings are ignored, so only 7)

Also sorry <@98957339243073536> I got distracted doing this :P The DNG converter is a special kind of hell made just for me, almost no documentation, no help command and the order of parameters matter with some being incompatible with no warnings or errors other than "Missing File"

Hmm, it appears cjpegli quality number for quality number is worse than paint.net jpeg exporter with 4:2:0 However the size of the output is much lower with the same quality number It means I can have a higher or similar apparent quality image with the same or less file size, however I just have to shoot higher with my quality specification

quality scales differ between different software / encoders, e.g. mozjpeg and libjpeg-turbo are also not quite the same

Jpegli requires slightly higher

Quantization tables in JPEG may optionally use 16 bit constants rather than 8 bit. I wonder why would anybody use 16 bit quantization constants. Quantization constants at 255 are already so strong so the quality is unusable. I see no purpose for higher quantization constants.

How are the 16 bit constants treated

I would assume it’s just an expansion of dynamic range to give more precision in the quantization

I dont think it’s likely the 16 bit values are treated as extensions to the 8 bit quantizers

E.g i think 65535 maps to 255.0

But you get finer control over the quant

<@688076786525143117>

It's an extension. Higher values decrease the quality further. The max value is 32767 and this just wipes the image entirely.

😳lol that’s odd

Would it possibly not do that if the jpeg had a higher bit depth

Perhaps the idea was originally like that.

you'd *think* so, yeah

Strange design

12 bit jpeg with 16 bit dct is in the standard afaik

In baseline jpeg, quantization tables contain 8 bit constants.

Now I'm really confused by the 12 bit jpeg thing. There is a bit depth field in the Start-Of-frame marker and it says 8 bits. Even if I take a 16 bit PNG and encode it using cjpegli, it still says 8 bits.

So my previous testing of supposedly 12 bit jpegs are invalid.

Both SOF and DQT indicate 8 bits inside jpegs created by cjpegli.

even though 12 bit JPEGs are defined in the official spec pretty much nothing supports them so we are stuck with 8 bit JPEGs because 12 bit ones wouldn't work with software made over the course of decades

they are incompatible with almost everything because no one thought it was necessary to implement support for them

that's why jpegli does not do anything related to them

That would make sense, but people say jpegs are actually 12 bits.

So I did another experiment. I took a completely basic grayscale jpeg, decoded it using djpegli to 16 bit PNG a it has indeed more than 256 shades of gray. So these had to come from somewhere.

This still isn't clear to me how it works exactly, but considering jpeg being 12 bits and observing that 8 bit quant table is enough to completely shred the contents of the image, 16 bit quant table is simply useless and that's my point. 😉

8 bit jpegs use 12-bit dct coefficients and 12 bit jpeg uses 16 bit coefficients

Ah that makes sense. So it's just overhead to improve precision. I suppose decoders are free to use the full depth of DCT coefficients but most of them didn't do it. Anyway, I think 8 bit quantization tables would be enough for high quality 12 bit jpegs as well. Considering that q=90 the constants are around 20, so for 12 bit, the equivalent would be 80. But I guess they just added the option for those who want to make heavily compressed HDR images.

Actually I've noticed that cjpegli uses coarser quantization values than cjpeg for chroma, presumably as a choice instead of chroma subsampling. That could actually make use of 16 bit quantization tables.

It still has chroma subsampling, it just doesn't change based on quality level

It's turned off by default.

8-bit jpegs are internally 12-bit, 12-bit jpegs are internally 16-bit. Probably the 16-bit quant tables are only useful for 12-bit jpegs — though I imagine you may want to use high quantization factors for the highest frequencies also for 8-bit jpegs when doing low quality encoding — a factor like 300 or 400 is still meaningful when the coeffs themselves are 12-bit.

Thank you for clarification. djpegli seems to use all 12 internal bits while decoding an "8 bit jpeg" to 16 bit png. I don't see any way to select the "12 bit jpeg" in cjpegli as 16 bit png will encode just into "8 bit jpeg".

yes, jpegli only encodes 8-bit jpeg — 12-bit jpeg is not commonly supported by anything so if you go there, you lose interoperability

interoperability is the main selling point of jpegli; if you don't care about that, it's better to use jxl instead

Thank you for explanation. That also explains I could decode jpegs produced by cjpegli with ancient decoders. If both encoder and decoder attempt to use the full 12 bits in 8 bit jpeg, could they reach more precision than 8 bits?

jpegli uses 32-bit floating point math internally

and I believe libjpeg-turbo uses 16-bit integer math by default actually

So in principle it's possible to get higher than 8 bit precision in "8 bit jpeg" if the source is more than 8 bits as well.

it's a bit more complicated than that

The DCT doesn't result in even precision

A very noisy 8x8 block will have an effective bit depth that's *lower* than 8 bits.

While a smooth gradient can have up to 10.5 bits of precision

12-bit DCT was chosen so that the inherent loss of the DCT wouldn't be *too* bad in any situation. As a side effect it can be better than their goal in other situations.

I knew the extra bits gained after decoding a jpeg encoded from 8-bit source are just compression artifacts.

mm

One just can't tell if the extra precision is valid or not.

jpegli tries to make use of that extra precision, since banding is most visible in smooth areas, where the effective bitdepth reaches 10.5

Sure it's better to try to use the maximum of the available precision.

Even the 16 bit quantization tables might be useful in some cases. cjpegli by default uses a few values around 200 in the chroma tables and these could go over 255 in lower quality settings, although 16 bit quantization tables are not compatible with baseline jpeg.

ECT can save a few bytes in pngs already optimized by oxipng. Although I came across a file that was already perfectly compressed by pngout and ect couldn't improve it. Deflate is simply hit and miss. I'm trying both `--allfilters` and `--allfilters-b` but the latter doesn't seem to do any better. I will continue experimenting.

On what image though? <:thinkies:854271204411572236>

time to blow up our PCs [⠀](https://cdn.discordapp.com/emojis/1194236343220441099.webp?size=48&quality=lossless&name=UwU)

you know you can use slower compression settings like `-9xxxx` right ? Don't go with `-99999` tho <:KekDog:805390049033191445>

also, use `--mt-deflate`

Didn't know about more 9s, the help suggests just one. --mt-deflate is juts to enable multithreading? oxipng uses multiple threads by default.

yeah --mt-deflate is multithreading, and almost always gives smaller output (than without)

Anyway, I don't think it's worth crunching a single image for hours just to save a couple bytes. But I will give it a try juts for fun.

well you could go with simple `-99`, ..., `-90050` they are far from "encoding during hours"

You mean I'm underestimating?

overestimating [⠀](https://cdn.discordapp.com/emojis/895863009820414004.webp?size=48&quality=lossless&name=av1_thinkies)

Just -9 took half an hour.

With the multithreading it scales almost linearly, and --allfilters-b does add a lot more slowdown in my experience

Yay, `-99 --allfilters-b --mt-deflate` have beaten pngout finally.

Why not? <:KittyLaugh:1126563616343216268>

because **he** does not want to encoding during days <:KekDog:805390049033191445>

I still need to find what you did to better compress after jpegultrascan [⠀](https://cdn.discordapp.com/emojis/654081052108652544.webp?size=48&quality=lossless&name=av1_Hmmm)

All you need to find out is to donate some pizzas <:KittyPizza:1147750033018605680>

well I can, but you need to go down here lol

pretty sure it's just Huffman tables

but then I need to find the right tool

why would you even do --allfilters <:KekDog:805390049033191445> that's just a waste of time compared to what you want

Perhaps someday. Then I do expect one of those XL family pizzas though <:KekDog:884736660376535040>

~~And the answer to why my jpegs are slightly smaller will disappoint you lmao~~

[⠀](https://cdn.discordapp.com/emojis/890866831047417898.webp?size=48&quality=lossless&name=SadCheems)

~~JXL Family Pizzas~~

Because I don't know what I'm doing 😄 I thought allfilters means try all filters, to find the best combination, right? Anyway. Is there any clear explanation, how the level settings -9...99999 works? it seems that different parts of the numbers are assigned to different parameters internally.

yeah allfilters tries all filters obviously, but your saves will be negligible compared to increasing `-9xxxxx` iirc the first number is use for ECT to know which compression "mechanisms" to try and others are number of zopfli (et al.) iterations but I'm not sure

I think each number corresponds to a different setting inside ECT, based on people changing each digit at a time to get savings rather than just higher number=better

```-# Select compression level [1-9] (default: 3). For detailed information on performance read Performance.html. Advanced usage: A different syntax may be used to achieve even more compression for deflate compression if time (and efficiency) is not a concern. If the value is above 10000, the blocksplitting-compression cycle is repeated # / 10000 times. If # % 10000 is above 9, level 9 is used and the number of iterations of deflate compression per block is set to # % 10000. If # % 10000 is 9 or below, this number specifies the level. --pal_sort=n Attempt to sort the palette of PNG images. Only has an effect on files where palette may be used. n specifies the number of different strategies tried, the maximum is 120. Multiplies the compression time. --allfilters Try most png filter strategies. Multiplies the compression time. --allfilters-b Try all the filter strategies of allfilters and also the ‘brute force’ genetic filtering, which is even slower. Genetic filtering may be ended by the user, more information on the command line. --allfilters-c Try several png filter strategies. Cheaper than regular allfilters, but still improves compression. --mt-deflate --mt-deflate=n Use several threads to compress png, gzip, and zip files. Multithreading is not used on very small files. If a number is specified, n threads are used, otherwise the number of logical cores is used. May decrease compression ratio. ```

```fhanau: You could try replacing -10080 with some bigger numbers, e.g. "-20100" or "-30140" or "-50300" or "-60500", although I haven't tested these settings. Note that --mt-deflate may decrease the compression ratio, although this should not happen when using very high settings.``` so there are some balanced values, higher than -9, but not so slow, like: `-20100 -30140 -50300 -60500`, some optimizers use `-30060` as the most optimal one

So `-30060` means 3 full cycles of 60 iterations at zlib level 9, if I understand correctly? And `-99` is just 99 iterations at zlib level 9, right?

seems so

Alright. Thank you.

huh, turns out ECT used to have mp3 support too, but was disabled due to metadata corruption

It's also 90% if statements for every option https://github.com/fhanau/Efficient-Compression-Tool/blob/master/src/main.cpp

Yeah I've tried to read the source but got lost.

Some people do use `-30060 --allfilters-b --pal_sort=120` together.

yeah

I suppose those heavy settings don't work on jpeg.

It seems ECT considers only modes 1...5+ in jpeg compression.

I'm sure that the arrangement of scans and Huffman could be brute forced too.

So, -99 can be better than -30060

just -99 probably doesn't work, because it's supposed to be `If the value is above 10000`

-99 does work ```$ ect -9 -gzip --mt-deflate in.gz Processed 1 file Saved 8.16KB out of 8.72MB (0.0914%) $ ect -99 -gzip --mt-deflate in.gz Processed 1 file Saved 27B out of 8.71MB (0.0003%)```, same as -00099, so no blocksplitting-compression cycle repeat

then maybe `-90099`? or yeah, just -00099

Or I could do something like -30100

seems like higher than 60 there are significant slowdowns without much improvement, so it's not just a random number and also not always higher is better, except for `-99999`

-60 couldn't compress the file already optimized by pngout, -99 can.

now, I wonder if going above 9 for blocksplitting-compression cycles would be nice

for some already highly optimized images maybe, but like for typical random images it might not be very optimal or only slightly better with much slower optimization

Yeah I suppose -99999 is something like -e 11 in cjxl, you can use it if you want to turn the PC into the space heater.

I've discovered that pngout works the best for flat colored images, where the PNG filter "none" is the optimal filter.

An these are then hard to compress by anything else.

PNGs using mixed filters can be usually compressed by oxipng or ect pretty easily.

According to the documentation, `--mt-deflate` may decrease the compression ratio. Shouldn't it be better to run ECT without multiple threads?

https://tenor.com/view/try-it-and-see-give-it-a-try-try-try-and-see-try-it-yourself-gif-24095805

But from my tests, --mt-deflate gives almost always a smaller file

and it's far faster

If I recall, --mt-deflate tries to guess where block boundaries would be best, and then can multithread based on that as a very nice bonus, but sometimes it won't find any splits and will stay singlethreaded

--mt-deflate seems to be indeed slightly better. I don't understand why they tell the opposite in the docs.

I see, `--mt-deflate` can actually decrease the compression ratio, sometimes it produces a file bigger by 1 byte, that's really significant difference. 😄

What is the best way to encode a jpeg? It can either be preferably 2160p, but also 1440p or 1080p It needs to be 250KB or less WHat should I use and what settings should I use

I would prefer lower resolution over weird blocking

use jpegli which comes with the libjxl binaries: https://artifacts.lucaversari.it/libjxl/libjxl/latest/

it's `cjpegli`

I've also heard about mozjpeg

jpegli is intended to surpass mozjpeg

use XL converter. you can select the desired final size in this app (and downsample/resize first) as well as with the official libjxl's terminal commands for cjpegli

using distance should be preferable, starting with 0.5 unless you require an exact filesize, try slightly increasing distance values in 0.1-0.2 increments

Very cool python shit with QT6 awesome

gotta do the funny

`cjpegli` can targeting the size: ``` > cjpegli --target_size=250000 8.png 0.jpg Read 3508x2480 image, 2325205 bytes. Encoding [YUV d1.000 AQ p2 OPT] Compressed to 250164 bytes (0.230 bpp). 3508 x 2480, 15.401 MP/s [15.40, 15.40], , 1 reps, 1 threads. ```

cursed ass program

Can't even debug it

jpegli uses "RGB jpeg" to store XYB, but is it possile to skip the XYB transform and store RGB diretcly? There are niche use cases, where any color transformation is undesirable. Different quantization tables are also used for each of the XYB channels but "RGB jpegs" usually use the "luma" table for all three channels.

XYB is opt in, it already doesn't use it by default

but if you don't use XYB, it uses YCbCr

JendaLinda wants to store RGB

you should be able to use RGB encoding with this arg ``` -x key=value, --dec-hints=key=value color_space indicates the ColorEncoding, see Description(); icc_pathname refers to a binary file containing an ICC profile.```but idk where is that `Description()` <:KekDog:805390049033191445>

Will I get RGB with XYB quant tables? 😄

I think I will be using jpegli only for YCbCr for now.

that’s unrelated to RGB vs. YCbCr JPEG, it’s just the colour profile of the de-YCbCr’d-if-needed pixel data

(sRGB or Display-P3 or ProPhoto or…)

ohhhhh

[⠀](https://cdn.discordapp.com/emojis/1194236617813147729.gif?size=48&quality=lossless&name=NOTED)

the format is indeed a bit cryptic but it’s basically a series of 3-character identifiers that specify the type of colourspace (RGB vs. greyscale), the white point, the primaries, the rendering intent and the transfer function

for example, sRGB with relative rendering intent is `RGB_D65_SRG_Rel_SRG`

Rec. 2020 + PQ and perceptual rendering intent is `RGB_D65_202_Per_PeQ`

Display-P3, `RGB_D65_DCI_Rel_SRG`

etc.

thanks [⠀](https://cdn.discordapp.com/emojis/674256399412363284.webp?size=48&quality=lossless&name=av1_pepelove)

I thought this only sets the color profiles but doesn't change the behavior of the encoder. Anyway. XYB requires an ICC profile for decoding. What happens if the source image already uses non standard ICC profile? Will the ICC profiles be somehow merged together? Or can ICC profiles stack on top of each other? Once I came across an image, where there was an error during editing and the ICC profiles had to be applied twice in a row to get the correct colors. This had to be done manually as I didn't find a way to double the strength of the ICC profile.

if an image with a custom ICC profile is converted to an XYB JPEG, the colours are converted from the colourspace described by that profile to XYB, instead of from sRGB

it’s nothing fundamentally different

(although I’m not sure we “scale” XYB to make room for wider gamuts)

Ah so the original colors are moved around according to the original ICC profiles already during the conversion to XYB, so the conversion from XYB to RGB doesn't need to take the original ICC profiles into account, right?

correct

XYB is absolute and is therefore able to uniquely identify the absolute colours represented by the original pixel data as interpreted using the original ICC profile

So there's the advantage of XYB JPEG. Having normalized colors, so no more fiddling with different color profiles. And if something doesn't work correctly, one can tell on the first glance.

Wide gamut just means X and B have a bigger range than what they have when the source is sRGB, just like HDR just means Y has a bigger range than when the source is SDR. More or less.

right, but for JPEG/ICC, we need to map some specific range of X and B to 0-1, and I have some vague memory of <@179701849576833024> and I doing that for the range spanned by sRGB, but I’m not sure we ever changed the mapping to be dynamic based on source gamut

I also have a vague memory of that

Ah, I see. Didn't you just make the range large enough to cover P3 or something? I don't think you need much more for that...

People have made animated pixel art movies in gif format that are several minutes in length. Gif apparently can handle animations of hundreds or thousands of frames without problems.

https://maskray.me/blog/2024-04-23-when-qoi-meets-xz

can't they compare it to jxl ?

they can only use toy formats

(oops, this is larger than I thought)

I thought QOI might be useful for embedded system like Arduino or so, but QOI doesn't support low color graphics, so RLE it is.

Yep the results aren't that impressive when comparing to BMP or TIFF

I thought PCX might have been a fair comparison.

HEIC should be tested as well

Alright, here goes HEIC as "near lossless" ` Image 1 20009 bytes Image 2 27925 bytes Image 3 117341 bytes ` No idea what settings Gimp uses.

Tested grayscale before and HEIC was the biggest

HEIC is completely broken in Winddows 11. It decodes as white rectangle.

I did some tests with jpegli. Even at distance 0.5 jpegli tends to blur chroma so it has similar effect as choma subsampling. I think at very short distances chroma shouldn't be blurred that much.

Just like libjxl did?

I'm not sure but does the typical decrease in color saturation around black lines.

does this not just ruin the point of using qoi in the first place?

Better to compare the speeds as well

QOI is a nice experimental pilot in designing simpler data formats that don't require years of study to understand. Hopefully its shortcomings can one day be addressed in a way that doesn’t make it substantially more complex. Perhaps a qoi2

There is clearly real demand and value in simplicity

if anything it is a great way for people to get into compression

c:

A great toy project... But if a more complex format can compress better and faster, it means simplicity is not the only meaningful criteria

The author clearly announced that version 1.0 is the final

There have been many forks however

the QOI format supports other colorspaces but the spec only defines sRGB and nothing else which sucks https://qoiformat.org/qoi-specification.pdf

GameMaker has implemented QOI for long time

Google Earth Pro has QOI support strangely enough

For what...?

I think people want formats that are simple yet good enough. Only time simplicity should be sacrificed is if it leads to significant benefits and it usually doesn't. For example, most of the bitstream features of jpegxl, like spline detection and removal, or detection and separation of an image into multiple layers for more efficient encoding, are probably not going to be used by any actual encoder any time soon because of the work and complexity involved

Even though theoretically it could improve quality and efficiency of lossy a lot, and layer detection might even benefit lossless encoding by for example detecting and separating solid shapes and gradients

if you want something simple just use png

I think qoi was made because png isn't simple

it's just throwing bytes into Deflate

No, That would be farbfeld

:)

yeah yeah it has headers and a tiny bit of filtering too, big deal

just use ppm.gz if you want something extra simple

png deflate is already way simpler and better than qoi

people dont know what they want

Someone wanted something even simpler but still good enough for their purposes, unfortunately they left some very low hanging fruit on the table probably

Like for a lossless format it would be nice to store HDR images, floats, and both types of alpha

for exporting in the "movie maker", QOI was added in version 7.3.6 https://support.google.com/earth/answer/40901

Anyone know how to get the ffmpeg encoder commands used in a video? Like I wanna replicate the encoding from one video to apply it to another file

the question is unclear ... what did you tried ?

(and this discord have users and fans of ffmpeg but we are more versed into still images)

real answer is you can only do that if the person tagged the video with the commands

so you just look at the tags

the settings are not automatically stored in the tags anywhere at least for most codecs

Afaik

Uh I didn't know this so-called "Movie Maker" either

Does it internally convert to YCoCg-R for lossless compression? Because that's usually some very cheap compression gain...

Nothing beats RLE in simplicity.

<@219525188818042881> by any chance, do you know a Webp optimizer other than cwebp ? I don't even know if lossy-webp files can be losslessly recompressed [⠀](https://cdn.discordapp.com/emojis/654081052108652544.webp?size=48&quality=lossless&name=av1_Hmmm)

No sadly. I never really use WebP

Pingo claims it can optimize it, but haven't tried it yet

oh, gonna try it thx

only thing it can is to convert PNG (perhaps others?) to Webp <:kekw:808717074305122316>

Should also be able to optimize lossless webp

Or well, may or may not result in smaller files than cwebp. Might try if it's any better when I'm home <:KekDog:884736660376535040>

~~I would be disappointed if it's much worse than cwebp lmao~~

well at least on my end it keeps crashing for webp input

but perhaps bc of wine/linux

Lmao

Wouldn't be that surprised if it only supports png and jpg input

Oh hey, I'm pretty close to your freezer where your pizzas are

[⠀](https://cdn.discordapp.com/emojis/853492103145324554.webp?size=48&quality=lossless&name=woag)

Why do people still use Windows icons in bitmap format? PNG icons are supported since Windows Vista. It would make sense if the programs ran on Windows XP as well.

In Gimp, it's just a matter of a checkbox.

most programs and tools only output Windows XP compatible .ico files and the people making .ico files don't know the history or features of them and just see ico files as a collection of multiple images in different sizes and nothing else and I've also seen instances where people don't even know that

Windows XP compatible in the sense that only the higher resolution sizes that aren't supported by XP are PNG while everything else is bitmap although I have seen some cases where every size is bitmap but that's pretty rare

also I would highly recommend these tools when doing work with pre-existing ico files: https://github.com/jtippet/IcoTools

I don't know what software people use for drawing icons. It's apparently two decades behind.

Discord is one example where all icon sizes are in bitmap format.

GIMP is one of the only existing pieces of software that allows deciding what is and isn't a PNG

even programs dedicated to icons don't support making lower sizes PNG

it's a suprsisingly rare feature for software that allows creation of ico files

also Discord is a beacon for everything not to do when making software

so I'm not surprised they somehow managed to make a horrible ico file that's worse then what almost every single piece of software outputs

kinda like how they somehow managed to break one of the core fundamentals of Chromium's/Electron's architecture

and also how every single time you open Discord it wastes disk writes re-creating the same PNG files

This looks great. I was making some icons some time ago and I had to write my own tool to pack the pngs into ico.

Gimp could make png icons but there were not thoroughly optimized.

what exactly? while I hate electron I need more things to hate about discord

one of Chromium's main design points is it's multi-process architecture and one of it's core intentions/benefits is stability where if one of the non-main processes crashes or goes down then it can just be restarted without taking down the whole application however Discord somehow manages to break this, Go to pretty much any Chromium based browser or Electron based app and try killing the GPU process and watch as they all recover just fine meanwhile with Discord the entire thing goes down.

Discord is one of the worst pieces of software yet it's on a very **very** very large amount of people's computers and is used every single day

the total combined amount of computer resources that have been wasted with all of Discord's incompetence must be astronomical

that I can agree with

although I would say that applies to nearly every piece of electron based software (except Code), but discord manages to stand out even amongst them

yeah Discord goes above and beyond to not just be Electron levels of bad but somehow even worse

I just recently got a new PC and it's incredible for me to see discord use more CPU AND GPU than Doom Eternal at maxed settings with raytracing <:facepalm:884892069225717840>

odd, I’ve never observed anything like that

I don’t use its video streaming features much, though

one of the other bad things about Discord is that every single time it updates Windows considers it a different program meaning that stuff like data usage and tray preferences get a new/different entry or reset

ah, yeah, I also have to add it to VPN every time as well

because it just creates new folder for every update

even worse it sometimes moves between ProgramData and AppData between updates randomly

for no apparent reason

I think Chromium based browsers have a similar update procedure yet don't have the same problem(s)

there's probably some extra steps that they do that Discord just flat out doesn't

this is awful.....

Ah so that's why Discord icon in system tray is still moving to hidden icons. Discord was also disguised as Github updater in the startup program list, at some point.

Um... Hot take but personally I think if you're foolish enough to think it's a good idea to "install" discord in the first place you kinda deserve what you have coming to ya...

I can't believe so many of you apparently didn't see anything wrong with doing so.

You should just have 1 browser window for all of your electron-based apps.

well at least dissent is comming eventually

its mostly usable

Have you heard about JPEG Quant Smooth? It does pretty good job cleaning non-photo images that were "accidentally" saved as jpegs.