JPEG XL

one time I used an implementation of NIST's recommended randomness tester, on a 1GiB randomly generated file

OOM-killer saved me after it ate up all my 80 GiB of swap

btw, it's still <#803645746661425173>

lol

ill run `paru -Syu` and it saves me a lot

does mac have an oom killer?

i have no idea

probably not?

what happens when oom then? i usually have enough swap

linux is like the only OS on which asking the OS for RAM never returns an error, so it needs a way to recover from overcommitment gone wrong

this is one of those times when `Alt + .` comes in handy

tells the MathKernel to abort the operation

on other OSs, applications may choose to abort if malloc returns an error, or they may choose to try to turn off safely if possible

like what happens if i run this ```cpp int main() { char* array[1024]; // lazy allocation for (int x = 0; x < 1024; x++) { array[x] = (char*) malloc(10000000000); // 10GB } for (int x = 0; x < 1024; x++) { for(int y = 0; y < 10000000000; y++) { array[x][y] = 69; } } } ```

for example, if allocations fail in mpv, it just aborts

since the assumptions are that a video player can't operate under conditions of low memory

and aborting wouldn't break something too badly

in ffmpeg it returns ENOMEM and it's up to the caller what they want to do about it

probably segfault or something when you try to use the null pointer returned from malloc

no but it doesn't actually allocate the 10GB, it's like a Sparse file

btw, returning to the earlier topic, for some reason I find this so funny a thing to read about a desktop-sized system: > 1024-bit octa-channel LPDDR5-6400

on Linux, yes. Not necessarily on other OSs.

it's likely the compiler will optimize the second loop to a memset but regardless of that, malloc returns NULL on allocation failure, and attempting to dereference NULL will segfault your process

mac it does as well

as bob beck once put it, "if you try to `strdup(NULL)` it goes BANG!"

even if it's sparse, there isn't necessarily overcommit

hang on lemme test

some offtopic is fine, but it's too much, would be harder to find some old comparisons and etc

let's move to <#806898911091753051> then

for posterity, convo continued here: https://discord.com/channels/794206087879852103/806898911091753051/1215098216454955122

For the love of Jon.

New compression results

interesting to see that breakdown by category — so it's mostly pixel art, text, icons/ui stuff where webp is still doing great and jxl is lagging behind... If you could send me a set of originals, I may want to take a look at some point at trying to improve some heuristics

Amazing! Do you classify manually?

HDblog uses jpg

Webp is used for exp photos in the 100 kb - 200 kv range

Yes, it is based on my judgement. I tag based on the primary composition of the image, but can also add tags for deviations, i.e. a screenshot of a video frame is film, one with some playback ui overlaid is film-ui. You could say all game screens are ui, but I didn't because it's very different content than typical app/web ui.

Amazing

``` wintime -- cjxl --disable_output -d 0 -e 1 OpenTTD.png JPEG XL encoder v0.10.1 5f67ebc [AVX2,SSE2] Encoding [Modular, lossless, effort: 1] Compressed to 784618.7 kB (5.195 bpp). 49121 x 24598, 153.367 MP/s [153.37, 153.37], 1 reps, 16 threads. PageFaultCount: 2810494 PeakWorkingSetSize: 6.833 GiB QuotaPeakPagedPoolUsage: 35.24 KiB QuotaPeakNonPagedPoolUsage: 67.06 KiB PeakPagefileUsage: 11.01 GiB Creation time 2024/03/08 00:44:28.366 Exit time 2024/03/08 00:44:46.819 Wall time: 0 days, 00:00:18.453 (18.45 seconds) User time: 0 days, 00:00:03.609 (3.61 seconds) Kernel time: 0 days, 00:00:49.609 (49.61 seconds) ``` An 80 MB, 8bit (Stored as 24bit) PNG. I'm surprised it used so much memory, seeing as even raw 24bit is under 4 GB

I don't remember for sure what djxl uses before writing the PNG, but it's probably floats

This is cjxl

``` wintime -- cjxl --disable_output -d 0 -e 1 Minimap.png JPEG XL encoder v0.10.1 5f67ebc [AVX2,SSE2] Encoding [Modular, lossless, effort: 1] Compressed to 566.5 kB (0.135 bpp). 8192 x 4096, 198.380 MP/s [198.38, 198.38], 1 reps, 16 threads. PageFaultCount: 52984 PeakWorkingSetSize: 196.5 MiB QuotaPeakPagedPoolUsage: 35.24 KiB QuotaPeakNonPagedPoolUsage: 8.492 KiB PeakPagefileUsage: 194.2 MiB Creation time 2024/03/08 01:17:27.999 Exit time 2024/03/08 01:17:28.362 Wall time: 0 days, 00:00:00.362 (0.36 seconds) User time: 0 days, 00:00:00.046 (0.05 seconds) Kernel time: 0 days, 00:00:01.015 (1.02 seconds) ``` A 8192 x 4069, 4bit file (Again stored as a 24bit PNG)

what if you use ppm and --streaming_input/output?

49121 x 24598 in raw 24bit is just under 4GB In 8bit (which is what the image actually is) it's just over 1GB So something is using up an extra 7GB or more with the streaming on, unless I've made a mistake

but neither of those has streaming on?

Streaming encoding is always on unless at e10

that's what I thought but apparently not

I forgot that streaming input effects the density too, more secrets to learn....

Shame PPM doesn't have 8bit, so I'll have to make a 4GB intermediate file out of a 80MB PNG to do it (And then probably end up with a worse result than just optimizing the PNG because of sprites and tiles)

also streaming input only works with ppm/pgm

Yeah which is why I'm dreading the 4GB PPM

you can pipe into cjxl, right?

<:YEP:808828808127971399> you can: `convert foo.png PPM:- | cjxl - bar.jxl`

didn't test it with --streaming_input though, but it will probably work

oh no, it doesn't <:AngryCry:805396146322145301> `PNM decoding failed.`

weird, I would think streaming_input would work with streams, but I guess not <:KekDog:805390049033191445>

how big is the actual png?

> 49121 x 24598

Ironically this was meant to be a 'small scale' test of this one 1572353 x 786777

I mean big as in file size...

> An 80 MB, 8bit (Stored as 24bit) PNG

ah okay, so not too bad

The data itself is in 8KB chunks like I said before, so could be a lot smaller. Already 98% compression ratio

did you do the massive res one yet?

No, I've only got 60GB free and that in raw would be 20GB alone, hence trying the smaller one first

Turns out I made a slight miscalculation

The full size image would be over 1TB in 8bit raw

3TB if 24bpp

phoronix test suite updated its encoder and decoder benchmark to 0.10.1 (had 0.7.0 before) in case anyone wanna contribute

just in time for 0.10.2

avx512 would also be interesting for these tests, but most likely it's disabled by default though, mostly for lossless e1, which is not tested

can anyone look if its parameters are alright? wondering if its going for the newer recommended defaults/efforts, modular streaming and more than 1 thread

https://github.com/phoronix-test-suite/test-profiles/issues/186

it might be a good idea to add the most balanced efforts for lossy and lossless and maybe even lossless e1 with like 50 reps

ah, it's not accepted https://github.com/phoronix-test-suite/test-profiles/pull/187

weird profiles, q90 or q80 doesn't make much sense for speed tests, same as lossy for png or jpeg and there is no other efforts

shouldnt encoding from a jpg source use the lossless reconvertible path? wondering as i see some conversion/export features in apps (like xnviewmp) doing full recompressions instead of going for almost instant compressions from jpg with guaranteed lower filesize

well for pts its still very early in the 1.6.0 cycle so maybe that could change?

Jpeg recompression has to be explicitly asked for when using the library rather than just cjxl

except q100, it's the same lossy compression even for jpeg if I remember correctly

The library API works on pixel buffers (or byte buffers), not on filenames.

yea, to pass a jpeg or png you need to bring your own decoder for that

and then pass the pixels

hello there I was curious about JPEG benchmark with differents encoders, like this one: https://github.com/google-research/google-research/tree/master/mucped23 https://raw.githubusercontent.com/google-research/google-research/master/mucped23/jpegli_study_elo_plots.png I read other benchmarks too on the same subject, and we can clearly see jpegli winning in all of them. But I come across some JPG "in-codec" optimizers (JPG input → smaller JPG output with matching checksum) that don't touch the metadata nor ICC So I was **really really really** curious to see if by using this kind of optimizer as a post-operation (aka running it on each output of the different JPG encoders) would **change the ranking order** Therefore I'm asking if anyone still has some JPGs from a benchmark result, and would be willing to share (as I can run these optimizers - and they might take some time)

Mostly to optimize losslessly within JPEG, what you can do is try different progressive scripts, and optimize Huffman tables (and use arithmetic coding, if you're ok with going outside the de facto standard)

Jpegli mostly does that already by default - not as exhaustively as default mozjpeg though, so with mozjpegtran you might be able to get some additional small savings

ELO score ? like in chess ?

apparently...

Electric Light Orchestra

oh, that's good to know. Perhaps why jpegli is winning is because it does better progressive scripts (?) for example with simple `cjpegli in.png out.jpg` `out.jpg` is 559 436 bytes optimized with ECT: 557 771 bytes optimized with jpegtran (mozjpg): 557 955 bytes ultra brute force scripts: 556 942 bytes ECT and jpegtran were almost done instantly, and the 3rd one took 545 seconds with 16 threads Effectively I've seen much more gain size with other encoders than cjpegli using the 3rd optimizer

so it's a 0.4% difference for ultra brute force

well jpegli seems to use this script: ``` Start Of Scan: 3 components Component 1: dc=0 ac=0 Component 2: dc=1 ac=1 Component 3: dc=1 ac=2 Ss=0, Se=0, Ah=0, Al=0 Start Of Scan: 1 components Component 1: dc=0 ac=0 Ss=1, Se=2, Ah=0, Al=0 Start Of Scan: 1 components Component 2: dc=1 ac=1 Ss=1, Se=2, Ah=0, Al=0 Start Of Scan: 1 components Component 3: dc=1 ac=2 Ss=1, Se=2, Ah=0, Al=0 Start Of Scan: 1 components Component 1: dc=0 ac=3 Ss=3, Se=63, Ah=0, Al=2 Start Of Scan: 1 components Component 2: dc=1 ac=1 Ss=3, Se=63, Ah=0, Al=2 Start Of Scan: 1 components Component 3: dc=1 ac=1 Ss=3, Se=63, Ah=0, Al=2 ``` which I guess boils down to first the 1:8 image, then then the 1:4 image, then the rest.

well not quite, some of the data for 1:4 is missing

but you get the basic horizontal and vertical gradient in each block with that first AC pass

(above are the first 7 passes, there are some more after that to get the least significant bits right) ``` Define Huffman Table 0x10 0 2 2 1 2 5 4 3 0 3 1 1 0 0 0 0 Start Of Scan: 1 components Component 1: dc=0 ac=0 Ss=3, Se=63, Ah=2, Al=1 Define Huffman Table 0x11 0 1 3 1 8 2 2 2 2 3 1 0 0 0 0 0 Start Of Scan: 1 components Component 2: dc=1 ac=1 Ss=3, Se=63, Ah=2, Al=1 Start Of Scan: 1 components Component 3: dc=1 ac=1 Ss=3, Se=63, Ah=2, Al=1 Define Huffman Table 0x12 0 1 3 3 2 6 2 3 0 3 1 0 0 0 0 0 Start Of Scan: 1 components Component 1: dc=0 ac=2 Ss=3, Se=63, Ah=1, Al=0 Start Of Scan: 1 components Component 2: dc=1 ac=1 Ss=3, Se=63, Ah=1, Al=0 Define Huffman Table 0x13 0 1 3 2 4 4 6 3 1 1 0 0 0 0 0 0 Start Of Scan: 1 components Component 3: dc=1 ac=3 Ss=3, Se=63, Ah=1, Al=0 ```

https://cdn.discordapp.com/emojis/872898333239279666.png?quality=lossless&size=48

and definitely I've seen many files getting 10% reduction with the scripts brute force so definitely cjpegli is good at that

maybe e10/e11 will be also useful for jpegli, though the difference is small just for optimization but maybe in the future use some other slower modes and more accurate butteraugli

for mozjpeg the difference should also be small, unless there is some large unused or stripped metadata

blame me for that 😛 it's something people are familiar with 😛

it make sense for side to side evaluation

this is not actually side-by-side, it's in-place with original on the side... but yes

wait, there are efforts in `cjpegli` ?

not directly, but different settings can give different speed and quality (at least in benchmark_xl), so it's not a bad idea to have efforts it just takes effort and time to bring different settings into efforts <:KekDog:805390049033191445>

I'll do my own benchmark I'll try `jpegli`, `jpeg` "vanilla", `mozjpeg` and `guetzli` any recommendation for more JPG encoders ? And what about the metric for comparing images quality ?

sjpeg probably about metrics comparisons, like this one https://canary.discord.com/channels/794206087879852103/1020724883241574472/1020755005386534963 there were others i also did some, but i found visual comparisons more useful, though more time consuming if it's not just a few images

"jpegli seq" means sequential, `cjpegli -p 0` "unopt" means no huffman optimization (`--fixed_code`)

so you basically could make 3 effort settings based on that, but I kind of doubt it matters. It's pretty fast anyway.

It rarely seems a good idea to sacrifice ~10% compression just to make it go from a blink of the eye to half a blink of the eye

does jpegli make progressive jpegs or baseline jpegs?

progressive by default

how does jpegli achieve faster decompression speed than libjpeg-turbo?

you need to add `-p 0` to disable progressive and do sequential/baseline jpeg

for baseline that is

oh these numbers are for multi-threaded, I guess libjpeg-turbo doesn't try to use threads

ah, that would make sense then

in single-threaded it should be more similar I suppose

since I'd expect libjpeg-turbo to be the fastest implementation by design

since it focuses on speed above pretty much all else

exactly — though it is based on manual SIMD code I think, while jpegli is using highway, so probably for some platforms, jpegli will be faster 🙂

(platforms for which libjpeg-turbo doesn't have good SIMD yet)

yea, I'm guessing this is on x86 though?

it's actually on my laptop, an M3

so ARM/NEON

that might make a difference then

why is progressive jpeg so much slower than baseline, btw? I assume since it's slower across the board the algorithm itself is slower

I do think libjpeg-turbo has good NEON code paths though, it's probably the main platform where you want to optimize speed since so many phones are ARM

yeah, it makes less sense for jpeg because it's already fast, but some sort of bruteforce mode so that no additional optimizers are needed at all I think would be useful and for slower efforts add like more butteraugli iterations as in libjxl or something like that, so that fast modes would just be like extras, because it's possible and to be like lubjpeg-turbo if someone needs it

progressive is slower both encode and decode because it inherently has worse memory locality and just more entropy-coded symbols to process (though it does give better compression, since basically you can do "poor man's context modeling" by switching up the huffman codes between scans)

ah, right, I suppose going in raster order would make simd much easier

and finally integrate jpegli into jxl, e.g. with some compilation option, maybe even with some advanced use cases than the usual libjpeg api <:YEP:808828808127971399> maybe even with some compatible cli encoding options if the output image will be jpeg

i'm dreaming about some kind of "libjpeglixl" that is a wrapper that implements _both_ the libjpeg api and the libjxl api, and magically allows you to decode both jxl and jpeg via either api. Most of what is needed for something like that we already have.

ideally you should be able to just pass a JPEG to libjxl and decode it, and it will switch code paths based on whether it starts with 0xFFD8 or not

in theory, that makes the most sense to me

as libjxl contains all the code needed to decode a jpeg, you just first have to encode it to a JXL via AddJPEGFrame or whatever

and then decode the JXL

yeah well we could make a shortcut and not actually do the JXL entropy coding/decoding and just read the jpeg and then render it

the trickier thing is allowing to decode jxl images via the jpeg api — I suppose something sensible has to be done in case the image has alpha or is an animation

Veluca i've attempted a custom jpg encoder for tgcom24

I don't think this is necessary, if you can decode both via the libjxl api

well it could be nice to use it as a drop-in replacement for libjpeg-turbo and automatically get some amount of jxl support even in applications that haven't updated (and maybe never will)

not sure that will actually work though, will it?

many applications that support jpg and also png probably either check filename or initial bytes

to know which decoder to use

so they probably won't actually support jxl even if the interface does

Good point, they might give up before even trying...

I've seen an application which detects JPEG by searching for JFIF and EXIF and thus doesn't support bare JIF.

The libvips Windows binaries can now be build with jpegli as drop in replacement for mozjpeg/libjpeg-turbo. My first test were quite dissapointing though. jpegli was slightly slower and produces much larger (30%) images than the mozjpeg path. I guess this is because some defaults which are different between mozjpeg and jpegli. Setting 'interlace'true in libvips produces 50% smaller image with jpegli. mozjpeg shrinks only a bit. Interlaced jpegli is smaller than mozjpeg, baseline mozjpeg is smaller. Are there optimal settings for jpegli when replacing mozjpeg? I might to a comparison table with all the libvips settings if I find the time later tonight

jpegli and mozjpeg both produce progressive (interlaced) jpeg by default i believe (https://discord.com/channels/794206087879852103/805176455658733570/1216404497409249380) so that's one option to set

Interesting. I'll probably have to check which defaults libvips uses. It seems to produce non-interlaced with no parameters. The subsampling mode makes a difference as well. The big differences seem to come from the libvips defaults, not jpegli vs mozjpeg itself. So it's more of a specific tools implementation. I'll do some testing either way though. Many people might interact with an encoder through something like libvips or imagemagick. If the defaults produce worse results with jpegli, it might give a wrong impression what it's capable of

30% larger, but how is the quality? Perhaps it's defaulting to a higher quality level.

Visibly there was no real difference. Decreasing Quality to match mozjpeg size resulted in notable worse quality for jpegli. I think it's down to chroma-subsampling. It's in auto mode per default in libvips and might behave differently between jpegli and libvips. So my mozjpeg images might have used 4:2:0 and jpegli 4:4:4. I'll try to create a table with different settings and results

maybe mozjpeg defaults to optimising huffman tables and jpegli doesn’t (or only in progressive mode), or something like that?

if there’s a switch for that, might be worth trying

Ok, at higher qualities (higher than default 75) jpegli is 10% smaller. Subsampling and mozjpegs auto mode was also mentioned in other benchmarks. Jpegli need an explicit input for subsampling, Mozjpeg does make reasonable assumptions based on the Quality levels

You cannot assume that mozjpeg q80 = libjpeg-turbo q80 = jpegli q80. Quality scales are not quite the same.

there is a simulation for the same size as libjpeg in benchmark, but that doesn't mean the default Q's are the same https://github.com/libjxl/libjxl/pull/2146

jpegli ignores the libvips subsampling option and always used 4:4:4. With explicitly set options jpegli produces smaller files than mozjpeg with the same options, as expected. Yeah, ofc. Quality scales do differ. I was just confused by the big difference with default options at Q75.

for jpegli cli it should be automatic for lower qualities if I'm not mistaken

I think most of jpegli's benefits are mostly in the Q75+ range anyhow

yeah, and the Q scaling will always be different, it shouldn't be the same, it's not some metric with a consistent quality

I mean, in terms of like libjpeg-turbo

yeah, I just mean that quality close to libjpeg Q75 for jpegli can be reached at other values and about Q in general

and needs the same size comparison, not the same Q

and yeah, jpegli still needs improvements for lower qualities and some blockiness fixes, but from what I understand it's not going to happen anytime soon and there are other priorities for now

jpegli excels at higher qualities. mozjpeg has always been better at lower qualities

low-quality JPEGs to me aren't particularly useful so I like jpegli, but if you regularly serve mid or low-quality JPEGs then mozjpeg should be better

Decided to try and replicate this, but I'm not sure how to reduce the colors using the delta pallete, unless I'm meant to preprocess it somehow? https://fixupx.com/jyzg/status/1329067794995011585?s=20

https://x.com/jyzg/status/1329078139700428813?s=20 https://x.com/jyzg/status/1329105633417768970?s=20

Oh that's fun, the embed is interpreting emojis

Ah, that'll be why, removing `--modular_palette_colors=0 --modular_lossy_palette` results in an identical output file with cjxl, so either I'm doing something wrong or cjxl is

Here's what I've been trying `cjxl -P 0 -m 1 --modular_palette_colors=0 --modular_lossy_palette Test.png Test.jxl -d 0.6 -e 10`

could be that it is broken, it hasn't been worked on or benchmarked for a while...

Between this and the extra channel decoding, it seems I'm finding all the lost bits of code haha

The encoder side of everything palette related (both lossless and lossy/delta) could use some improvements. It's mostly naive/unoptimized code, and in the case of delta palette, the code is more of a "proof of concept" and apparently broken at the moment... feel free to open an issue about this, it should at least do something

<@532010383041363969> do you happen to still have the palette JXL file from that test?

https://github.com/libjxl/libjxl/issues/3433

no

Huh, so there is actually a roundtrip test for it, but it never actually verified the feature worked at all https://github.com/libjxl/libjxl/pull/622

well it does test that it is smaller than lossless and quite high quality

so is it that we somehow manage that without the delta palette?

I wonder if the density improvements over the years have simply made it hit the target anyway

Could also be a cjxl bug...

The test tests libjxl, not cjxl

Ah, good point

started my benchmark, should be finished by the end of the w-e

guetzli is so mem-hugry lol

How can I have the bpp of the image ? Is `filesize / (height * width)` good enough (even if it takes the header size) ?

the header *should* be included as it's required to decode the image

(if the filesize is in bytes, don’t forget to multiply by 8)

about the only thing that can be removed is EXIF unrelated to decoding/displaying the image (but really that should be stripped off well in advance of tests of the encoder)

nice thanks

yeah when I said header I meant metadata let's just hope encoders don't add metadata when input doesn't have any

do keep in mind jxls always have color metadata

so it must assume sRGB or something if the input has no metadata in that regard

though I think you can override that with flags to cjxl?

ah <:YEP:808828808127971399> ``` -x key=value, --dec-hints=key=value This is useful for 'raw' formats like PPM that cannot store colorspace information```

that is correct

What

I guess this is enough <:KekDog:805390049033191445> jpegli (420 422 440 444), mozjpeg, jpeg-turbo, guetzli each with many steps for quality and standalone + "instantaneous" optimization + script bruteforcing

I wonder how many days it'll run

<:kekw:808717074305122316>

hdblog has jpegs. and also webpp thumbnails

what's this about ``` The installer exited with a non-zero exit status. ERROR: /usr/bin/ld: /usr/lib/libavif.so: undefined reference to `svt_av1_enc_init_handle' LOG: ~/.phoronix-test-suite/installed-tests/pts/jpegxl-1.6.0/install-failed.log ``` trying to install in phoronix test suite

looks like you don't have svt-av1 installed or there's some issue with that

but I do...

also why libjxl want svt-av1, lol

the benchmarking tool can link to libavif to evaluate avif

Likely dday uses aurora visiolarum

The encodes autoupdates so is not public available perhaps

Devs don't know if is Cloudinaryy

ETA: 2d15h

<:CatSmile:805382488293244929>

and that is I let it run without interruptions so it'll take even more time lol

was just an ffmpeg-related error I think which was broken due to incompatible svt-av1-psy version

Avif quality 100

JPEG XL is becoming like av2 january

managed to get 8949 B (0.001 bpp) lossless on a 10k file I know it's an easy one to encode, but that is crazy

webp vs jxl

## JPG encoders + optimizers benchmark - cjpegli v0.10.2 e1489592 [AVX2,SSE4,SSE2] - guetzli 1.0.1-4 (Arch extra repo) - libjpeg-turbo version 3.0.2 (build 20240125) - mozjpeg version 4.1.5 (build 20240322) Github link with script and all the graphs: https://github.com/T-3B/compresssion_benchmarks/tree/main/JPG%20(encoders%20%2B%20optimizers) **The graphs were made by <@548366727663321098>, a huge thank to him** ### Conclusions "Unfortunately" optimizing the JPG does not change the encoder to choose for the best ratio bpp/ssim2. I've seen much higher size reduction for JPGs found on the Internet (up to 10%), but it is quite small with this benchmark. It is quite safe to say "**always use `cjpegli` 420 for BPP < 1** (surprisingly followed really close by mozjpeg), **otherwise `cjpegli` 444**", jpeg-turbo indeed is fast but who can *not* bear sub-second encode time with cjpegli... There a little exception here: the konosuba is quite an unusual image with lots of colors (and difficult to compress). For that one, the winner is `cjpegli` once again but with 444 as chroma subsampling. I love that one. **ECT optimization is definitely worth it**, as it is almost instantaneous.\ Scans bruteforcing (`jpegultrascan.pl`) is only for people (like me) worrying about maximum compression at the cost of really slow speed - note that since it's bruteforcing, it can be done using several threads. Here are some graphs (many more on the GitHub link):

and this comparison was done without any ICC profile so if you add XYB JPG to this benchmark, I think cjpegli will just destroy anything else <:KekDog:805390049033191445>

Which encoders do you prefer?

Dzgas

I got pinged ?

I did ! my mistake ! I deleted the post 😄

I missread something in the page you posted

no problem haha

I managed to improve libjxl

Jon Is still tuningg

a good idea in general is to ignore fab because he just says random things without context or elaboration

fab was not kicked abruptly because he is neurodiverse and this community favors inclusion and tolerate noise. I guess he passes by highs and lows which makes its contributions more or less nonsensical depending on the day

neurodiversity should NOT be normalized

and I think the fab is something else entirely

he left and deleted his messages ?

¯\_(ツ)_/¯

idk

confirmed or speculated neuro diverse?

just say AUTISM

he has said in the past that he is autistic, so confirmed if we take his word for it

imo he's just ITALIAN

huh. I had seen speculations before and Luca said not to do it and was wondering

do you know what program is used for those graphs?

as I said all these graphs were done by <@548366727663321098>, and myself I don't know which tools he uses (and want to know too) <:KekDog:805390049033191445>

R and ggplot2

I should have realised by that sideways legend...

👀

I'm not open to normalize my neurodiversity to minors, in any way. I prefer being to do what I want and the normal people just straight die

Autistic will never get along neurotypicals

They agne different interest and different autism

And also a different version of brain

I prefer fanpage.com and medical doctors to Juzt die than to cure my medical conditions

I love being alone

And phil other People that's is what give me an interest

And spoiling people on intense internet whatever is that

I do not cure about That rules that exists in the reality

I'm autistic and you deserve to be with me and serve me with respect

Gemini is my slave According to my mental brain

In my brain there is a single opinion, to tell other people to f::: themselves

Wot lmao

20:18

Now JPEG X/ beating all codecs by mos score 0,561 great

The lower presets are improving

Comparison now JPEG XL

Are you... Re-encoding the encoding comparison? xD

MozJpeg ``` cjpeg.exe -baseline -notrellis -sample 1x1 -quality 90 -tune-psnr -quant-table 1 IN.png > OUT.jpeg ``` this is the most frequent thing I use in 2024. sometimes sample 2x2

JPEG XL now is giving usable results as avif q43 comparable

Because i fixed the errors

I didn't denoise disgruibe or something touch the image or the profile

Just the Devs AOmedia that improved the software,

They are casual i'm using datas of people who live Jarre where's it live

Thomas people don't know aboutAI compression

Improvement to rtc 11 and effort 1 JPEG XL

Tuned on bigger dataset

10:14Rome

Avif vs jxl irish and romanian singerz

After stealing her data

The right is how really looks on youtube

Complete

Nothing crashed i finished my review work

Is now 50% better;

I don't think is anymore accessible

The App is smart i made a screenshot then closed the tab with edge canary and Jon refreshed the tab

based

0,250bpp screenshots now approaches Transparency

Metrics correlate more

I denoised everything was possible

An extremely unscientific and limited test of FFMPEG speed for WebP vs JXL

Run on a single image, a single time (Although I ran each command around 3 times separately without logging), with whatever defaults FFMPEG has set unless written above

Also, FFMPEG still has effort 9 as the highest, although not necessarily the worst considering the memory and speed penalty...

yeah, you really need streaming input (and possibly output?) and several cores to let JXL stretch its legs for lossless

All things considered, I thought those were rather good results. Smaller and faster in every case, only getting better as effort went up

one thread parallel encode supremacy

thankfully with the new memory efficency stuff you can run a lot of parallel encodes

it's still using a lot more mem than a single encode with multiple threads

if i were to run a bunch of encodes for two weeks what would you want to see

We've been trying to decide on new default settings, so knowing the best effort or parameters for size/speed across a wide test suite would be perfect for that

perhaps nightly builds could have some capability to tally/share gain summaries, with cpu/ram info and optionally the source image if its just 1/few not too large ?

might be out of scope for libjxl though and more suitable from 3rdparty to implement in their own utilities but itd only help set their own defaults

Kitchen objects because in that category i have difficulty to get good vmaf

I renounced to do the encoding

Jxl is good but avif was a very fast development,

Spidermario how to make this encode better is e8 q11

I optimized this encode for dell’Msi Pro MP241X

And I have a Samsung rs580

I improved loudness normalization in Facebook

Another round <:CatSmile:805382488293244929>

very interesting effect from gaborish....

hopefully humanity can adapt to become more creative when reposts are destroyed so quickly

maybe we shouldn't constantly recompress stuff

why not

keep in mind that her hands are heavily photoshopped and jxl in q10 enhances stuff and is poor at metrics even in the eyes

because i've never Seen the original horrible stuff

that's basically what lonnie is saying

as an autistic person i can't differientate between different av01 comp1essed hands

because i'm racist

hee's was better

What lmao

Gemini was updated today to be temperature 1-2.8 because the reality musicians want money as income

Looks like jpegli zeroes AC quite quickly. And inv_gaborish(gaborish) should probably be made closer to the identity function.

when this video was made, gaborish filter was not yet implemented ? https://www.youtube.com/watch?v=FtSWpw7zNkI or is it a regression ?

how is now? https://jon-cld.s3.amazonaws.com/test/distorted/225284/jxl-225b6884-pdc0-e6-q10.png

i did while taking meds, so i didn't considered the video use;

It was, but the encoder side gaborish has been changed since then, I suspect causing a regression in terms of generation loss.

should I open a PR ?

I think so

But let's just Jyrki work

My encoder have terrible generation loss

This is an example for fb

Because I optimize only for shareability

Not for the true image quality that matters

Jxl latest is this https://artifacts.lucaversari.it/libjxl/libjxl/latest/ but i think is alpha

On desktop I get a different encoder with blur qualities

In this example the foot changes a lot

Too much use of motion vector

This is first fix for epf#1

I wonder if optimizing the JPG file before lossless recompression with `cjxl` change something in the JXL output 🤔

1 way to find out™️

yep

it's been 10 minutes my CPU is optimizing a JPG file <:KekDog:805390049033191445>

Somehow I've never gone hyper autism optimization mode on jpg files <:KekDog:884736660376535040>

~~Maybe I should start doing that and shave off a few bytes at most~~

well `ect -9 -strip -progressive` already does a great job (even better than mozjpeg's `jpegtran`)

and is faster than 0.5 second

benchmark with several JPG optimizers (not ultrascan tho) https://encode.su/threads/3987-Jpeg-Optimizer-testing

and the slow-as-fuck bruteforcer: https://encode.su/threads/2489-jpegultrascan-an-exhaustive-JPEG-scan-optimizer

which, for smallest output, should be used with `-b 13 -t 16 -s -i`

well, there are also ``` -j Strip APP0 segment for 18-byte savings (generates non-compliant JPEG that may be incompatible with some software) -a Use arithmetic coding (unsupported by most software; jpegtran support required) ```But I don't want to break my image 🙂

If I want to break compatibility for max compression then I'll just go with arithmetic coding on jpeg which should be about 7% smaller <:KekDog:884736660376535040>

Oh wait, that's an option there XD

but it indeed provides you the best JPG compression

yeah so you can do arithmetic encoding and bruteforce scans

I requests your review

Gotta try the mega slow thing on this image <:thinkies:854271204411572236>

https://jon-cld.s3.amazonaws.com/test/ahall_of_fshame_SSIMULACRA_2_modelD.html

Yes Fox try reviewing

you, or me ? <:KekDog:805390049033191445>

You have a very awesome eyes

You ~~since I'm too lazy to figure out how to use jpegultrascan atm~~

🍕

Maybe someday™️

fine I'll do it you still owe me a pizza <:KekDog:805390049033191445>

Reviewing what exactly?

Sure, I'll deduct that from the pizzas you owe me <:KekDog:884736660376535040>

lol ``` 376261 Folder.jpg 361274 Folder_ultrascan.jpg 306466 Folder.jxl 306602 Folder_ultrascan.jxl ```Optimized JPG for this specific image gave a bigger JXL

is there a way to "unoptimize" JPG ? I mean to have a bigger JPG file, still lossless

There probably is if there's an option to force an output even if the image is bigger I guess

What image did you optimize though? <:thinkies:854271204411572236>

Not even the fox image I sent smh

that the one's I was optimizing before your sent yours lmao

I started your encode with both arith and non-arith

jpegtran without any switch?

pixel art with 8x8 pixels per pixel could fit nice in a jpeg !

1 pixel per block exactly, would it beat png ? I'll test

nope, since it still optimizes by default but using jpegtran with `-revert` seems to do the job thanks

It is because I had some ancient jpegtran version. It does not have -revert switch, and does reverting by default

But I really expected cjxl to give a smaller JXL "JPG-recompressed" if the input was smaller shouldn't it be the case?

Great now the colors aren't dead

E7 14:13

No, because this optimization is about using huffman with higher effort. And jxl decompresses to coefficients, so that existing compression does not matter. As far as I understand

Is from 0.7.0 and up that jxl had this problem

Is there some problem on this images?

Id say to not change anything from

it's just that smaller JPG does not mean it'll be a smaller JXL

2022 and 2023 2024 specs are a bit different in the requirements the last isn't standardized

It includes 3 documents

<@219525188818042881> ``` 1140555 GHVfbX5a0AAdJsJ.jpg 0s 1129229 GHVfbX5a0AAdJsJopt.jpg 36m38s 8 threads 1043474 GHVfbX5a0AAdJsJopt_arith.jpg 39m46 8 threads ```You still can win 18 bytes by removing the APP0 segment, but that would result in a non-conformant JPG

🍕

Hm, nice difference. Maybe I should start wasting a lot of electricity on optimizing jpegs <:KekDog:884736660376535040>

Usually yes, but I expect the brute forcing is causing data that JXL doesn't handle as well

And one of the 'steps' of the JXL transcoding is using arithmetic actually, one of the reasons they don't transcode along with being so rare

I know :p

:<

The arithmetic one could be made a little smaller <:thinkies:854271204411572236>

https://issues.chromium.org/issues/41288624#comment16 arithmetic coding not supported in blink who would have guessed... The argument is similar to the one given for jxl

Oh wait, that's added metadata that got removed <:KekDog:884736660376535040>

wait, even changing the extension in Discord and it still adds metadata ?

Don't think so

But it had 6 bytes of metadata <:thinkies:854271204411572236>

I was running tests on an 8k image at 4am last night (as you do), and by the end I realised I'd spent 20 minutes and all my RAM to save 5% after already shaving off 40% in 12 seconds

Totally worth it™️

Actually that reminds me...

I mean... I sent 2040 fox images in a random Discord server and almost all of them are around 4k res and all are optimized <:KekDog:884736660376535040>

Oh wait, 2040 messages with I think 8 images on average per message, so about 16320 images XD

then both ECT and jpegtran does not know about that none could shrink these "6 bytes of metadata"

I remember a few months ago I was bored and curious (a dangerous combo), so I set the quality to nothing and cranked the effort to max. I fit 800 artworks into under 1.5MB... And some were still recognisable!

I ran it you say to me what you used

😁

jhead ``-du``

Or ``-purejpg`` which is a combination of params including ``-du``

Does that earn me 1 more pizza point? <:KittyDrink:1126563386482770012>

Or do I have to steal them from your freezer when I'm in Maastricht or Aachen again since that's very close to where your freezer is located <:KittyUwU:1147753612529913938>

yeah Ig, thank you <:KekDog:805390049033191445> but didn't I earn a pizza point for FLACCID and 1 for JPG ultrascan ?

FLACCID doesn't work lmao

I got told the dev the issue, it's fixed now

But yes, 1 pizza point for jpg ultrascan, but haven't I earned about 10 pizza points by now? <:thinkies:854271204411572236>

cfr. awcy server

how lol but yeah ofc if you can find my house come and I'll buy some pizzas for both of us there's a pizzeria just 100m away <:KekDog:805390049033191445>

Are you sure you want me to find your house? <:KekDog:884736660376535040>

don't you live in my walls ? <:KekDog:805390049033191445>

and tbh I think you already know the village I'm in finding the house shouldn't be that difficult :trollhq:

That's probably true <:KekDog:884736660376535040>

Heh, haven't found out where you live, but found out other things XD

wdym other things lol

😳

Oh wait, I accidentally revealed your name lmao

~~Now the message magically self destructed™️~~

bwahaha like you didn't noticed <:kekw:808717074305122316>

Somehow I didn't <:KekDog:884736660376535040>

But at least it's very easy to find anyways, so guess it doesn't matter

And lol, totally forgot about the 2021 flooding. Was pretty close to where you were at the time ~~nice youtube vid~~

yeah, just go to my description and you got my full name in my GitHub repo

<:kekw:808717074305122316><:kekw:808717074305122316><:kekw:808717074305122316><:kekw:808717074305122316>

totally forgot about it

Unfortunately lacks even the color gamut fixes https://artifacts.lucaversari.it/libjxl/libjxl/latest/

See us tomorrow

Yes, I was somewhere far uphill, so didn't see much water. Mostly just heard non stop sirens for 3 days

I think i will try, should i do on this build

not even my video™️

Better than av2

<:KekDog:884736660376535040>

Why 100 Euros? <:thinkies:854271204411572236>

I would've assumed lowering thread count with `-g 3` would easily lower memory to below the default, but it seems like it allocates almost the same regardless of threads Single Thread `-g 3` ```wintime -- cjxl --disable_output -d 0 "C:\Users\jonat\Pictures\Resonite\2024-03-30 03.52.09.png" --num_threads=0 -e 3 -g 3 JPEG XL encoder v0.10.2 e148959 [AVX2,SSE2] Encoding [Modular, lossless, effort: 3] Compressed to 8052.5 kB including container (7.767 bpp). 3840 x 2160, 4.169 MP/s [4.17, 4.17], , 1 reps, 0 threads. PageFaultCount: 233678 PeakWorkingSetSize: 352 MiB QuotaPeakPagedPoolUsage: 34.8 KiB QuotaPeakNonPagedPoolUsage: 10.75 KiB PeakPagefileUsage: 424.7 MiB Creation time 2024/04/05 15:36:33.605 Exit time 2024/04/05 15:36:35.808 Wall time: 0 days, 00:00:02.203 (2.20 seconds) User time: 0 days, 00:00:00.218 (0.22 seconds) Kernel time: 0 days, 00:00:01.984 (1.98 seconds)``` 8 Thread Default ```wintime -- cjxl --disable_output -d 0 "C:\Users\jonat\Pictures\Resonite\2024-03-30 03.52.09.png" --num_threads=8 -e 3 JPEG XL encoder v0.10.2 e148959 [AVX2,SSE2] Encoding [Modular, lossless, effort: 3] Compressed to 8052.7 kB including container (7.767 bpp). 3840 x 2160, 18.442 MP/s [18.44, 18.44], , 1 reps, 8 threads. PageFaultCount: 182085 PeakWorkingSetSize: 215.2 MiB QuotaPeakPagedPoolUsage: 35.05 KiB QuotaPeakNonPagedPoolUsage: 17.26 KiB PeakPagefileUsage: 255.5 MiB Creation time 2024/04/05 15:36:45.221 Exit time 2024/04/05 15:36:45.888 Wall time: 0 days, 00:00:00.666 (0.67 seconds) User time: 0 days, 00:00:00.875 (0.88 seconds) Kernel time: 0 days, 00:00:02.515 (2.52 seconds)```

even noticeable for jxl, probably because mozjpeg uses aq/trellis differently so the high frequencies are gone pretty quickly need new photon noise api extension for jpegli <:KekDog:805390049033191445> https://github.com/libjxl/libjxl/issues/3461

I fixed aurora and jpegli on dday and Hdblog

Provided the encoders are that of the page hall of shame and aren't customized for theirs page

Still I like denoise

But not de,noiser

Denoise meaning like the game of crimson the one you discovered swatches on fb

HDblog refused that pill of g......

Is a professional site tjeu are right

Photography teams can get Their format

Those distorfionz aren't acceptable for enthusiast Photographers but are acceptable for web uaage

Which flooding? En France?

This one

Looks huge!

It was, 243 deaths and 54 billion Euros in damage

Mediaset helped me to do jpeg xl compression

we had fucking army boats in our streets to evacuate people <:KekDog:805390049033191445>

smh the only part of the map where there are boat and helicopter 'icons'

My mother completed the jxl comparison

All not only 3 images

A true jxl mom™️

Is too tuned on my taste please wb change everything for aomedua

What were the results?

More sharpness in Windows and stairz

In every single piece of image, anyway not bar

Now it's smaller <:KekDog:884736660376535040>

Only 92 bytes though

so it's with something else than `jpegultrascan` ?

just when I thought the war was over <:KekDog:805390049033191445>

Nope <:KekDog:884736660376535040>

you're answering the first question ?

Both

nice then

because I can just bruteforce settings to find what you used x)