JPEG XL

<a:kms:717156594596446288>

Does your eye color have an influence on your perception of color? I would assume not, but then again, why not? Is there research on this?

Idk if there's research, but even independently of eye color I kinda doubt that everyone's perception of certain colors is the same

Bc the only way to describe colors is relative, right?

If you ask me what red looks like, I can tell you that blood is red and fire trucks are red. Even if you see the color I see as, say, yellow when you look at those things, you'd still see them as being the same color

https://discord.com/channels/794206087879852103/794206087879852106/810163961378635797 Continue SuperREP LZ77, some flag information, LZ77 algorithm -m1..-m3 compression modes implement LZ77 compression algorithm. -m1: input file split into chunks of L bytes (specified by -c option, 512 bytes by default). For every chunk, the program stores SHA-1 hash. When later it encounter L-byte chunk with the same SHA-1 value, it replaces new chunk with reference to previous one, assuming that chunks are equal. -m2: same as -m1, but only weak hash stored for every chunk. When the program encounters chunk with the same hash value, it rereads old chunk from input file in order to compare data. -m3: same as -m2, but the program compares bytes before and after equal chunks in order to extend match as much as possible. -l option may be used to specify minimum match length. On decompression, every squeezed chunk is restored by reading the contents of previous equal chunk from output file. The compression algorithm in -m2/m3 modes rereads the same chunks in order to compare them with current data. This puts heavy load on OS I/O system and disk cache. The algorithm requires that input file when compressing (except for -m1 mode) and output file when decompressing is seekable. If that's not true, you may use -temp option to tell the program to create temporary file used to store a copy of all uncompressed data. The algorithm also need to know size of input file in advance when compressing. If the program cannot determine the file size (i.e. when compressing from stdin), file size should be supplied via -s option. Values larger than actual filesize will work as well, and by default 25gb is assumed.

`-m1` sounds very risky. Hash collisions can happen. SHA-1 is 20 bytes, so when using it on 512 byte chunks, the math is not that hard: there are 256^512 possible 512-byte chunks, there are 256^20 hashes, so on average, there are (256^512) / (256^20) possible chunks per hash, i.e. every hash has (on average) 256^492 chunks that will collide into that hash.

You can just as well replace every 512-byte chunk with it's 20-byte SHA-1 hash 🙂

It's not really risky since the probability of a collision is extremely small.

Yes, philosophically we will never know for any human sense if different people really experience it in the same way or not. But I mean more pragmatically: you can test whether someone can see a certain contrast between two colors or not, and things like that. We know e.g. that some people have mutations in the genes that code some of the cone cell types (usually L or M), causing them to malfunction, have a somewhat different response curve, or not work at all, causing different variants of color blindness. This can be tested and is relatively well-understood.

Is it? The probability of two random chunks colliding into the same hash is very small (I suppose it should be roughly 256^492 / 256^512). If you have a 1 GB file though, what's the probability that there are two colliding chunks amongst those 2 million chunks?

Low!

SHA-1 collisions only happen with targeted attacks against known weaknesses in the algorithm.

Getting a collision via brute force / bad luck over lots of data is close to impossible.

it's not just 2 million times the chance of a single collision though

there are 1 trillion pairs of chunks that could collide in a 1 GB file

yeah. Still pretty small 😄

I guess

1 / 256^20 is a small number

There are 2^160 possible hashes. Under the birthday problem, i.e. when all possible pairs are considered, you have to halve that. So 2^80 is the order of magnitude you need to approach for collisions to be likely.

If billions people would use it to compress terabytes worth of data, it can happen

hm, 10 billion people times 10 terabytes of data, with 512 byte blocks, is 2^67.4 hash. 40 byte blocks and it's 2^71 hashes. That's getting close. Still, "starting to be possible" chance for a single collision in 100 zetabytes of data seems good enough for most uses. ZFS apparently uses 256 bit hashes, which blows that out of the water. Guess we could store the collective knowledge of humanity in a single ZFS system and don't have to worry 🙂

it's just philosophically I don't like the idea of a compression algorithm that relies on luck for correctness – even if the chances look safe enough

Just curious probably have other compressor similar SuperREP?, (use similar compression algorithm), or new compressor have better implement? I plan use SuperREP like this, precomp + SuperREP + LZMA(or LZMA2) https://github.com/schnaader/precomp-cpp

Does it compress better than zpaq?

From my understanding, everyone has the same capability to perceive colors the same way, unless they have a cone/rod deficiency, which would result in a type of color blindness. The main difference between color perception and identification is psychological. It is based on how you mentally catalog colors. In what ways do you group colors, how much have you studied colors to learn more obscure names. Can you identify maroon vs burgendy, chartreuse vs lime, mother of pearl vs eggshell, and dark orange vs brown? Does your culture and language even have words for these subtle differences? Linguistics of color are pretty interesting.

here's a very short introductory video. It's from vox, so take it with a grain of salt, and if interested, look into the sources. https://www.youtube.com/watch?v=gMqZR3pqMjg

A lot of basic misunderstanding has stemmed from these cultural differences and labeling. People mis-assume that because a group of people did not label something as blue, or could not differentiate purple from green, that means they suffer from a color blindness, or that they see colors differently, when in fact they see the same stuff, they just don't catalog it the same mentally. They could be trained to catalog using a different system and learn the distinctions between subtle differences that matter more to other cultures. They don't go through a radical transformation where they suddenly see all new colors. What they see is the same, they just know more words to better describe or differently group the color spectrum.

So if you see any pop-psychology claiming that one race, or group of people in history couldn't see "blue", or were physically different, they are repeating old bad assumptions that were disproved

If you really want to dive into the world of "color", Ray Maxwell had a podcast for a few years that mostly focused on color. He's spent decades studying and learning about color deeply. **Episodes list:** https://twit.tv/shows/maxwells-house https://www.youtube.com/watch?v=tKDSd3c75b0&t=14m30s

Thanks for the pointers, it's interesting stuff indeed.

Yep, cool video, thanks! Luckily, we can make sure that humans at least are **looking** at the same stuff. Or, mostly, same. No media queries for how exhausted you are or of what age or what’s the colour of your wall… Yet 😉 . This also cool: https://youtu.be/32LGlKwACfI

there is variation in the matching functions

how much is contributed by eye color, I don’t know

(the reference I actually had in mind plotted the XYZ matching functions instead of LMS but I can’t seem to find it again)

https://twitter.com/brucel/status/1361378464335597570

https://github.com/w3c/csswg-drafts/issues/5889

Fun fact: the original name I used when making FUIF, was FLOF

FLOP is funnier

Free LOssy image Format was my reasoning, but "lossy" and "lossless" both start with "lo" and it could still do lossless, so FUIF made more sense

https://tenor.com/view/party-minions-dancing-party-time-happy-gif-15045334

especially since it means "party" in my native language, which is kind of fun

And then came Pik...

and it has good ambiguous pronunciation properties, which is important in image codec naming conventions

yes, pik is not a very good name in my native language

spoiler: ||it means dick||

Pik image

Free LOssy Photo/Picture/Image Digital Imaging Kit

android 12 is adding avif https://www.androidpolice.com/2021/02/18/the-first-android-12-preview-lands-today-with-more-changes-than-we-expected/

Great, then in about 5 years, android apps that want to use avif will no longer need to package their own avif decoder

more than 5 years probably

Heck, even my not-so-old Samsung Galaxy Note 9 won't decode them by default!

Smartphone adoption is also necessary for a codec to be successful

I think that I'll have JPEG XL and AVIF decoders included sooner in Samsung Gallery (or maybe even faster in Google Photos) than in One UI 2.5 that's only getting security patches now

I think that on Android a better solution would be including the formats in the apps themselves, because we **__still__** can't rely on those goddamn Android OS updates!

First Google Photos, it's OEM-independent, is really popular, is integrated with one of the biggest to-be use cases – Google Photos online service, and can even act on its own as a standalone photo gallery app without logging in.

Getting in the apps themselves is the only way to make rapid progress

Photos and Camera apps could use jxl if only to save internal storage on the phone while still using old jpegs

Chrome can get jxl support

Apps for facebook, twitter, etc can do the same

Then OEM photo galleries, Samsung is the biggest (the most users will get it) and probably wants to be associated with novelties.

And then camera apps, in order to create the JXL files. Again, Samsung should be the first. Don't worry about Pixels, they always get the latest Android, so they'll get their JXL decoders and encoders with the OS itself.

Getting the decoder binary size down on ARM to make it less of a burden to include a jxl decoder in apks has been something we have looked at again, and probably will keep looking at

And if Google decides to include JXL encoder inside GCam, then lots of tech-savvy ppl will want to try it out

Especially those who install it unofficially (modded GCam APKs)

If they notice that it provides better compression, quality AND their apps can view it, they'll switch to it

Full adoption will be a long battle, but I hope we can get a momentum going where software devs feel pushed to jump on the bandwagon and add jxl support

I should make an update of this old blogpost: https://cloudinary.com/blog/one_pixel_is_worth_three_thousand_words

see how big 1-pixel AVIF, HEIC, WebP2, and JXL is

and also how compressed size grows for a N x N solid color image

https://mangadex.org/thread/425240/1/

`uncompressed` (lossless?) <:Thonk:805904896879493180>

<:PepeHands:808829977608323112>

We spent years designing this uncompressed image format

Long debates on big-endian vs little-endian

Interleaved or planar

Whether to do bit packing or not

It was rough

Bitter discussions

And then of course the header syntax

isn't that what BMP is for?

Human readable like ppm or not

Endless committee meetings about it

I am kidding, of course

A nice uncompressed format would be nice actually. PFM does not do alpha, PAM does not do float...

BMP is a surprisingly complicated and ugly format

It is basically a memory dump of the internal struct Microsoft used for storing images

Which evolved over the years

It can do some 'compressed' stuff like RLE and palette representation

It is mostly a mess to make a complete BMP reader

Not as bad as TIFF though

Anything Adobe touches becomes extremely hairy over time

https://c.tenor.com/f1-BGCzrFp0AAAAM/star-wars-smiling.gif

TIFF: for when you want png, jpeg, uncompressed images, and every color space ever invented, in the same file!

It also has the Adobe mentality of doing everything in every way possible instead of just choosing a convention and sticking to it

Which means a decoder needs to implement all possible ways of doing it

It's encoder convenience over decoder simplicity

You can do big-endian or little-endian

Not just for the data itself, for all the header stuff too, including the magic file signature

You can have 0=black, maxval=white

Or you can do the opposite, 0=white, maxval=black

You can do planar or interleaved

You can do tiles, stripes, or just everything at once

Just let the wookie win, for god's sake!

And then every now and then Adobe surprises you with yet another thing you can do in tiff

Is it turing complete yet?

we need svg levels of feature creep

Not afaik

The 24-bit floating point representation in tiff surprised me a while ago

It's not in the spec

But photoshop decided to start doing it

You can have entire PSD files as 'metadata' in TIFF

Can I include a TIFF in a PSD to create and infinite recursion loop?

TIFF forms the basis for DNG, doesn’t it?

It's microsoft in the 80's, why am I not suprised that they created a format which is as difficult to use as possible for anyone but them?

> DNG is based on the TIFF/EP standard format yep.

> DNG is based on the TIFF/EP standard format

It seems to me like it's more agree to disagree in standard ways

ah, so like “Office Open XML”? :p

(.docx, .xlsx, etc.)

How that got into ISO when we already had odt and the other OASIS formats still confuses me

Best guess: https://tenor.com/view/money-mr-krabs-gif-18326632

ah, I think this is the article I had seen on the subject: http://www.robweir.com/blog/2006/10/leap-back.html

actually, I’m not so sure, but still an amusing read

I think that's just an unfortunate consequence of Microsoft's obsessive backwards compatibility

Exif is also a collection of TIFF tags, iirc

News on chrome?

IMHO it's more of a topic for <#803574970180829194> than this channel, but OK

Gerrit still reports `Merge Conflict`

<:Google:806629068803932281>

Code has been reviewed, and recently the commit's author (Moritz Firsching) made some fixes based on that review.

Moritz is here fyi, in case someone hadn’t seen

https://www.phoronix.com/scan.php?page=article&item=clang-12-5950x&num=2

Jpeg-recompression is used as the main test <:PepeHands:808829977608323112>

Ugh, can we reorder things so that doesn't happen?

At least jpeg recompression does use the modular and vardct code paths, so it's not completely irrelevant

But only DCT8x8, trivial adaptive quant (all constant), no patches/dots, no epf, etc

I just almost mistyped bitstream as "butstream" and that doesn't sound very good

https://c.tenor.com/2y0fwDpQ6sIAAAAM/poop-shit.gif

``poop-shit.gif`` I hate it already

why do some gifs get shown in discord and others not?

https://tenor.com/view/hippopotamus-pooping-tail-wagging-excited-monday-gif-10133844

Well, guess they made it so that certain links get recognized

Like Tenor

that's disgusting

No shit

Yes shit, that's literally what it is...

https://www.technology.org/2018/12/05/why-hippos-are-spraying-their-dung-like-agricultural-machines/

Anyway, **bit**stream it is

How this is on-topic

No one cares

Bitstreams are on topic

Buttstreams not so much, I have to admit

Ok, so let's turn it into on-topic: Is JPEG XL butt-ready?

https://c.tenor.com/nEuA4w73KOoAAAAM/austin-powers-mike-myers.gif

The bitstream has been frozen for a while now

A month, officially. Almost 3 months in practice.

Come on, I've got a flamethrower

It'll unfreeze everything in a matter of seconds

https://tenor.com/view/kill-it-with-fire-fire-gif-13411041

Frozen is good though

Yep, it's kinda childish, but still a good movie

https://tenor.com/view/frozen-let-it-go-disney-disney-princess-elsa-gif-3597313

frozen buttstream

ok, now make jxl 2

sorry j><l 2

I meant frozen is a good state for a bitstream to be in

jxl 2 maybe in 2048

its actually pronounced "j cross l"

or "j chi l", if you must

Jay Chill

it's actually french, _j'xl_

j'excelle, tu excelles, il excelle, nous excellons, vous excellez, ils excellent

https://www.megekko.nl/product/4278/279206/Socket-AM4-Processoren/AMD-Ryzen-5-1600-processor

Beautiful

It's true art <:PepeOK:805388754545934396>

firefox pdf?

firefox is not good to read pdf

chrome as pdf viewer won't spy you

Wtf are you saying

Nobody knows :p

in my inbox: "Hi Jon, I hope this message finds you well. I saw your image compression videos on YouTube and wanted to try those experiments myself. Do you mind pointing me in the right direction? I saw the code you used on reddit but I know little in programming so it wasn't helpful to me although I'm willing to learn. Is there a GUI tool to achieve generation loss? Any help would be appreciated. Thanks for your time."

you cannot believe how many people want to "achieve" generation loss as some sort of artistic effect

I deliberately reencode videos multiple times (even using different codecs) for R&D purposes, but for artistic effect? <:Thonk:805904896879493180>

The generation losses from WebP in your blog post were rather interesting though

You better get to know about ｄｅｅｐ ｆｒｉｅｄ ｍｅｍｅｓ

And <@!794205442175402004>, check your Facebook notifications 😉

facebook, ugh

nice page though 🙂

FB kinda <:kekw:758892021191934033>

https://youtu.be/JR4KHfqw-oE

Thanks! I'm trying to keep it as professional as it can be 😃

It's not going to be "boring" professional (as e.g. JPEG XL's official site), but "cool" professional (as big companies' Facebook fanpages).

<@!794205442175402004> you can check draft posts (it's not finished yet, so please don't post anything yet), but you can see an idea for my first post. And if you try editing that draft, you'll see... multilingual post! (You'll be able to get to know my native language and see it in action.) When I let you know that the post is finished, please make a Dutch version of it.

genuinely surprised he actually uploaded it 1000 times and didn't fake it by just running it through a compressor locally for most of it. That must have taken forever, even if automated

honestly if nobody makes a GUI for jpeg xl that works on windows 7 nobody will use

normal people don't know about xvniew and squoosh

define "GUI", because XNView is a pretty polished GUI, it supports everything, and is available on Win 7. I can create for you a GUI to encode/decode/view JXL files (I did it for FLIF), but no one is going to go out of their way to download a program that *only* handles JXL. When they could instead use something already established that does that and more

Plus, writing a plugin for an existing program is probably easier

I think a more valuable usage of time for adoption would be to create a Node.js wrapper around the JXL CLI. I did this for FLIF a few years ago: https://github.com/FLIF-hub/node-flif

it takes a LONG time to create that though, I don't have the time to do it (again)

Plugin should be better in yet another way – if it's a plugin for a popular program, you're going to reach *masses*.

A Node.js wrapper around a native CLI executable would mean JXL conversion could be very fast, and the API very easy to interact with by JS devs (most popular language, especially for web where adoption is most key)

yes but normal people don't know xnview squoosh

or paint.net or gimp or krita or megapixel (these require python 3.9.1 and windows 10 to work properly) gimp can also work on windows 7

The slowest part of building Node-FLIF was: 1. Getting actual native binaries (this should 100% be it's own versioned node module, see `7zip-bin`) 2. Finding out the limits of every single option in the CLI (what are the upper and lower bounds allowed for validation) 3. Writing very very thorough unit tests

WIC works for win7?

yes

but i'm using sasha plugin

from firefox dev and with latest version the dllhost is slow

and obviously the windows photo viewer can't view them

and i can't set them as wallpaper

If you can create a Node-JXL library, I can get you a GUI on XP+, OSX 10.6+ and Ubuntu 12+ (same as I did for UGUI: FLIF)

the problem is how users will create those files

presumably if they are on those OS's they are using a browser that will support JXL and they will be saving the images from the browser. At that point, they can just associate JXL files with the browser, so double-clicking them just opens them in a new tab

ok

and on android

on android it will be worse as they will need to compress

use values less than s7 q 78.1 or s9 q 90

because speed is 8 mpx second max

google what it will use?

google images

also do we need compression?

isn't 0.3.0 -s 4 -q 99.2 enough even if it has some discolouration?

You mean lossless?

why google is trying to optimize butteraugli?

in the next 1000 years even if a pdf in jpeg xl will weight from 13 mb to 6 mb

or a bit more

what is the gain in that?

I'm sorry, but I have no idea what you're asking there

we can't upload more in the internet

90% of the internet is video

when the other 10% will be full

what it will happen?

they will start deleting torrents etc?

The internet is someone else's computer, but there's no inherent limit on how much storage is on the internet

why

there is a limit on the number of addresses

https://en.wikipedia.org/wiki/IPv4_address_exhaustion

interesting

but each of those servers could have an infinite amount of data if they could store it

ah

how to know these better?

what to search on google ?

If i have time i will do a gui for the encoder in qt

This has a decent solution in the form of ipv6, with 2^128 possible addresses before we run into the same issues as with ipv4 https://en.wikipedia.org/wiki/IPv6

What do you want to know?

ok i understood

alex gui called megapixel doesn't work on windows 7, ultra 7z jpg feels like a rip of moises cardona program with show command line options etc. but you can't set custom settings like very specific qualities for exhale q 78.1 q 78.81

megapixel is good you can add presets etc

I had planned a more robust FLIF desktop app but by the time node-flif was done FUIF was being talked about and then JXL. https://camo.githubusercontent.com/a01164e5e8c9e1928dede8e5680abe469a72f2d1917c7b97fe8e6f94530fff70/68747470733a2f2f692e696d6775722e636f6d2f787350485753432e706e67

<https://www.reddit.com/r/discordapp/comments/lrenv7/they_finally_added_image_compression/> https://i.redd.it/a7fyfrbevfj61.png

PNG -> Jpeg? <:Thonk:805904896879493180>

🇳🇱 https://youtu.be/VtZ9_UmCd5M

<@111445179587624960> Did you attend the talk?

No, and I wouldn't understand anything without translation

Yeah, I assume that's why <@794205442175402004> didn't post it himself here.

But Youtube has at least some automatic translation and also slides in English

<:Thonk:805904896879493180>

<@!111445179587624960> that's just these slides: https://docs.google.com/presentation/d/1LlmUR0Uoh4dgT3DjanLjhlXrk_5W2nJBDqDAMbhe8v8/edit#slide=id.gae1d3c10a0_0_17

<@111445179587624960> If you give me the timestamp I'll translate what he's actually saying 🙂

Yes, I know, but maybe there is some additional information in the voice presentation

Well, in general, I understand

Automatic translation, lol. I did not talk about "muslims compression" or "five clowns", to be clear 😅

It's probably failed speech recognition and then translation done on that botched text

Yep, mostly when there is a mix of different languages

The granny components? Lol!

Pik fuif <a:partykitty:681240034895986696>

🍆 🥳

hey all, just dropped in here from the other discord server where <@!794205442175402004> gave a very nice presentation yesterday 🙂

Yes, it was about Muslims compression and five clowns who change the color of the granny components

Now I regret I missed it!

that's how I understood it !

https://twitter.com/JoanieLemercier/status/1363085114176122880

😬

meh, I use twice that with just my microwave everyday. #RealAmerican <:trumpsteaks:443185840949297153>

https://www.slashgear.com/google-photos-high-quality-might-not-be-near-identical-to-original-28661674/

When Chrome adds JXL support, I'll definitely open a support ticket for adding JXL support in Google Photos.

google photos team: "d6 seems good, right?"

*With 1/2 resolution downsampling

640k pixels ought to be enough for everyone

"We'll add them back later with AI"

(the missing pixels)

Haha pirated Topaz Gigapixel AI goes brrrr

What's D6, though?

`-d 6`, or `--distance=6.0`

Just run `cjxl -h -v -v -v` and you'll get all the switches available out there with a short description.

Is the JPEG XL spec freely available?

no sadly

It's behind ISO's paywall

the commitee draft is available

but it's quite outdated

I found a 2019 draft, but it doesn't seem to have anything about the container format; is that documented somewhere?

behind ISO's paywall

ISO doesn't seem to want the spec to be free, even if it's free to use

☹️

If it makes you feel better the core developers of the format are pretty miffed too and are trying to find a way to make it freely available.

Is the container format based on something else or is it something novel for JPEG XL?

Container is ISOBMFF

And optional

Oh ok, is it HEIF?

no

HEIF is only for video formats being used as image formats

JPEG XL is a (mostly) completely new images format

Ok I guess I will just reverse engineer libjxl

I thought BMFF was for video/audio only?

It's also used in J2K, iirc

It's just a simple box/chunk format, not that different from PNG: 4 byte identifier (usually human readable), 4 byte length

We don't require the container format around the actual codestream, it is optional

All the render-relevant data is in the codestream (including things like orientation and color profile)

So BMFF is just a metadata wrapper for you?

The container is only needed if you want to attach non-rendering metadata, including JPEG bitstream reconstruction data

Yes, we only use it like that

So only part 1 of the standard would be needed in theory to write your own encoder?

We don't need something like HEIF that also has ways to specify aux images for alpha, overlays, tiling, etc. Those are all handled in the jxl codestream itself

Yes. Part 1 is enough for e.g. images on the web where metadata is usually stripped anyway

The draft says "if the codestream starts with bytes { 0x0A , 0x04 , 0x42 , 0xD2 , 0xD5 , 0x4E }, the decoder shall reconstruct the original JPEG1 codestream as specified in Annex M", but libjxl doesn't look for that; was that replaced by the ISOBMFF container?

That is only for when there is an original JPEG encoded within the JXL?

Right

sorry if my answers are confusing, I'm a noob at this who's only virtue is being glued to a computer

Just like me bro

You want answers from someone who probably knows what they are talking about, look for yellow names

Ok thanks, looking at the libjxl code it looks like that is done using the container format now, so probably that signature is obsolete.

Yes, that's the old Brunsli which got removed from the spec. Now we have a `jbrd` box in isobmff for the jpeg bitstream reconstruction data, and the image data itself is encoded as varDCT that happens to be not very variable DCT 🙂

the jxl codestream signature is 0xFF0A

you can have a standalone codestream and that will decode

Ah great.

But if it's JPEG-1 in JPEG XL it will always be in the container, right? With signature 00 00 00 0c J X L 20 0d 0a 87 0a

yes, well at least if you want to preserve the bitstream reconstruction data

you _could_ just take the codestream and strip the rest, and it will still be the same image, but you cannot restore the exact same original JPEG file from it

Ok, sounds good. Thanks!

not sure if the `JXL` magic is there to stay

the `ftyp` "`jxl `" (with a space) is certainly there to stay though

because that one is registered with MP4RA

see https://mp4ra.org/#/atoms

ReadSignature() looks for capital "JXL ". It looks like that is what is on mp4ra also.

ah really

looks like you're right

I guess the lowercase ones are atoms

that's what the current draft spec of the container says, so I guess it's upper case JXL followed by ftyp lowercase jxl

these ISOBMFF containers are a bit verbose to my taste, it's nice that we managed to convince the rest of JPEG to make the container optional 🙂

Oh I see, so it has both. Thanks!

in the space ISOBMF needs to say "hi, this is a jxl" we already have a full image header which gives you the dimensions, color space, frame encode mode etc etc

HEIF is even worse, it adds extra verbosity on top of ISOBMFF

tbh I don't really understand why AVIF was based on HEIF. It is a monstrosity of a container format.

Yeah seems overkill for just a single image.

just to say "hi, this is an avif with alpha" takes about 400 bytes

plus: Nokia does claim patents on HEIF

everybody is like "that is nonsense, you cannot patent a container" but the reality is that they did, the patents are granted, and I know for sure that there are companies paying Nokia to use the heif container "just to be on the safe side"

so what's the point of making a nice royalty-free codec if you're then going to demand people to wrap it in a patent-encumbered container... I don't really get it

Of course if it does at some point come to AOM vs Nokia with the result that Nokia's silly patents get invalidated, that would be nice.

It's too bad that ISO needed to charge for the spec. Always seems to be someone wanting to get a cut.

That's the general ISO policy and it's their business model, basically. Which might make sense for non-royalty-free standards that are only ever implemented by large industry players where $200 for a spec is negligible compared to the millions they're paying in royalties and the billions they're making in profits. But for a royalty-free standard where perhaps hobbyists also want to take a look at it and maybe even make their own FOSS implementation, that paywall is a real obstacle, and most likely the number of spec purchases they're getting from it is very small anyway — there's an open source executable spec available for free, after all.

Right, it seems that for Internet-related things, the IETF is becoming the preferred standardization body. Either that or just create your own foundation and publish the spec on your own web site.

And for web specs, W3C and WHATWG have always made their specs freely available as well.

We developed brunsli in 2014, and I thought to save some development effort in plugging it in as is. Luca and others in our team rebelled and they wanted to make it right. I thought we didn't have time for it. Luca proved me wrong and made it right -- i.e., now brunsli-like functionality is based on the core tech of JPEG XL instead of being an ugly clued-in hack. This was not purely for aesthetics, since it also removed the brotli dependency from the core code stream and thus made it possible to have smaller decoders. Also, that effort lead to having better LZ77 features -- more similar to those of WebP lossless with 2d distances and pixel (not byte) based indexing and lengths

also, the context modeling is more compatible with the large palette mode now in the LZ77 use

so, the brunsli removal lead to the brotli removal, which lead the new options in pixel graphics (we haven't yet explored them in the encoder), and to a 123456 bytes smaller decoder

and to more bitstream expressivity: adding alpha to an existing jpeg, or doing saliency progression with it, are things that are now possible

by nature I'm pretty stubborn and don't allow this kind of things, but I'm learning to be more flexible -- and I'm certainly now happy that I didn't resist this change enough to stop it

there are two kinds of engineers -- those who build stuff and those who wrap already function stuff to containers and new layers of APIs -- the latter kind can have a weird aesthetic where doing something is 'dirty' and needs to be isolated from other code by lots of stuff that does nothing

in one effort I participated 20 years ago there was an engineer who would write a function that would just call another function -- to have a wrapper layer, no functionality was added or removed

at that time I was mystified by it

these are the same people who get really happy about mostly useless wrappers and containers

I bet when they get home they put two table cloths on their dining table to avoid spilling on the other

also I observe that the people who create the bloaty wrapper can be as proud of building the wrapper as if they had create the image formats in it -- and other people can participate in this fantasy

AVIF/HEIF images with alpha are required to have this string, in ASCII, somewhere: `urn:mpeg:mpegB:cicp:systems:auxiliary:alpha`

<:PepeHands:654081051768913941> I developed framework that just wraps around encoders)

in some cases, wrappers do have utility. DICOM is an example, or those things they have for geography

the image is important, but the "metadata" is also critical

then it makes sense to define a domain-specific container that is mostly specifying how to do the metadata

That is a jxl in an isobmf with jpeg bitstream reconstruction data

That is an avif

DICOM is a complex dictionary for key/values for structured data, I wouldn't consider it just a wrapper

DICOM is based on very very silly engineers making the first version

ACR-NEMA 1.0 is the grandfather of it

they defined the cable to be used for data connection to have 16 data wires

and they brought that hardware detail all the way to the format

that is why there was a lot of 16 bit alignment rules in it, like strings needed to have even sizes

not sure if that is still the case, haven't followed the field for the last 15 years

this was at the time when it was already clear that Ethernet and friends are going to replace dedicated communication hardware and networking is stronger than point-to-point connections

16-bit alignment is also a big thing in Photoshop's PSD format

seems tangentially related to progressive rendering (though I doubt that in most use cases, it will be long enough to have such an adverse effect)

I think a lot depends on how gradual the deblurring is. A 15-passes progression is different from a 2-passes progression in that respect.

I'm thinking of 4k and 8k photography

if we have 8k, that is the worst case, and it means 33M pixels

if we do that with 0.2 bpp, 33e6*0.2/8 one photo will be less than a megabyte, 825 kB

the first progressive version of it will be about 160 kB

let's take a slow internet connection of 20 mbps, 2.5 MB/s

160 kB will take take 64 ms to transmit

the rest we can send in 270 ms

the time between the final and initial goes into what I consider a 'preattentive stage'

to let human vision to orient where to look at rather than already look there

What about just full HD (2M pixels) but at 1bpp and on a 3G connection? E.g. viewing a full screen photo on a phone in rural areas.

the average mobile connection today is 17 mbps

Averages can be quite different from medians in these things

2e6/8 = 250 kB, i.e., perhaps 25 kB for the preview

average 3g speed in India is about 2 mbps

so, 100 ms for the preview and 900 ms for the full image

if 10 % data for the preview

(it can be 15 % at 1 bpp)

In Germany 3G is 5x faster at ~10 mbps

speeds are getting faster quickly

Germany: 20 ms for preview and 180 ms for the full image

we often have images at 2.5 and more bpp today

Argentina is somewhere in the middle with 3G speed: 60 ms for preview and 540 ms for the full image

https://blog.chromium.org/2021/03/speeding-up-release-cycle.html

> we are excited to announce that Chrome is planning to move to releasing a new milestone every 4 weeks, starting with Chrome 94 in Q3 of 2021. > Additionally, we will add a new Extended Stable option, with milestone updates every 8 weeks. <:Thonk:805904896879493180>

I hope it helps, their quality really went down like 2 years ago. Finally had to switch to Firefox

At least now when JXL is adopted it will come out a little faster

And faster decoder updates

While we're talking about JXL adoption in Chrome... https://chromium-review.googlesource.com/c/chromium/src/+/2693607/ It stalled after <@456226577798135808>'s latest patches, we haven't heard from Leon Scroggins (the reviewer) yet...

we are moving the semicolons and curly braces around in the C++ code

more seriously, there are production warnings that we need to resolve -- they are not technically challenging as far as I know, just a bit tedious

Anyway, I did something rather special.

I use pooled butteraugli to make a butteraugli style video. <:monkaMega:809252622900789269>

https://drive.google.com/file/d/1NW3AUgBC1CcTJODUK_DscjNaD_-xhyen/view?usp=sharing

https://drive.google.com/file/d/155Cbh-AG9T931Hs9P_TTmfY3H_S3KHK4/view?usp=sharing

Flickering and seizure incoming...

ouch, my eyes xD

But yeah, it's very interesting seeing how rav1e and aom encode differently.

this map of compression artifacts has a lot of compression artifacts

we should run butteraugli on that too

Butteraugliception

Rav1e is a lot more flashy

More frames changes?

Essentially, rav1e seems to be smarter as where to do the distortions vs aom.

This allows it to spend more bits in places that matter, but that does create a bit more artifacts.

However, in this kind of video, that is actually a big advantage.

Is it possible to replicate the gif format - animated series of images with a color table?

Yes, except jxl does not have a palette that can be shared between frames, every frame needs to have its own palette

But we do have arbitrary palettes: any number of colors, any number of channels, there can even be "delta" colors that are not absolute colors but deltas w.r.t. a predictor

And the palette itself gets compressed as if it was a nb_colors x nb_channels image

That is a world's first. Achievement unlocked.

butteraugli gets seriously upset about small inaccuracies in movement

I have been planning (with spider-mario) a version of butteraugli for video, but haven't been able to staff it yet

I wouldn't make big conclusions about butteraugli images for video -- but perhaps it is ok to use it to see where to look at when evaluating video quality manually

I would be interested in exploring a variant of butteraugli and/or ssimulacra that gets weighted according to how important the fidelity is in different regions. For still images, it could be saliency-weighted. For video, it could be weighted based on motion.

Maybe use a different norm depending on the weight, or something like that

so many options

<@!258670228819410944> Reply => short name for butteraugli: https://discord.com/channels/794206087879852103/804008033595162635/817730448577789983 Call butter eye? I wanted more vowels than there are in PSNRHVS-M and MS-SSIM-YUV, an association with the human eye... and with a small bread (gipfeli, zopfli, brotli). I deliberately chose an overly complex term to avoid creating homonym noise for something as specific as this. Voisilmäpulla, Finnish butter eye buns, translated to German is something like Butteraugebrötchen, and I took the liberty to invent a new pseudo-Swiss-German word from it, and butteraugli was born. http://www.food.com/recipe/finnish-butter-eye-buns-voisilm-pulla-326192 -- Tasty with filter coffee. https://encode.su/threads/2395-SSIM-MSSIM-vs-MSE?p=47367&viewfull=1#post47367

Does this have 5 or 6 colors? Why are the differences so jerky?

🤔 https://www.phoronix.com/scan.php?page=news_item&px=Exiv2-Looks-To-KDE > Exiv2 Looks To Team Up With The KDE Project > Exiv2, the widely-used C++ metadata library / tools for dealing with image metadata via EXIF / IPTC / XMP standards and ICC profiles is looking to join the KDE project.

https://www.reddit.com/r/programming/comments/lzi2vt/after_being_defended_from_google_now_microsoft/

What nonsense is this

Claims: 1. A computer system comprising: an encoded data buffer configured to store encoded data for at least part of a bitstream; and a range asymmetric number system (“RANS”) decoder configured to perform operations using a two-phase structure, the operations comprising: as part of a first phase of the two-phase structure, selectively updating state of the RANS decoder using probability information for an output symbol from a previous iteration, the state of the RANS decoder being tracked using a value; as part of a second phase of the two-phase structure, selectively merging a portion of the encoded data from an input buffer into the state of the RANS decoder; and as part of the second phase of the two-phase structure, selectively generating an output symbol for a current iteration using the state of the RANS decoder.

What is this two-phase thing? I don't understand this lawyerspeak, it is always so very vague and generic

<@!111445179587624960> what you think about the article, be honest is informative, are there some grammar errors?

also scope are you English?

No, as I said before, I hardly know how to write in English correctly, I never learned English on purpose (I learned it from games, movies and reading the Internet)

did you read the article i did

BTW the one from bluesword maybe the site is more popular

chipandcheese com how popular it is?

changed the site theme

Some people on that Reddit thread seem to think JPEG XL is a Microsoft project 🤔

Wrong megacorp

they are probably thinking of JPEG XR

Jpeg's naming scheme is pretty confusing

JPEG XS, XR, XL,XT

What are they, IPhones?

I don't like it either, it's confusing

I generally try to avoid those names

XS sounds like excess JPEG

There's jpeg, j2k, wdp and jxl

XS is something most end-users don't need to know about, since it's aiming more for the professional video production market or stuff like that (it's not a format meant to store files in, really, but more like a cable thing)

Then there is jpeg XT, which is literally a jpeg plus XT stuff so there it makes sense to keep the "jpeg" in the name

Isn't that first claim just a vague description of _any_ entropy decoder that does context modeling and/or chance adaptation?

<https://encode.su/threads/2648-Published-rANS-patent-by-Storeleap?p=68945&viewfull=1#post68945>

Maybe `djxl` could decode to JPEG XT if simple JPEG couldn't be enough for some images? <:Thonk:805904896879493180>

I've been thinking about 'smuggling' JPEG XT support together with JPEG XL

Not sure if that would help much. The only reason to decode to JPEG is to deal with legacy clients, and they will typically not support JPEG XT. Upgrading them to support JPEG XT is as much work as upgrading them to support JPEG XL 🙂

<@!416586441058025472> I can't say anything about the article, because for me the universal advice on how to encode images in JXL is quite simple (`cjxl image image.jxl`) and that's all, well, unless someone additionally needs to change the speed (`-s`) and quality (`-q`)

That i could added

also the article is beginner english on purpose

i didn't include words like "just"

The rest of the fine-tuning is more likely to be even useless (without a full understanding of how they work, in addition, the encoder is constantly improving and advice on some of the options may quickly become outdated)

yes, an image comparison with different types of pictures would be better

and more interesting

also the r/av1 post is

what_is_av1_codec_in_a_10_second_video_save_and/

but still i don't understand why Google SEO don't index it

https://news.ycombinator.com/item?id=26390040

https://twitter.com/jonsneyers/status/1369170699793952769?s=19

A member of an ISO committee on HN had this to say: > Most (if not all) people on the committees don't like the paywall. The accountants have to explain every couple years why its necessary. Then people seem to forget and we have to do it all over again. It's cyclical.

It is only necessary because ISO wants to be a private company that makes money by selling specs (and membership fees).

In my opinion they should be a division of the UN or something and just get some subsidies to pay their staff.

Countries can tax company profits to pay for those subsidies.

It's peanut money anyway in the bigger picture

Paywalling standards is crappy, but should be legal. We shouldn't prevent them from shooting themselves into feet. However, the moment some lawmakers decide to include ISO spec in their laws, they should somehow make that spec free or refrain from using it in the law (e.g. by using open standards or creating in-house freely available alternative). By simple logic you shouldn't be persecuted for not following law you can't have access to, it's different from having access to law but not using it (*ignorantia iuris nocet*).

Another thing is intellectual "property". Property laws were made to solve the problem of physical items' scarcity. Fortunately information can be 100% perfectly copied indefinitely, you're only limited by the medium (so you can only own the information medium, not the information itself).

It's not just about including ISO specs in laws themselves. What if my government uses JPEG XL images on its website? What if they publish laws online using formats or protocols standardized by ISO? I think also in those cases, those standards should be accessible for free.