JPEG XL

(I had someone else generate this for me since I'm not on the Steam beta branch)

Its in steam stable now

Youtube is encoding 88600 videos a second

WebP 🤮

So <@440391953545166848> if you want I can introduce you to the JXS spec writers if you don't know them already

Thanks for moving this here, the slow mode was killing me. I dont have any contacts with any of the people on the JXS committee, I'd love if you could share those contacts

Mail me at jon@cloudinary.com, I'll introduce you to them

Awesome, I appreciate that

If you're interested in JXS, you might also be interested in HTJ2K. <@553324745240608773> here can tell you all about that.

I'm not very familiar with HTJ2K at all. Our target is implementing the encoding on smaller FPGAs, which is also why we decided to go with JPEG XS. Does HTJ2K offer similar size advantages for hardware implementations?

Yes.

Happy to put you in touch with FPGA implementers if you DM me your email address and provide more information on the application you have in mind.

That does sound quite interesting. Were looking to use an open implementation if possible however, which is why we are going to such lengths to implement it ourselfs

https://medium.com/@migel_95925/supercharging-jpeg-with-machine-learning-4471185d885d learning the best Q-table via ML does sound legit, but does it work that very well? (I've also found a similar approach which was publicly released, https://medium.com/@colbyn/modern-image-optimization-for-2020-issues-solutions-and-open-source-solutions-543af00e3e51)

I kinda suspect a sort of metric overfitting, the best metric `compression.io` did use was MS-SSIM for example

We are basically doing something like this in our next q_auto. It works as well as the amount of real training data you have — which is subjective data, not metrics. This was the main reason we did the CID22 experiment.

would that outcome also make into jpegli, just in case? that would be an even more efficient JPEG encoder on top of the current jpegli if that observation is generalizable enough.

We're not trying to make encoders better, just get better at using them to produce consistent quality.

(and also to get better at using the best available codec for the image)

Especially for AVIF, getting consistent quality is kind of hard.

OTOH: https://opus-codec.org/demo/opus-1.5/

xiph.org demos are always worthy to look at and marvel.

Tim Terriberry was talking about this when I saw him in Palo Alto two months ago. Pretty cool stuff!

some AV1 encoders do not support 4:4:4 WebP has no 4:4:4, but like AVIF supports lossless RGB

I also helped until 10:00 AM of today to improve opus encoders

Better I mean more similar to the original recording

Before I spent about 27:40 minutes optimizing Tok Tik audio

Because with the Song alex velea monalisa there' was some muffling

svt-av1 is 4:2:0 only libaom has lossless rgb support, other encoders do not so it depends although I don't see the point of lossless avif screenshots even for high quality lossy, jpeg is enough and may be even better at higher qualities with much faster speeds

And I did managed toSolve that

Brave beta waz installed on my PC

Still it was badly mixed but the voice heaac quality was ok

It's amazing that you can understand what someone is saying at 90% packet loss

though, it doesn't work that well in actual use on unstable networks

Steam added it for HDR screenshots, but I'm sure there would've been something better

shouldve at least improved jpg saves. Their encoder gives really poor quality for all resolutions

inspired by the libjxl 0.10 post, and by the fact that wavpack 5.7 was just released with multithreading support, I decided to run a quick benchmark against flac

for convenience, I ran it on only one file, but it lasts 45 minutes

(green = multithreaded wavpack, orange = single-threaded wavpack, blue = flac)

oops, forgot the labels

x = compression ratio, y = compression time in seconds

decoding speed seems to be pretty much a non-issue (even the slowest single-threaded wavpack is over 300× real time)

(the test file was https://youtu.be/kp7VHwvRzoY )

uugh, YouTube’s new, mandatory dynamic range compression completely removes the fade in at the start, I hate it

of note: wavpack’s multithreaded decoding appears to work just as well on files that were compressed single-threaded

1.5 has released?!?!?!?! omg omg omg

Nothing difference 1.4 1.5 .opus file after encoding.

Mainly ML based low bitrate / high packet loss improvements

werent the 1.5 specific changes disabled by default ?

most of them, all the ML ones were but there are other misc changes that made it in

afaik error loss resilience is in but only noticable in severely degraded streaming situations

forgive me for my ignorance, but can someone enlighten me as to why you would *ever* choose baseline jpeg over progressive jpeg? with progressive, you get both: - smaller file sizes - progressive decoding what do you have to lose by going progressive? even ect or jpegtran can losslessly convert between the two, so you aren't even losing any data by doing so

faster encoding/decoding if that is important for such speeds, better compression on some images

Yes

Mozjpeg and jpegli do progressive by default for a reason. The only disadvantage is enc/dec speed (and memory), but I don't think that still really matters for jpeg in 2024. Even progressive jpeg is very fast to decode.

Jxl does simple progressive in two passes by default so you get the advantages but not really the disadvantages (which in case of jpeg come from doing many passes and no tiling, which is bad for memory locality so comes at a speed cost)

Wonder why Android (at least Pixels) produce baseline JPEGs by default, then

encode speed, probably?

Hardware encoding

That or just not bothering to change libjpeg-turbo default, I guess

I've news for you, rewritten all the yt quantuzers

How many times have you done so?

Zero

I have a video of my cousin singing with autotune and a Romanina song hluego

But I don't know but many papers were released in february

zero inMarch

i miss when psy wasn't utilized <@226977230121598977>

i dont use psy

The problem with PSY is a poor explanation of the principles of its operation, it is not written anywhere at all, no details. From my tests, I can conclude that PSY is forbids to updating the block with small changes and "twitches"

I also can't know that. but it seems that such a function is extremely highly configured in VP9, which makes it possible to observe not smooth updates of information in blocks, But Sharp changes in them. But it would be nice if some person who worked directly with PSY in x264 x264 libvpx -- would explain everything as it really is

you were right on this, stop with this psy, keep grain

even webp lossy without psy is acceptable

My techniques are improving

Very low bpp still most people recognize the object

Likely not good as JPEG XL

It cannot be as it would slow down the politicans x account

So I have to keep the WebP look

<@226977230121598977>

King of compression

https://media.discordapp.net/attachments/1145767720718172200/1205207845029552209/1088142012546564206.gif

I would really like to look at the WEBP q100, which has completely disabled the use of DCT

Do you guys know if there is a compressed vector-based image format like SVG?

I know there are SVG "optimizers" that try to losslessly strip SVGs of redundant/"useless" data, but Brotli does a much better job than these most of the time, even on compression level 1

I wonder if anyone has standardized such a format? like an `scvg` or something; `Scalable Compressed Vector Graphics`

Ah, I guess there's `.svgz`

yeah, even brotli -2 will beat gzip's most effortful setting

You can do both, optimize and gzip

Someone shared this meme exploiting a bug

What i'm worrying about is how SVT av1 psy behave with tune 0

yeah, but even then, with this image Brotli wins

sorry about that. I think Blue is spreading FUD, we haven't seen the problematic clip yet so we can't be sure exactly what's going on

<:PepeOK:805388754545934396> <#805176455658733570>

Works I did dday, blazemedia Hdblog

My Optimizations aren't lossless

Here I did a jxl like deblocking inspired techniques

The video was 243kbps and the encoding was suffering in vp9 cause av1 bitrate

Obviously that could be skipped as not important unless you buffer on tv

Final results with jxl filter

This between also the boosting technique i implemented on yt -yrm are enough for today

And in 1,8-3,1mb vp9 everything improved from one hour ago

This is 360p vp9

With h264 green tint perhaps

Is problem of my optimization except the one in blogs

On videos I'm poor

<@853026420792360980> <:Thonk:805904896879493180>

webp

i guess it's something in bmp, but if not, something is broken in ffmpeg conversion this strange block

yeah, it's just a weird source

if I had to guess it's an swscale bug

but without more info I can't diagnose

i checked and it's the same for imagemagick, so maybe it's not ffmpeg

just some weird data in the alpha channel when capturing, maybe it's a cursor or something like that

yeah, so it's not a conversion bug it's weird ffmpeg gdigrab behavior

<:Hypers:808826266060193874> https://gitlab.com/AOMediaCodec/SVT-AV1/-/issues/1920#note_1818055602

I was running oxipng on a directory and it decided to do this around an hour or so in

I've had it mess up a few times, fragmenting output across lines, changing the color of the window, all sorts of mess that mean I have to restart

I set `--preserve` so it's *almost* impossible to tell what files it has or hasn't already processed however luckily voidtools everything has it logged

idk if there's a better way to do this but I'm just going to move everything oxipng has already processed to another folder so it doesn't waste effort and time on the same files

you could do something like this `for /R . %I in (*.png) do ( oxipng.exe -v -o max --preserve "%I" )` so that it won't mess up in the middle of doing a full directory, only downside is it will only do a single file at a time and also the process will be created and then destroyed each time it's done with a file

oh. apparently it did finish as this is all that's left in the folder when I move all the stuff processed by oxipng out

Yeah, most times my garbled cmd is just visual, with it still working but the progress being unreadable

Haven’t had that occur on Mac or Linux

I don't know how to work with anything other than the Original Jpeg in 2024, everyone is chasing crazy resolutions and quality, but apparently no one takes into account the decoding speed images are photographed already at 8k resolution what other format besides JPEG can be opened in such a resolution and quickly by the way, I recently noticed how instagram previews have switched to webp hmmm It's funny that q95 - Jpg is superior to webp in high quality. just like mp3 is superior to opus But ordinary normis don't need to know about it, they can get crazy yesterday I thought for a long time how to get around this damn cloudfire so that i download 420 thousand art, as a result, nothing works: curl wget and other "low" programs. so I took all the links and wrote them in the html index and opened this. (its works, i downloaded 420k img) what am I talking about — I can't even imagine what would happen to the processor and memory if there was something other than the classic jpeg baseline yuv420 — because only it can be opened in the amount of 70 thousand on one page inside chrome. jxl, avif — definitely not possible... even png.

quite recently (a couple of months ago) I abandoned png for large images, and now I use jpeg yuv420 q100 (not loseless) in order to give away some of my work. I'm only doing this because it decodes quickly, and takes up less memory than all other formats (I was shocked by the speed of jpeg decoding in chrome, and how much RAM it takes up. it spends 6 times less memory, unlike for example xnview, probably because of some special way of store the decoded image, Not with pure pixels)

I assume baseline JPEGs? progressive ones would probably fill up ram, also iirc chrome uses libjpeg-turbo for decoding

yes, exclusively baseline. I always clarify (usually)

There is a strong difference between in memory usage and in decoding speed, which I involuntarily imagine is a different image format

while jpeg baseline is faster than any of the existing compression format at all (!)

How is "mp3 superior to opus"?

browsers or cdns can or by default lazyload parsing the rest of the images far below the fold (to save bandwidth mainly). 'infinite scrolling' got popular as it hides the rest of pages and only parses the next blocks

~~it is superior in size for the same quality~~ <:KekDog:805390049033191445>

its just a workaround like how progressive decoding is simulated for images - load a very blurry version of a pic as placeholder, then swap the blurry placeholder with the full image once it finishes loading

mp3 is so good at preserving cymbals

Don't open that pandora's box.

there is no point in making a blur beforehand. it can be done in HTML CSS

📂

<:KittyGasp:1126563391939547198>

https://cdn.discordapp.com/attachments/662412664252923924/1079978957086150799/soyboy-opus.jpg

https://tenor.com/view/yakuza-like-a-dragon-yakuza-gaiden-like-a-dragon-gaiden-gaiden-gif-6758678452832849548

My relief at that not being a video

Challenge accepted.

https://bsky.app/profile/videah.net/post/3ko4lkdt2jv2v

ok thats really bad unexpected

I encountered a handful on Google images already. Failed to copy and copy link to discord

i wonder if its 2+ framed avifs pretending to be progressive

new to libraw, which one is correct?

Define "correct" 🙂

hmmm....idk maybe as seen from DSLR's viewfinder display?

left looks like tonemapped to lower target peak brightness compared to right

you mean left looks brighter than right

I've seen avif at least 2 years ago, on rule34 of all places

as uploads? only ones i saw were served as delivery-only images by cdns

even then only for small images. according to cloudflare large images are brutally intensive to encode so they serve webp in their place (stuck maintaining 2 workflows instead of settling on any one format and leaving better compressed og jpg as the fallback)

it was from a cdn, indeed They do use it for thumbnails, but also normal posts, but it's more rare

yeah, but also compressed dynamic range in the highlights

but doesn't really look different for me aside from brightness

i was trying to point out that it's completely subjective because it's a photo from a camera

realism is kind of an objective reference...

on a brighter screen the right image might look more realistic, on a darker screen the left one...

I only realised yesterday that if I wanted to 'calibrate' image intensity for my monitor, then because I have the brightness set to half in the monitor's settings, I'd need to halve the value too (Or at least close to it)

brightness is subjective in reality because our eyes and cameras adjust to the environment

so the experience of something being "bright" can be the same when something is 1 nits and 10000 nits

imo thumb should be closer perceptually (your eyes' perception changes according to your *own* current and previous exposition to lighting). the raw has much more information and will be much higher quality when its processed to look similar to the jpg thumb. the hw jpg codec processes (badly though) the raw input both for thumb and a jpg final image

i imagine it's just this, with the thumb being that

Our eyes also adjust to screen brightness. But yes, surrounding outside both screen and camera frame is relevant...

nobody is going to see anything 50 nits when outside in the sun

and photographers still don't care about hdr

stop it. HDR is a curse

this is confusing. Because our eyes also adjust to a screen, it's even less relevant

they do https://gregbenzphotography.com/hdr/

With naked eyes I can't, at the same time, read a book in the sun and find my keys in my bag... Storing hdr image is amazing but for real, how many bits of range would I need to faithfully render an image on a screen to match the dynamic range of the retina?

that's why some dynamic compression is good and HDR shouldn't be purely about maximum realism

that's what subjective means

<:trollface:834012731710505011>

well, with video, a lot is about temporal brightness change

wth is webp2

nothing relevant it seems

they were going to make a real image format based on AV1, but it was canned in favor of "just chuck a plain AV1 stream inside a HEIF container" i.e. AVIF

about 12 stops https://dl.acm.org/doi/10.1145/1836248.1836251

(~5000:1)

that’s “at the same time” (no adaptation), so the range that you might want to show on a screen “ever” is wider

Thank you for the sourced answer!

At some point I heard that non-CRT screens can't even display 8bit per channel colors, they use temporal dithering to compensate. Is this info outdated in 2024? How many bits screens can display without dithering?

i think there are some crappy TN panels that are natively 6 bpc and use temporal dithering to get to 8, but otherwise afaik every panel is honest about its bit depth

CRT screens cannot even display more than one pixel at a time, they use temporal dithering to compensate 🙂

lmfao

now i'm thinking about making a crt so big it has to use multiple electron guns

but that sounds like hell cuz the tv signal is tied to how a single gun works, so you'd have to buffer the signal or something

Slow-Mo guys did a slow motion of OLED and plasma screens. Very interesting

if one takes into account random variations in CRT output (i.e. “noise”, but if I just said “noise”, one might think I was referring to the 16 kHz sound), how many bits of dithering is it equivalent to?

if someone is claiming dithering to be a specific defect of non-CRT screens, it strikes me as the sort of argument/misconception that is sometimes used to argue that vinyl is superior to CD

(reminder: with proper dithering, the only deviation between the true signal and the quantised signal is noise https://secure.aes.org/forum/pubs/conventions/?elib=11829)

https://youtu.be/cIQ9IXSUzuM?t=12m15s > Dither doesn’t “drown out” or “mask” quantisation noise, it actually _replaces_ it.

(a bit more on this: https://www.strollswithmydog.com/sub-bit-signal/ https://blog.kasson.com/the-last-word/detectability-of-visual-signals-below-the-noise/)

<@604964375924834314> you who are ressourcefull, do you have something which explain how to do noise shaping ? I can grasp the big principle, but miss something when I try to go down an algorithm for it (in 1D)

Back when I heard that statement, most LCD monitors used VGA connector anyway

nothing off the top of my head, but I’ll make sure to get back to you if I come across anything

I'll post it here if I find something before you

So I gave it a try: https://positron832.neocities.org/blog/2024/mp3-vs-opus-triangles/

😳

So yes, opus sucks here

low bitrate MP3's are poorly encoded using parameters without modification, which makes their sampling rate lower than OPUS, which is unacceptable. According to the documentation, mp3 up to 64 kbps can work in stereo at 48000 hz

As the instrumental is literally same dark

Just open headphones on a conor maynard or justin bieber tracks

Or Travis Scott

They have lot of this night sound

I optimized the audio but don't know if sound good to average people

I just did abx on the file you can had followed

<@416586441058025472>❌

Do you know of any encoders capable of that? I recall LAME using lower samplerates at low bitrates regardless if a high samplerate is specified

Ok argument

What wrong on how justin bieber and matteo milazzo sounds in a instrumental

Use paint with the painter yellow colour

--resample 44.1 --resample 48

You're right sounds like he has rinco in his mouth

yeah

<:BlobYay:806132268186861619>

stereo 32 kbps 44.1

I just tried it and wow you're right

<:PepeOK:805388754545934396> Knowledge is power

but here is the same story as with JXL q0 - the developers decide for the user that if he really needs to compress so much, at least let it be possible to listen to it.

Please tell me it sounds Better''

Make a new screenshot, ogg file

I'm not Justin buwbeR fan

❌

Is still same

Or worse

I wonder that Ringo in his mouth vocoder where it Comes

And if engineers can fix it the damage I cared

LAMEworse quality parameters retain higher frequencies more, experiment from -q 0 to -q 9

It still sounds bad but I made in this way 2 years ago and yt fixed after 2month

At least the bass sounds better

I'll have to play around with the encoders' settings. Going offline

Spamming with vp9 videos isn't the solution

You have to trust the engineers that can like give requisite of adopting and adoperating the right and precision tools

No i think with the edit i did not anymoore

Ok now i'm comparing on a phone in my memory

This video hurt me... From it being rendered in 480p yet uploaded as 4K, to all the missing info and facts https://youtu.be/NxzMAYckaV4

what I learned from this: 1. Lossy DWA compression exists in EXR, which is very cool. 2. Blender apparently handles different bit depths incorrectly, either when exporting, importing or for the pixel diff tool...

for dwaa yeah, it helps a decent chunk and it's still significantly better then PNG for exporting. Oh PNG, how you ruined countless exports.

also note that when working with RGBA you cant compare exr to png

well unless the alpha is always opaque I guess

also I dunno what blender does, EXR is a special format that is really flexible and allows data that "isn't seeable" so to speak. For instance with a png the max value will be 1.0, this isn't the case in EXR

so comparing EXR to PNG is often not really possible to do as a true "Apples to Apples", though the comparison can be considered accurate even if the reasons are wrong, it is true that EXR can retain significantly more information that PNG can

Comparing 32 bit to 16 and then saying "PNG IS AWFUL" does make you wonder how thorough his testing actually is...

more like 24 bits to 16 bits

nice thing with float is that you basically can't get clipping, which isn't true of png

he compared 16 bits to 8 bits, and also the diff he was looking was definitely incorrect, something was wrong there

how do you mean?

I mean technically he was comparing 32 bit float to 8 bit ints ;P

with ints, you get N bits of precision, and the range is intrinsically tied to the precision. With floats, these are decoupled, so it's easy to go to arbitrarily huge numbers without hitting a limit. With ints it's a lot easier to accidentally hit the maximum value and get clipping. E.g. in audio usually the range for ints is -2^15 to 2^15-1, and for floating point it's -1.0 to +1.0. You can't go over 2^15-1, but you can easily go above 1.0 in intermediate workflow steps with no issues.

oh, that probably explains the significant diff between 8 bit PNG and float source then, because there was definitely something wrong there

I need a refresher... How does lossless WebP multithread again? Is it tiles that reduce compression, or something else?

Found a typo that reduced config.thread_count to 1 instead of 6 in an application using libwebp, so trying to figure out if that's causing major slowdowns (without having to complie and then try to use it myself)

DWAA/DWAB is always half float

maybe this is a reason for "bit depth incorrectness"

half float?

it's not rounding errors

major clipping could maybe explain it, though

16 bit float (half) yes. Also Blender usually loaded float images as a half float too. Do not remember if they changed this behaviour. But it possbiel switch to 32 bit float in image properties

even if you compared say a double float PNG, half float EXR would still be better since PNG clips at 1.0

just scale your content into the 0-1 range 💀

why would such software not do that...

that sounda horrid

No. It will not. PNG use fixed precision 16bit across whole 0-1 range. EXR DWAA/DWAB have float precision where values higher than 0.5 slowly degrade to 11-12bit precision.

Good visualization: https://www.shadertoy.com/view/4tVyDK

does DWAA lead to more fidelity loss? yes. However even with that the range of data you can keep with EXR + DWAA is still significantly higher then PNG. Hell PNG can't even store alpha correctly, even completely disregarding any other benefits, PNG's lack of proper alpha handling makes it unsuitable for any high quality render that uses alpha unless it's for distribution

and ofc you can always just use lossless compression

Yep, PNG is doomed. In 3DCG/VFX no one use it in right mind

Watch image format creators ruin a great image format with one funny line

[av1_dogelol](https://cdn.discordapp.com/emojis/867794291652558888.webp?size=48&quality=lossless&name=av1_dogelol)

what's that?

"The colour values in a pixel are not premultiplied by the alpha value assigned to the pixel."

lol

also, I don’t know how often it’s used in practice but EXR supports per-channel alpha

so that you can have e.g. coloured glass and the like

Yep, EXR as a container more like PSD. More interesting feature is variable bit depth and color management per channel/layer

So a game called `Content Warning` released yesterday, and is free to keep until tomorrow afternoon. It features recording video clips on a camera, that are saved to your actual computer... I was naturally curious so I took a look myself...

`Running FFmpeg encode... -y -thread_queue_size 512 -r 24 -i C:\Users\jonat\AppData\Local\Temp\rec\6d633436-44c4-4737-a1e5-b7d8f4fa46e1\eb14f1a8-a2b6-4d26-972d-3daf85a576cc/Test%04d.png -f f32le -ac 2 -ar 48000 -thread_queue_size 512 -i C:\Users\jonat\AppData\Local\Temp\rec\6d633436-44c4-4737-a1e5-b7d8f4fa46e1\eb14f1a8-a2b6-4d26-972d-3daf85a576cc/audio.raw -f f32le -ac 1 -ar 48000 -thread_queue_size 512 -i C:\Users\jonat\AppData\Local\Temp\rec\6d633436-44c4-4737-a1e5-b7d8f4fa46e1\eb14f1a8-a2b6-4d26-972d-3daf85a576cc/mic.raw -map 0:0 -map 1:0 -c:a libvorbis -ac 2 -ar 24000 -filter_complex amix=inputs=2 -c:v libvpx -cpu-used -5 -deadline realtime -pix_fmt yuv420p -speed 1 -preset ultrafast C:\Users\jonat\AppData\Local\Temp\rec\6d633436-44c4-4737-a1e5-b7d8f4fa46e1\eb14f1a8-a2b6-4d26-972d-3daf85a576cc/output.webm`

It saves every frame as a 32 bit PNG then encodes a VP8 video at awful quality, which may be a stylistic choice for 'analogue horror', but to me it just looks like a blocky mess

(Naturally it looks better in a smaller window/Discord's preview)

What surprises me is the ffmpeg binary bundled in the game also has VP9 and Opus among many other encoders already, but I guess they wanted compatibility an speed above anything else

thats interesting

jpegli 😵

https://m.youtube.com/watch?v=qugf6TBsq1o&themeRefresh=1

Not bad

Does premultiplied alpha just mean pixel values on a white background essentially in practice?

What does it matter if it's a black or a white background?

Premultiplied vs non premultiplied

Wouldn't premultiplied be more complex to render?

https://community.adobe.com/t5/photoshop-ecosystem-discussions/change-in-exr-open-from-cs2-to-cs3-can-this-be-fixed/m-p/1521042/page/4#M918

an excellent post on alpha

note "premultiplied" is a lot of a misnomer, people are now more or less using the terms "associated" and "unassociated" now due too it being a much better representation of what it is actually doing

I don't fully understand yet. So far they seem like equivalent terms because in practice the RGB values are premultiplied by the alpha

Zap's post later on explains it in more detail https://community.adobe.com/t5/photoshop-ecosystem-discussions/change-in-exr-open-from-cs2-to-cs3-can-this-be-fixed/m-p/1521072/page/5#M948 but the TLDR of it is associated alpha better models how light actually works. particularly around how renderers work another useful post on the matter is https://groups.google.com/g/ocio-dev/c/ZehKhUFqhjc particularly this part > a nice intuition for what would > otherwise be corner cases. Consider a pixel where rgb > alpha, such as > 2.0, 2.0, 2.0, 1.0. Nothing special about this - it just represents a > 'specular' pixel where it's emitting 2.0 units of light, and is fully > opaque. A pixel value of (2.0, 2.0, 2.0, 0.0)? Nothing special > about this either, it represents a pixels that's contributing 2.0 > units of light energy, and happens to not occlude objects behind it. > Both of these cases can cause trouble with unpremultiplied > representations.

Captain Disillusion has a video about this : D https://youtube.com/watch?v=XobSAXZaKJ8

his video isn't wholly accurate, he gets across what makes it needed, but his technicals aren't quite right, he kinda falls under the "taking the premultiplied part literally"

> Chris, > > since you claim to know exactly what I mean, without actually > discussing this with me outside this rather bizarre thread, I > would like to clarify that I, as the original author of OpenEXR, > completely agree with Zap Andersson's post earlier today. 😂

this thread is almost as good as a chromium thread I once read, I wish I could find it but it was lost to time I suppose. There were some extraordinary good and bad takes in it

None of these pixel formats are designed to realistically imitate the behavior of light, which is why it's hard to intuitively reason on the subject

If 0=transparent and color values are premultiplied, and linearly represent the amount of light, then that gets pretty close but wouldn't there still be black fringes?

Ignoring everything else about RGB pixels. Ignoring gamut, color space, so on and so forth. RGB, when you're talking about associated alpha, actually does do a fairly decent job at it. With Associated Alpha, we can actually think of RGB as actual units of energy itself and alpha as actual transparency of an object. Think of it like a candlelight. With a candlelight, you have the luminous glow of the flame, but it has no solid object except for the flame itself, which would be built, let's say, 50%.

You need to completely disregard the idea of premultification. It is not what it is. It in no way represents how the actual Alpha Association works.

There is a reason why we are moving away from the term. it's because it makes no sense at all.

But most of the time the digital values do not linearly represent energy

another good one from zap which indirectly explains to why its a misnimer https://twitter.com/aaronkennedy0/status/1583891205339086849 > Exept it's not a hack. Say a raytracer shoots 10 rays into a pixel, 7 hit a red object, 3 hit nothing. The hits are treated as alpha=1 and misses 0. When you average the samples for the pixel you get 0.7 red and 0.7 alpha. Without "muliplying" anything, "pre" or not.

thats irrelevant,

how light is shaped doesn't really matter, its still a representation of energy

When interpolating or resizing it matters

that doesn't seem immediately relevant to the topic at hand

Because the difference in energy between two digital values is distorted by the gamma curve

This doesn't really seem relevant to the topic at hand, unless I'm missing something

Isn't the topic about mixing/resizing/resampling and blending pixel data with alpha?

and it's common for tools that do interpolation or resizing to convert to linear light before doing the operation, and then back. It doesn't really matter.

It's pretty relevant

and when I say common, I mean you have to do that

blending yes. if you dont linearize before composting its wrong regardless

^

doing it in non-linear light is wrong regardless of any alpha stuff

I thought most tools were more naive than that

Bad tools are...

krita...

kdenlive...

bad tools are common

Gimp at least handles it correctly

Okay, maybe it's not relevant if clever tools that linearize when mixing and resampling are also common

rather then being clever its a matter of being right and wrong

krita vs gimp is a good example of wrong vs right https://cdn.discordapp.com/attachments/719811866959806514/1216585057112821851/image.png?ex=660a26a0&is=65f7b1a0&hm=29e2c0ee0ff4369a2f0c562d3c97c4721e2bfaa9d83a44fbed862b2ec270582f& https://cdn.discordapp.com/attachments/719811866959806514/1216585057561739284/image.png?ex=660a26a0&is=65f7b1a0&hm=31b405536b45b2e83a3d692726e217c99a37c120a8aa2b90b84b3913d98f8e8c&

did Adobe ever get their shit together?

iirc they still use stupid hacks

It is risk knowing that tooling like OCIO if I remember correctly will treat all alpha as associated simply because it turns out if you treat unassociated alpha as associated it's not really that bad in a lot of cases

also here is a quote from troy sobotka, author of hitchhikers guide to digital color (excellent resource btw) > It is closer to the basis of how light transport works under an RGB model. This is easily verifiable when we think about something like a reflection on glass; there’s no degree of occlusion, and solely additive light. > Only associated alpha handles that. > (Gritz was one of the earlier people using the TIFF terminology, and it is damn wise as it helps people not get stuck thinking it is a simple math thing; it’s not!!! There is literally only one encoded state of the light, and it’s associated alpha. The other one isn’t encoded, and it can’t represent occlusion and emission properly.)

ah crap stupid discord mobile

What's wrong with calling it premultiplied? It has to be multiplied if it's unassociated

because you're not doing any pre-multiplication

You're not doing it because it's already premultiplied :)

no

if you happen to have unassociated alpha data, then doing the multiplication for the unassociated over operation give you associated alpha data

``` unfa 04/27/2021 9:33 AM I keep thinking in terms of "pre-mutiplied" alpha, where I'd only want to affect the alpha component. troy_s 04/27/2021 9:34 AM You obviously don't want to scale occlusion beyond the 0-100% range, but for light it is acceptable, and required. This is all associated alpha, which is why premultiplied is a beyond shit term. It makes people think about math, and the wrong math. unfa 04/27/2021 9:35 AM Oh yes, sometimes this creates problems. I hope we'll soon have a way to choose affected channels soon, so I can avoid having alpha >1 when I wanted to increase exposure. troy_s 04/27/2021 9:35 AM You never want to separate alpha and RGBA. Ever. They are associated The sole time this is required is for colour operations. If you think "Are these light emissions associated with the degree of occlusion?" it is a helpful mnemonic. (edited) ```

but if you never had unassociated alpha data to begin with, that multiplication never happens!

hope OCR didn't bugger this too bad

it literally isn't

I mean in some cases i could be

but its completely irrelevant to how it works

Ok, so if you want to increase emissions separately from occlusion...

Although occlusion will always still multiply emission...

It makes sense for RGB values to be out of range but not occlusion/alpha

But the emission is still a multiple of occlusion...

So the term still makes sense to me...

what

its not, what happens when alpha is 0?

Fully occluded means that pixel has no emission either.

the pixel is what's doing the occlusion

if you have an alpha zero it means the pixel has no substance you can see, nothing solid

does that mean the energy disappears? No!

Take, for example, the glow of a candlelight. We can't see the glow unless it's bouncing off dust in the air. Does that mean the energy disappears? No, it's still there.

If you have an occluded pixel then yes, the energy from that pixel is reduced by a factor of it's occlusion.

energy doesn't disappear just because it has nothing to reflect off if

It's a multiple

it will keep going until it hits something that does

but it isn't occluded??

like let's say it's the front-most pixel

and has alpha=0 but RGB>0

Energy can escape from partially occluded pixels but only by a factor of its original energy times the occlusion factor

You need to find what you mean by occluded?

can you define*

Meaning that it was already multiplied, or pre-multiplied if you will

STT took a crap

Well from what I'm reading, it means that those pixels are going to be covered up by a different texture.

If you are blending two textures together using an alpha mask

As the simplest example

let's have a concrete example, say 1,1,1,0 as the background, and 1,0,0,0 as the foreground

what do you get?

Occluded just means covered up

A pixel that's partially occluded would be blended with another pixel's color using linear light blending

the result would be 2,1,1,0

no occlusion, just the foreground pixel adding light

If both have an alpha of 0 then the result would be transparent nothing

wrong

that's only true of unassociated alpha

No, that's just what you expect when you add two of nothing together. You get nothing

go read the links that were posted earlier

well assuming the background of the media player has no solidity, it would *look* like nothing

despite the energy still being there

the energy would reach the camera

a candle flame is transparent but emits light

that light will reach the camera, even if there is no solid background

Not if I put an object between the camera and the candle

it can depend on how you are modeling the light. if there is nothing to reflect off of in cases it will be as if there is nothing

That's occlusion

but how is that relevant?

its not unless its partial occlusion

https://en.m.wikipedia.org/wiki/Alpha_compositing

if you put something in front, you get occlusion regardless of whatever alpha the background might have had

It's relevant because the alpha channel represents occlusion when it's associated alpha

and partial transparency is something that both associated and unassociated alpha can represent

yeah, which means alpha=0 is perfectly sensible, and obviously the color can't be a multiple of it

occlusion can be a bit of a hard term here. I prefer the term solidity, it means the same thing, but its easier to visualize

exactly this. if color was a multiple of alpha, this would break a lot of things

Occluded meaning that it's covered up by something else

Which is a good way of describing what is happening to the pixel values

After blending

and when you have a pixel with a value like 1,0,0,0, that means it has color, which will show up on your screen, but it won't occlude anything

If it's zero then it's fully covered up or invisible

no, if it's zero it isn't occluding anything behind it

it isn't itself occluded

only if you have a renderer that assumes background is reflective or the camera can see the energy directly

0 means invisible right?

I tend to think that cameras can see photons

it means light passes right thru it

but if it's emitting light itself, it will be visible

it transforms the over operation into the add operation

if you set alpha 0, a lot of renderers will only show if the background is solid

Lonnie, I don't think that is how it works.

why you should never use alpha

i onyl go for beta

based

I find that a bit silly, but, I think either way it completely contradicts what Pashifox is saying

Sorry, I meant to reply to this.

a candle flame is nearly transparent, but is highly visible because it emits light

it better models how humans see light

If alpha is zero it doesn't add anything

it does

do the math with the equation on the wiki page you linked

with associated alpha, it's C_out = C_foreground + C_background\*(1 - A_foreground)

what happens when C_foreground is 1 and A_foreground is 0?

It's wrong to call it occlusion then

its not

you just have a wrong interpretation of it

Because it's not being occluded by the pixel underneath

If it's being added

> occluded by the pixel underneath

????

since when do objects closer to the camera get covered by objects further away

Candlelight is really a perfect example of this because you can have a unit of candle light that you cannot see because it's not being reflected into your eyes or the camera, but you can see it in part energy onto things around it.

Well if the pixel underneath is covering up the texture on top... when blending two textures together based on an alpha mask the distinction between "top" and "bottom" is a meaningless one

And occlusion just means which one is covering up the other

Why would the pixel underneath be covering up the pixel on top that doesn't make sense?

The alpha mask or alpha value is a weight of how much each pixel covers up or gets covered up

"Layers" work top down, You can think of the topmost layer as the closest object to you and the bottommost layer as the object furthest away.

For something to block something down, it would have to be in the forefront

Now you can do special stuff where you take a mask and bring it to the forefront, That is possible. But something in the back will never block something in the front under normal circumstances.

ofc directionality is based on light source

I think the alpha channel represents how much each pixel will block or get blocked by the pixel it's blending with

no

the alpha of the background pixel is actually irrelevant to the blend operation

That's what the word "occlusion" makes me think and understand

And then it makes a lot of sense to say "premultiplied"

If you have something with alpha one in the foreground, and it will block out what's in the background, thats about it.

Because the emissive values are indeed a multiple of how occluded they are

do you live in an alternate reality where a tree outside occludes your window curtains?

My transparent glass window pane is not being occluded by a tree outside when the light shines through, it's just shining through. But when combining 2 pixels together the word "occlusion" just means which pixel wins

no

what?

All it means is how much does the foreground block the background.

Foreground and background are kind of arbitrary depending on how the computer algorithm is implemented.

no

no its not lmao

It's just blending 2 colors together

foreground and background is *fundamental* to alpha compositing

It's literally a pixel

like please read the wiki article you linked

and?

every operation is defined in terms of foreground and background

monitors display pixels. They're not arbitrary at all.

and foreground and background are not treated the same

if you want both inputs to be treated the same, you need to do something other than alpha compositing :)

An alpha value is just a weight of which pixel gets more weight

no

Is the final color closer to this pixel? Or that pixel

well a candle flame has an alpha of nearly 0

With associated alpha, all the alpha does is say how much does it block the background? That's it. That's all it does.

but it will still show up much more than whatever is behind it

its literally a modifer that determins how much background color to add to a pixel

thats it

nothing else

if you have 1 alpha, you add none of the background

C_out = C_foreground + C_background*(1 - A_foreground)

if you have 0 alpha, you add all of the background

Unless you're outside in the sun in a brightly lit environment. Then the candle flame will look almost invisible because of how bright the surroundings are. I'll re-read the article to make sure I'm understanding what it says

indeed, as long as what I'm saying is true in at least one situation, my point is made

Keep in mind, you cannot see light unless the light is being reflected into your eyes. But that does not mean the energy disappears.