JPEG XL

I mean that the model is tuned so that the number you specify will likely achieve the bitrate across a large corpus of audio files

the model being good should have the *opposite* effect of making files usually be close to the specified bitrate.

right yeah

In my world VBR with target bitrate is ABR, true VBR is when I specify quality and let encoder do the rest. The same as with image encoders - we specify quality, not approximate desired size

it's not a target bitrate

It's a target quality, that's by design a similar number to bitrate

the units are of the same order of magnitude, but it *is* a quality setting

I specified 256, got 255 and 249 for two files. I think it IS target bitrate.

sample size of two

And I really don't care how it is called internally. It is called bitrate, results in very close bitrate - it is a bitrate. 🤷‍♂️

here's a random file I found l ying around

``` $ opusenc --bitrate 128 a.flac a.opus Encoding using libopus 1.4 (audio) ----------------------------------------------------- Input: 48 kHz, 1 channel Output: 1 channel (1 uncoupled) 20ms packets, 128 kbit/s VBR Preskip: 312 Encoding complete ----------------------------------------------------- Encoded: 10.02 seconds Runtime: 0 seconds Wrote: 196692 bytes, 501 packets, 13 pages Bitrate: 155.329 kbit/s (without overhead) Instant rates: 150.4 to 256.4 kbit/s (376 to 641 bytes per packet) Overhead: 1.09% (container+metadata) ```

It's literally VBR mate idk what else to say

You can say "I don't believe you cause the option says bitrate on it" but it's not actually ABR, it's true VBR

You could, read the docs, where `--vbr` is the default. If you want ABR, you can use `--cvbr` for constrained variable bitrate encoding (same thing), or if you want real CBR you use `--hard-cbr` for hard constant bitrate encoding

You can choose not to use Opus because you ignore all the information in front of you and decide it isn't what it says it is

It is has variable bitrate, I know, but not having quality setting and instead having bitrate turns it to ABR. This is how it was called at least since LAME. I don't know what is happening under the opus hood, but for me as external observer it is not true VBR just because of this reason. Once more - I converted two files with 44 and 11 kHz, and 11 kHz file resulted in bigger opus file. It is a nonsense for me

Once again, i remind you, it literally has a quality setting

and it is literally not ABR

You're straight up wrong about that.

I will not use it because it is not usable for me, definitely. I can't stand with the situation I described. So let's stop here

337 nero abr is the best

Opus is still behind

Yes you fed it two files and the quality setting you passed roughly matched the bitrate of the output file, **completely unusable** 🤓

Useful at 230 kbps rate for ffmpeg libopus

Or ffmpeg libopus

I have also suggestions for the bitrate to use in reddit

Tomorrow i will send

You are not required to use it, but you're making a decision based on objectively false information, that is well-documented to be directly false, and you are deliberately ignoring all the information that is available to you.

Do you think it is ok? When files with 4 times different "information" in it are encoded in the same resulting file size?

I tried 4 random files I happened to have lying around: ``` % opusenc --bitrate 128 Devin\ Townsend\ Project-The\ Mighty\ Masturbator.wav /tmp/a.opus % opusenc --bitrate 128 Laibach\ -\ Jesus\ Christ\ Superstar\ \(24kHz\).wav /tmp/b.opus % opusenc --bitrate 128 Timeless\ Miracle\ -\ Heaven\ in\ hell\ \(Demo\)\ \(2006\).wav /tmp/c.opus % opusenc --bitrate 128 I\ Won\ The\ Loudness\ War\ WARNING\ THIS\ MIGHT\ DESTROY\ YOUR\ SPEAKERS\ \[WSg_6Osx-eE\].flac /tmp/d.opus % for x in /tmp/*.opus; do echo; opusinfo "$x" | grep "Average bitrate"; done Average bitrate: 124.2 kbit/s Average bitrate: 120.2 kbit/s Average bitrate: 127.9 kbit/s Average bitrate: 180.6 kbit/s ```

Yes it divides it by the sample rate beforehand

I mentioned this

64 kbps for the latest 1.4.0 is considered low as it only changes the RDO

fab shut up

I guess one could make the criticism that the scale being made to correlate so strongly with bitrate makes it hard to intuit

So you have improvements when the song is 1:15

they don't have 4 times different information in them. It's perfectly possible that most of the audible content in the file is in lower frequencies.

like, what’s “d1”? is it `--bitrate 128`? `--bitrate 160`?

Ok, if it is logical for you, then we should definitely stop here, because our vision is completely opposite

For examples in si tu no estas by Nicky jam he says medallo

For coding it now 115 kbps is considered medium

I'm literally reporting what the documentation says. if you think the documentation is lying you can check the source code as well, it's all open source, and available online

But it was possible even with libopus 1.3.1-166

Just difficult you had a roughe sound

tbf what does "d1" mean by itself?

Bitrate were 400kb

sure it's libjxl's default but it's definitely not transparent

it’s supposed to mean a just noticeable difference

but it has drifted somewhat from that

You are intelligent do you ever studied?

I don't think that documentation is lying, I'm just saying that the whole opus behavior is not ok for me. And it is pity, because of it's quality

I agree with you

well what you think the "whole opus behavior" is directly disagrees with what the documentation says so if it is not lying then you are mistaken

Very useful feature, btw

It's 22:36pm I'll continue afternoon

Hi 😊 all

see you later

blocking is not a useful feature tbh because it doesn't actually hide messages, it just shows that blocked messages exist

it's mostly useful to prevent people from sending you unwanted direct messages

But it does not annoy with it's content

I think Oleksii accepts that it converts the specified bitrate into an actual quality, but thinks dividing by the sample rate is bad. He thinks the 11kHz file should definitely be a lot smaller. Because it has a lot less information in it, a quality targetting VBR mode should be able to make it smaller.

That would be true if the file actually had way less information in it

Exactly

Indeed, I made that point before. The 11kHz file may in fact still have most audible information in it.

This is how all other audio lossy encoders with VBR behave

In which case Opus was already throwing away most of the higher frequency content, which means that reducing the sample rate was just removing the data Opus was removing anyway.

It could be that other encoders aren't as smart about what is or is not audible.

Of course, this is just speculation, but I think your test is not very conclusive.

I always interpreted it as "d/100 people would notice in a blind test at original size"

So the default is 1/100 people at 100% zoom for JXL

not always the correct choice

I suspect that around d1 is probably the only point where this might approximately hold

Here's an example of two files that have identical audio content ``` $ oggenc -q 5 48k.flac Opening with flac module: FLAC file reader Encoding "48k.flac" to "48k.ogg" at quality 5.00 [ 94.1%] [ 0m00s remaining] - Done encoding file "48k.ogg" File length: 0m 10.0s Elapsed time: 0m 00.0s Rate: 211.8824 Average bitrate: 15.7 kb/s $ oggenc -q 5 8k.flac Opening with flac module: FLAC file reader Encoding "8k.flac" to "8k.ogg" at quality 5.00 [ 92.2%] [ 0m00s remaining] - Done encoding file "8k.ogg" File length: 0m 10.0s Elapsed time: 0m 00.0s Rate: 955.5662 Average bitrate: 6.1 kb/s ``` One's 8k, and one's 48k. They *should* be the same bitrate when you encode them to vorbis, but they are not.

d10 would likely be noticeably different for way more than just 10% of people

The difference here is only the sample rate. both of them are pure tone 440 Hz files

just a sine wave

Certainly a non-linear scale, but from d 0.5 to 3.0 like libjxl reccomends it probably lines up

d3 meaning 3% of the people wouldn't notice a difference doesn't line up at all

I can't say the information "quantity" is equal

it does appear from the spectogram that the highest amplitudes are all in the lowest frequencies

it's psychoacoustically weighted

lower than, say 6 kHz

information that makes a bigger different to what you notice is given more importance

Sorry, didn't get you

11 kHz to 20 kHz is barely an octave

They sound completely different

well tbf 11 kHz audio is capped at 5.5 kHz for audible frequencies

ah, right, 11 kHz was the sample rate, sorry

sure but you don't need that much information to actually sound all that different

if you hard lowpass any piece of music at 5.5 kHz you'll hear something wildly different, even if *most* of the auditory information is in that band

Yeah, Opus only spends a small number of bits on high frequency content

(originally) 11 kHz file is actually a bit bigger than 44 🤷‍♂️

yes, it certainly isn't perfect

This is (one of two) the thing I'm talking about

Ok, guys, it's time to sleep here

good night!

huh I just noticed that most Opus devs now work at AWS

https://www.amazon.science/blog/neural-encoding-enables-more-efficient-recovery-of-lost-audio-packets

There were a number of Opus people who became the video codec team at Mozilla, who then worked on Daala, which turned into AV1. Then Mozilla laid off the whole team, and some got hired by Amazon.

Do not be fooled by those thungs

Well, to sum up my complains to opus. Encoded two different music style and complexity files, one classic (1st part of Moonlight sonata) and one speed metal, both with --bitrate 256. Attaching spectres to get idea about files complexity. Here is what opus produced (TAK - lossless encoder to estimate complexity). As you can see, opus ignores very low complexity of classic file, and produces file with higher bitrate than lossless encoder. So no, for me, --bitrate is not a quality setting, it is a target bitrate. And here I'm stopping this hard offtop.

No continue

133465 40ms ffmpeg libopus newer if you want probably would sound also good or 172135 40ms ffopus of this night

Don't expect me to know more about audio and abiura. I can do one song at time.

I have two file of texts showing file audios in bitrate.

Updated bitrates screenshot with vorbis -q 8.0 files

In my opinion the bitrate on yt seems lower why

I did a playlist before this and I found lower quality than this

The resampling maybe they changing internally something at 5.00am

I have Philips uh220 since one year and it seems the playlist before this gotta an bad ugly sound

Why hevc 60fps is so fast to decode I didn't realized it was so fast to decode

<@1102876331986927627> new font dly High quality font for your MIUI

I did another version of that

The First is already downloading i'm sorry i will download even this

I didn't increased anything still 1358 tracks i only downloaded 12 of the ones i got ingested

But on this I need to it All

https://github.com/fraunhoferhhi/vvdec/issues/158#issuecomment-1875915496

Less autistic music version of that

https://open.spotify.com/playlist/7b93FT0p9XBgMOC6Y1N8Tw?si=MxzFEpmpTDy1zY55F6bhCQ&pi=e-0Fpya7wMQcyU

Anyway is fake as hell English will never listen to Summer music as where they live is -27c curve polare artico

https://music.youtube.com/playlist?list=PLnXJEiJ6Zb-z0lMg6fxsARz0zZmmvs7Io&si=a8gtBYxd_bUsI_YT

Even simplified to the max it doesn't appeal English because they have different climate.

And why my autism is so predictable by Fb?

Is there are pdf paper you can me link

Not avm codec

What prompts should I ask to chatgpt 3.5?

Rare example where parametric v2 is more efficient at higher bitrate than 64 kbps

64 kbps isn't the limit of specification this 64,21 kbps is the HEVC one

The HEVC one I sacrified video to get better audio

I downloaded with telegram the HEVC one

I'm complicating more ytm algorithm to get close to this

https://music.youtube.com/playlist?list=PLSC65hLOsV-rQKd-VrIcaB44TjnPSxaR7&si=Wdzc1wZhV_wpBCY8

Is a 4 minute work

I know how to do to even more in one minute plus 2 seconds

YouTube tell me the error 433

It's a new unknown error

Apparently rolled out now only I I unlocked this

https://music.youtube.com/playlist?list=PLnXJEiJ6Zb-zadV28_3VlfY9SOPF-WtB5&si=Q6yyZl3jt2_2ASK9

<@321486891079696385>

I finished entire YouTube music algorithm

Autism audio reliever

Which is the best encoded?

Assign a score from 1 to 10 to the average of my encodes

<@1102876331986927627>

I would like your opinion on this

Calm down fab

Also evaluate this image

Can you do those two tasks

I'm new to HEVC only 6 month experience

I don't know

Don't tell lies

I am new to hevc. Only 0.3 month of experience

I'm getting this error when trying to build libjxl-metrics-git from AUR for many weeks now: ``` [ 4%] Building CXX object lib/CMakeFiles/jpegli-static.dir/jpegli/bit_writer.cc.o a2x: ERROR: "xmllint" --nonet --noout --valid "/home/damian101/.cache/yay/libjxl-metrics-git/src/build/cjxl.xml" returned non-zero exit status 4 a2x: ERROR: "xmllint" --nonet --noout --valid "/home/damian101/.cache/yay/libjxl-metrics-git/src/build/djxl.xml" returned non-zero exit status 4 make[2]: *** [CMakeFiles/manpages.dir/build.make:74: cjxl.1] Error 1 make[2]: *** Waiting for unfinished jobs.... make[2]: *** [CMakeFiles/manpages.dir/build.make:78: djxl.1] Error 1 make[1]: *** [CMakeFiles/Makefile2:393: CMakeFiles/manpages.dir/all] Error 2 ```

💀 https://xol.io/blah/introducing-vcc/ https://www.opennet.ru/opennews/art.shtml?num=60413

Since a lot of people here already have scripts or methods to test and compare different encoders/tools, I thought I'd ask if someone could compare [Oxipng](https://github.com/shssoichiro/oxipng) to [ECT](https://github.com/fhanau/Efficient-Compression-Tool) Ideally with all speed settings (`-o 0` to `-o max`, with/without `--fast` for Oxipng. `-1` to `-9` with and without `--allfilters` for ECT) Results comparing time, size and memory usage I've done some brief testing myself but don't have an easy way to log memory usage, and being on a 7 year old CPU the higher levels can take considerable time too. Thanks in advance if someone takes it up

I was doing something similar ! to reproduce a part of https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=174429822

and I saw they used pingo and ect for png, but didn't timed it 😄

I found this and so far ECT is half the memory usage of Oxipng while being around 3 seconds slower on default, although I'm only testing one or two images at one or two settings https://github.com/cbielow/wintime

Default ECT vs `-9` ``` PageFaultCount: 9383991 PeakWorkingSetSize: 330.5 MiB QuotaPeakPagedPoolUsage: 25.2 KiB QuotaPeakNonPagedPoolUsage: 5.438 KiB PeakPagefileUsage: 328.5 MiB Creation time 2024/01/10 19:43:24.298 Exit time 2024/01/10 19:44:00.669 Wall time: 0 days, 00:00:36.371 (36.37 seconds) User time: 0 days, 00:00:07.390 (7.39 seconds) Kernel time: 0 days, 00:00:28.937 (28.94 seconds) ``` Just a tad slower ``` PageFaultCount: 25908451 PeakWorkingSetSize: 330.2 MiB QuotaPeakPagedPoolUsage: 25.2 KiB QuotaPeakNonPagedPoolUsage: 5.438 KiB PeakPagefileUsage: 349.2 MiB Creation time 2024/01/10 19:47:14.237 Exit time 2024/01/10 19:54:51.023 Wall time: 0 days, 00:07:36.785 (456.79 seconds) User time: 0 days, 00:00:21.093 (21.09 seconds) Kernel time: 0 days, 00:07:15.671 (435.67 seconds) ```

For 3.4% further improvement

objective number one is to bench jpegxl but I wanted to find a worthy opponent for png. I don't know if I will have time to review all png optimizer... ect seems to be a good horse

Yeah, this is a 8,000 x 4,000 image I'm testing on, so I'd try JXL but don't want to slam my RAM with another 6GB encode

😄

6 Gb is just a chrome session, almost nothing 🙂

Chrome's already using 3 and I've only got 16 total xD

Well, after some testing it seems `-5` on ECT is the best, still half Oxipng memory with a further 10% savings compared to default although at double the time JXL is the best after that at `-e 7` reducing by a further 25% in only 7 seconds (ECT was using about 70)

Windows 11 consumes 4gb-5.5gb when idle. 🙂

Yeah, around that on my stagnating Windows 10

Some modded win11 like tiny11 use only 1 to 1.5

most of it scales based on total memory

makes background services run faster

Well, Windows doesn't count cached memory towards used memory, but yeah, a lot of the time it's fixed percentages

(lol, extract from the png spec : PNG is pronounced "ping" http://www.libpng.org/pub/png/spec/1.2/png-1.2-pdg.html)

<@238552565619359744> I didn't found a pingo version for linux, crap...

Btw, I always recommend using oxipng first before using ECT, as it does more things like remove unused alpha channel, bit depth reduction and stuff. Also, it lets you remove metadata WITHOUT removing essential color metadata.

I use pinga for jpeg/png optimization. However, it does not support 16 bpc pngs, reducing it to 8 bpc. But other of that it works great and also have various options about removing not necessary data\metadata

In my case I want to keep all metadata due to storing date, location and players in game screenshots that I take

makes sense, but it's optional anyway

Just did a quick test and ECT also removes unused Alpha channels and reduces bit depth, although ECT lags slightly behind Oxipng at max in those cases too Around 0.01% difference, from 4 MB to 1.87 KB and ECT ended up 2.16 KB)

It does not remove unused alpha channels in my experience

with unused I mean a fully transparent alpha channel

I did a simple test with ffmpeg at compression level 0, got a 32 bit file out, ran that through ECT and got a 24 bit result

It seems on my 1 bit image test Oxipng is actually far better than even genetic filtering on ECT. Using zopfli only took 9 seconds for 1791 bytes, ECT took 140 seconds for 1797 bytes. Also 28 MB of RAM instead of 32MB for ECT Miniscule differences, but interesting

Could you please share it?

<:PepeHands:808829977608323112>

400 seconds on max ECT vs 10 for zopfli on OxiPNG, and ECT still looses by 12 bytes on this

(yes it's horrendously artificial)

It saves as 396 kB webp, is it intended?

That's... Strange...

Maybe you're getting WebP from Discord's CDN?

Trimming ?... from link helped

For 64 bit images (Exported from Krita as 16 bit per channel) ECT is far more efficent, although neither palleted or lowered the bitdepth like for 8 bit``` wintime -- oxipng -p -r -o max --fast -Z SmileOxi.png Processing: SmileOxi.png 49559 bytes (99.83% smaller): SmileOxi.png PageFaultCount: 575218 PeakWorkingSetSize: 879.1 MiB QuotaPeakPagedPoolUsage: 32.89 KiB QuotaPeakNonPagedPoolUsage: 10.22 KiB PeakPagefileUsage: 995.6 MiB Wall time: 0 days, 00:05:04.566 (304.57 seconds) wintime -- ect -p -r -k -5 SmileECT.png Processed 1 file Saved 28.17MB out of 28.21MB (99.8314%) PageFaultCount: 1853274 PeakWorkingSetSize: 157.7 MiB QuotaPeakPagedPoolUsage: 25.2 KiB QuotaPeakNonPagedPoolUsage: 4.906 KiB PeakPagefileUsage: 175 MiB Wall time: 0 days, 00:00:11.623 (11.62 seconds) ```

Didn't crank ECT up because of how long Oxipng took compared to the 8 bit file, so trying that now

Yes, pinga failed on this image. 3346 without stripping anything, 2391 with

https://www.phoronix.com/news/GNOME-46-Alpha-Released Jxl wallpaper is coming in this version

sure

done

Did you Exported it?

https://fxtwitter.com/nixcraft/status/1748315798714622174?t=syzYxePsuQ_Ier2lD3dB4w&s=19

I learnt that while reading the spec the other day, my friends were not amused

ninja'd here https://discord.com/channels/794206087879852103/794206087879852106/1194949619382226944 😄

https://youtu.be/RYJf7kelYQQ?t=276

I know it is supposed to be "ping" but I always say Pee 'N Gee...

Everyone says that

Just like how everyone says gif not jif, despite what its creator says.

i say jif

yes, it's "Dschee Eye Eff", not "Gee Eye Eff" afterall

It's "ping" because it stands for Pirtable Network Graphics

And "jif" because it's the Giraffical Interchange Format 🦒

when i hear that argument i say jfag

maybe it was Portable *Internet and* Network Graphics instead 😉

Here we say gif (and it's derivatives, that assume that it is something small and of female gender), and png is something like "Pee 'N Gee" © but with local specific 😅

https://youtu.be/Nrk8sqZfsgI

could have been pong then 😄

I hate when people try to logic out why it's "actually supposed to be gif with a hard g"

the same people likely don't say "jay-feg" for JPEG image

considering that in English, a g followed by an `e` or an `i` is usually soft it makes sense, people just want to invent pronunciation logic where it doesn't exist

Gift

it would be more consistent with git, gift and get, though

though not with gib

ah, sorry, for gib, it varies

gist, ginger, giant, gin

it's pretty variant

if you dig into it there's really no consistency

a problem with gift is that if you say gif like that it sounds too similar

that's why i hate when people try to logic it out

English pronunciation is pretty random

monkey and donkey don't rhyme. why? /shrug

it's also not even consistent within regions

my dad was born two states east of me and he pronounces "hover" like it was "hahver"

and no he's not from Boston

I read somewhere that when given an unknown word, English speakers guess the correct pronunciation 75% of the time.

which is like 2 bits of entropy in every word

something like americans getting it right vs brits because spanish sounds are what is in most other languages

I'd believe that in the south but I have no exposure to spanish where I live

having a 25% chance of not being able to pronounce a word you have only ever seen written is pretty high

here is an argument not based on trying to logic out the pronunciation: hard g is vastly more popular, purely due to that being how the average english speaker thinks it's supposed to be said when they first read it.

indeed. I believe the figure for Mandarin is about the same.

and mandarin doesn't have letters

well Hanzi has radicals but it's not the same thing as letters

it's true that "gif" is much more likely to be said by those who haven't heard it before

I say "jif" mostly because when I grew up the format already existed and my dad called them "jiffs"

since he knew about them

||i refuse to call anything yiff /joke||

if you want to cite some authoritative source, the OED cites both as pronunciations for that word

it just straight up lists both

should we start calling JXL "Jixel" or should we stick with "Jay Ex Ell"

at least in Chinese, the characters are often picto-phonetic, with one component indicating pronunciation.

yea that's what a radical is, isn't it?

Hanzi and Hangul scripts are constructed from radicals

i've always had the idea you can figure out the sound by the radical. I never figured it out but some of my friends did

this was from learning when i was 6-12

https://en.wikipedia.org/wiki/Radical_(Chinese_characters)

this is what I'm referring to

yeah. the components of a han character can be phonetic, semantic, or neither. about 80% of characters have a phonetic component.

but then you have the Japanese students who see the Kanji but since they don't sepeak Mandarin they just have nothing to go on

it's all memorization

since iirc there's minimal correlation between the pronunciation of a Kanji in Japanese and the correpsonding Manadrin pronunciation of the corresponding Hanzi character

tfw you have fewer characters (like 2000 common ones instead of 5000) but every single one has between 2 and 5 arbitrary pronunciations you have to learn.

they have to teach people how to read through middle school

and I don't mean like, in the US where english class is about literature

it's just straight up reading

kind of an inconvenient language ngl

japanese pronunciation is similar to at least the cantonese

it's still chinese

go (wu region/south china) and kan(han dynasty) readings are from chinese

I think it's like, one pronunciation based on the Japanese word corresponding to whatever the han character means, and then additional ones based on however it was pronounced in different places in China like various numbers of hundreds of years ago.

anyway it's jif

maybe I should learn some variety of Chinese instead of Japanese

just learn mandarin

more useful

that's a variety of chinese :p

No way, it is gif

brb learning Taiwanese Mandarin

Bit too late, we have probably the easiest case - what you see is what you pronounce, no exceptions afair. Unfortunately it is the only easy part of language 😅

going to pronounce every name wrong because what i see is what i pronounce

including yours

As you wish

My view: people can call it whatever they want. I like something that rhymes with pixel (jixel or yixel), but I am not even doing that myself and instead use the "just spell it" approach (so Jay Ex Ell in English), so who am I to say what's the correct way.

0.8 bits, no?

a jixel must surely be an element of some kind, just like a pixel (picture element) or a texel (texture element)

"Joy Feg", it's Joint, not Jaint 🙂 Yeah that's a good point, nobody does that. I guess an acronym does become a word on its own in a way, and typically you just spell it but if there is a pronounceable part in it that looks like a syllable, you just pronounce that part like a word.

1/log(0.25)

A jixel is a giraffe element 🦒

where is that formula from? shouldn’t it be: -(0.75·log₂(0.75) + 0.25·log₂(0.25))

yours goes to infinity as the probability of getting it right goes to 100%

If you can guess correctly 75% of the time, that's less than one bit of info you have to guess

and 50% should have maximum entropy, and it’s 1 bit

I miswrote it, I meant `log(1/0.25)` but yea I suppose what you wrote is more correct

not necessarily, because "50% probability of guessing correctly" for a *fair coin* is 1 bit of entropy

but that's not what's happening here, since a 50% probability of guessing correctly for how an english word is pronounced is a different thing

consider a case where you only have a 10% chance of guessing it correctly

vs 90%

these aren't actually the same scenario, like they would be for a coin

since 10% chance of guessing correctly would mean the letters have minimal information in them that tells a trained speaker how to pronounce the word a 90% chance of guessing correctly would mean that the letters have a lot of information in them - almost all the information needed to actually pronounce the word

right, my formula is something like the entropy of guessing whether someone will guess correctly, rather than the entropy of guessing correctly

but AFAICT, that’s all we can compute from just “people get it right 75% of the time”

in the case of guessing correctly for a coin, there's only one way of beign incorrect but that's not the case with english words there's a whole massive space of theoretically possibly pronunciations

and being correct is choosing which of the many of them, not choosing which of the 2 of them

so in that regard a word has significantly more than 2 bits of entropy

since 75% of the time you get it right

we'd need to figure out which words are those which are guessed correctly, and then how much information is contained within a correction of pronunciation of the remaining 25%.

who wants to fund a massive study

:D

<https://archive.org/details/pingo-linux> if you want it (the Windows build does run with a translation layer like Wine, but you'll have to handle errors on filenames with certain characters)

on the Remez algorithm: I meant https://en.wikipedia.org/wiki/Remez_algorithm

this is particularly useful for floating-point approximations where each operation is not so much ideal mathematical one

oh this, that's how I generated the srgb approximation polynomial in the first place

oh then that's good

I thought you just generated a Chebyshev polynomial

not an arbitrary one no

it is rare to see a Remez implementation that can handle fp inaccuracy as well though, except for sollya AFAIK

use rational polynomials! they are the best 😛

and feel free to copy algorithms from https://github.com/libjxl/libjxl/blob/main/lib/jxl/base/fast_math-inl.h

jxl-oxide copied a lot from there

or from lib/jxl/cms/transfer_functions-inl.h for that matter

yeah make sense

coming up with those approximations is loads of fun 😄

does that work better than a higher-degree poly approximation though?

I think it will be faster than higher-degree poly because of data dependence, not sure

it also works much better for most functions

in terms of accuracy

Probably irrelevant for modern computers, but I keep remembering this every time I read "polynomial" https://youtu.be/hffgNRfL1XY

this stuff is definitely still relevant for whoever deals with image processing

of course assuming that we don't use a naive horner rule

I think a pairwise horner rule is frequent enough?

https://github.com/ar-lex/cue2tracks/blob/master/cue2tracks

really, in bash?

is HWY_REP4(x) effectively the same thing as `x, x, x, x`

Yep

out of curiosity, is floating point multiplication and addition commutative?

I know it's not associative

I'm wondering if a compiler can take `b + x * a` and generate a single multiply-add instruction

FMA gives a different answer than doing the operations separately, but IEEE-754 allows, in some cases, value changing optimizations as long as the value is closer to the true mathematical result than what you'd normally get, as long as there's a way for the programmer to control this behavior.

I'm avoiding intrinsics since I want the code to be maximally portable but I'd also like the compiler to generate the relevant optimizations when targeting an instruction set that supports them, e.g. x86_64

In C, this is controlled via `#pragma STDC FP_CONTRACT`

what's that do?

if you give it the argument ON, it allows contracting operations into more precise operations, like contracting `b + x * a` into an FMA. If you give the argument OFF, this is not allowed.

The default behavior is implementation-defined.

is `#pragma STDC FP_CONTRACT ON` permitted by ISOC99?

lemme check

looks like that's when it was added

note that most compilers default to it being on by default

according to cppreference.com:

> ISO C language standard requires that C compilers support the following three pragmas, and some C++ compiler vendors support them, to varying degrees, in their C++ frontends: > ... > #pragma STDC FP_CONTRACT arg (2)

here is the C version of that page without the C++ talk: https://en.cppreference.com/w/c/preprocessor/impl

oh ty

It says `(since C99)`

it does

I'm defaulting to C99

....

> warning: ignoring ‘#pragma STDC FP_CONTRACT’ [-Wunknown-pragmas] thanks gcc

real helpful

rip

alternatively you could use this function? https://en.cppreference.com/w/c/numeric/math/fma

incurs a dependency on libm.so

which I currently do not have

it does look like GCC has switch `-ffp-contract=fast`

which enables the generation of FMA instructions even when it doesn't produce the identical result

and it's enabled by default

Yep

what are some good Werrors, btw?

I have `-Werror=implicit-function-declaration` on as one example

and `-Werror=format-security`

I tend to werror everything and then disable warnings I think are bad.

can't do that reasoanbly cause compilers differ

your compiler might not warn about something that someone else's does

and then suddenly your thing doesn't build with another compiler, or another version of the same compiler

warnings should simply be entirely disabled outside of development 😄

hm, gcc 13.2 on godbolt doesn't emit an FMA unless I add `-mfma`. Gotta check which gen added fma and which gen gcc defaults to.

I wonder if `-ffast-math` breaks fast-inverse-square-root and other bithack algorithms

gcc defaults to `-march=x86_64 -mtune=generic` iirc

unless you tell it otherwise

oh, FMA is Haswell and newer, or Piledriver and newer

so, FMA is x86-64-v3 I guess 🤔

oh wow, `-march=native` makes a massive difference

cuts the time down by almost 50%

that depends on the compiler settings (`-fp-contract` I believe is the flag)

I tried disabling `-march=native` and encode times increased over 90%

turns out fma is pretty great

yup, see discussion

yep that

just got to it now 😄

yup yup, fma is pretty useful for these things

it pretty much halves the cost of DCTs and the like

too bad it's super arch dependant what you actually get 😢

I wonder if gcc is smart enough to replace `a += b * c` with `a = b * c + a` and then add an fma instruction

ah, on your prev question: afaiu mul and add of floats *is* commutative

good, so that means teh compiler can turn a + b * c into an fma(b, c, a)

I don't see what that has to do with commutativity.

(other than in an imo very trivial sense, that is by no means sufficient)

ah, yep, IEEE-754-2008 defines `fusedMultiplyAdd(x,y,z)` as `(x × y) + z`, so you need commutativity of addition for the specific example you gave.

if `fma(a, b, c)` is defined to be `a * b + c`, and addition is not commutative, then you can't replace `c + a * b` with `a * b + c` before contracting into an fma

but since it is, that works fine

I do think fp addition and multiplication are commutative, just not associative

I know, but it seems trivial to me since you could just write your code differently. (Since, presumably, if it wasn't commutative, the difference would be more than small enough for your use case, even if it prevented automatic reordering by the compiler.)

You could, but then I'd have to rewrite the code

<:Madge:859968580996956200>

im lazy

yes if it weren't i could just literally put the addition on the other side

but <:Madge:859968580996956200>

😄

<@167023260574154752> did you hear about Java's fma btw added in Java 9

I discovered this when I was working on jxlatte

I have not

so they created an annotation called `@IntrinsicTarget` which is wrapped around some standard library functions

it's not something you can define yourself

but it's a flag for the jvm that calls to this function can possibly be replaced with an intrinsic by the jit compiler

so I was like "oh cool, I'll just replace all my fmas with calls to `Math.fma`" and everything will be faster

and then it *was* until

until someone reported that on their system with the last commit it slowed to a crawl

and I was like "hm, I wonder why?"

turns out **if** your system has fma instructions, them `Math.fma` is replaced by those instructions in the jit compiler

on the jvm

if it *doesn't* then it wraps each `float` argument to a `java.lang.math.BigDecimal`

performs a BigDecimal multiplication followed by a BigDecimal addition

and then calls `floatValue()`

Lol

no, it doesn't just do `a * b + c`

💀

Actually the C fma function might have the same issue

and there's no way to determine at runtime whether or not this will happen

that's the best part

to the java class files, they're just calling `Math.fma`

which does a thing

I think the spec says that it has to give the result that you'd expect given only one round of rounding, unlike the double rounding you get from a*b+c

you can't determine at runtime if the call to `Math.fma` *would* be replaced by an fma instruction

and if not, instead just do `a * b + c` which is much faster than the pure java implementation

Might even be a ieee 754 violation to have a function called fma that doesn't work like this

yea, but it makes the function entirely useless

because it becomes unusably slow without the instructions

like, if you don't care about the precision loss for whatever reason there's *no way to tell the jvm* to replace `fma` with `a * b + c`

it just won't

it calls BigDecimal

I'll send a letter to the ieee people asking for the next revision of 754 to include a fmaButIdcAboutPrecision function

At startup run a benchmark and then use byte code patching to select the faster version

(I have bad ideas when I need to sleep, good night)

oh god

fuck no

> This method corresponds to the fusedMultiplyAdd operation defined in IEEE 754-2008.

yea, but still

it's kinda useless

They could at least use a more efficient way of doing it than BigDecimal

Like using doubles if the inputs are single precision

Or pair of doubles when the inputs are doubles

(which is what libc implementations presumably do when there's no fma support in hw)

Actually, what probably happened is scientists writing slower but more accurate number crunching routines using such techniques, and then asking Intel and IEEE to speed it up for them.

hey, that's more or less how ifuncs do work

https://sourceware.org/glibc/wiki/GNU_IFUNC

by the way unfortunately there seems no easy way for opportunistic fma

in fact, FP_CONTRACT is possibly the only easy way

otherwise you have to manually probe the cpuid and equivalent

(for example, GCC `__builtin_fma` would turn into a libm call unless FMA is natively supported. but `__has_builtin(__builtin_fma)` always return true in either case. I know this is technically correct, but I want to know whether it's useful...)

found glibc's implementation: ```c double __fma (double x, double y, double z) { return (x * y) + z; } ``` 🤦‍♀️

it *could* be special-cased in the compiler

clang does that for i.e. memcpy, IIRC

so does gcc

if you actually want glibc’s memcpy, you have to really want it

fma as an instruction probably is, but what about the software fallback? `Computes (x*y) + z as if to infinite precision and rounded only once to fit the result type.`, but that expression rounds twice, so it isn't compliant.

I would not be surprised if the compiler special cases how `__fma` is compiled

this created some funsies with ffmpeg's allocation it always mallocs with posix_memalign instead of with malloc

since a lot of simd doesn't work on unaligned memory

turns out there was a regression in glibc that made posix_memalign slow

following the thread is wild tho

https://sourceware.org/pipermail/libc-alpha/2023-August/150653.html

> There were a LOT of other size requests. The smallest I saw was TWO > bytes. WHY? I'm tempted to not fix this, to teach developers to not > use posix_memalign() unless they REALLY need it ;-)

it was eventually fixed tho

that's one heck of a performance regression

you are very likely to look at the fallback implementation. search for other files named `s_fma.c`..

(I do agree that glibc is structured in a somewhat confusing way though)

I did find another file, forgot to post about it

Since it's a math.h function, I looked in the math folder

But turns out there's an ieee754 folder with the proper algorithm

yeah `sysdeps` is a whole can of confusion

pretty much *any* file can be overridden from those directories

Kinda confusing because the proper implementation is just normal double math. Way more of it, but ought to work everywhere where the simpler impl works.

I believe you can't even try to implement fma if you don't know which fp impl is being used (as opposed to IEEE 754)

Oh yeah, I guess the fallback gives a decent answer even on non 754 systems

Forgot those exist

are there any systems that have 4-byte floats that aren't IEEE-compliant

I mean there used to be but do they exist anymore

https://youtu.be/nJlZT5AE9zY

Part of the automated pallete selection based on CIELAB mentioned near the end of that video https://fixupx.com/moulberry/status/1741995924237533515?s=20

That file opens fine on gimp.

which file?

that's fine, I don't mind in this instance

also, sorry for the late reply lol

why is the xyb image look black and white? am I encoding it wrongly?

XYB jpeg? shouldn't be

not unless the original is grayscale

source is a png file

I wonder if your viewer is struggling with RGB jpegs

I tried 3 different apps