JPEG XL

the capability should be there, but it should be restricted to a user controllable degree

even from the point of general use standpoint, a bright image could make it hard to focus on the rest of the page so normalization is good there anyway

to me it's similar to pages that autoplay sound - it's unexpected sensory input which can be annoying (and even embarrassing when others are around)

Depends on how dense the code is, but here's an inverted one and it should still work

interesting, with one QR code reader I can read that fine, the other doesn't detect it to be a QR code

while I'm not sure if QR specification itself allows that, I think there is no other technical reason inverted QR code can't work

it's mostly a matter of finding finder and alignment patterns

In general it seems like optic recognition of inverted things is often bad.

Even tesseract has problems with that

https://cdn.discordapp.com/attachments/1006242291968655481/1120336171449450556/Tumblr_l_116901570484102.png

lol

Xll

Time to spill avif compression.

Spilling avif compression on a subtly textured background like wood, concrete, marble, or a carpet could be interesting

if I'm using 8-bit JPEG for example, I'm not able to reach 16.7 million colors (as advertised on sites like apple.com correlating with '8-bit color') because only Cb & Cr are responsible for color reproduction right? Meaning there's likely a substantial difference between 8-bit RGB & other color spaces in terms of color reproduction

rgb(10, 10, 10) and rgb(118, 118, 118) count as “different colors” in those 16.7 million, so ycbcr(10, 128, 128) and ycbcr(118, 128, 128) would, too

but the only way to have Y=0 or Y=255 is with R=G=B=those values, so chroma cannot vary for those

they're defining "color" to include luma here, not just chroma

since RGB -> YCbCr is an invertible linear transformation, it's fair to treat them as equivalent for this purpose

Okay that make sense. That’s what was confusing me

do keep in mind that stuff like libjpeg will frequently convert YCbCr to RGB to you for convenience

so internally being YCbCr isn't necessarily exposed

I’m assuming an XYB -> RGB transformation is also reversible?

Does that mean 24bit RGB is the same as 24bit YCbCr?

Afaict they have the same range of color representation

you'd have be be careful when you say "24-bit" because do you mean bits per channel, or bits per pixel?

per pixel

So 8 BPC?

but the actual number of bits doesn't matter

there's also range issues, many video-based things that work in YCbCr will use limited-range

just so long as they're the same, is it the same colors?

Which is essentially 7 BPC right?

RGB -> YCbCr is an invertible linear transformation. They aren't the "same" but they can be converted from one to another

without precision loss

(contrast with, say Linear Light -> sRGB, which is a *nonlinear* transformation)

oh okay

So JXL supports 8, 10, 12, 16, 24, & 32 BPC right?

& the 32 BPC mode is float only, so greater precision in a particular section of the colorspace right?

it really depends

modular is entirely integer

and in theory uses up to 32-bit integers, but they're not scaled

& JXL can mix and match modular & VarDCT blocks in the same image?

they're just, whatever is coded, is.

for modular, that is

I thought JXL supports arbitrary channel bits up to some int?

VarDCT strictly speaking doesn't "have" an internal precision, since the specification only defines how to decode VarDCT in terms of real numbers

In practice, the maximum BPC for a jpeg xl image would be 24 or 32?

32

Okay, but the fact that it is floating point only doesn’t change anything?

well for modular specifically it's an entirely integer mode

no real number arithmetic

For VarDCT

VarDCT internal precision is really whatever you choose to use when you decode it

This is what confuses me

essentially the way VarDCT encodes real numbers is by encoding raw integers and a number of scale factors

in practice current implementations will be ~24 bpc at most, which is likely enough for the foreseeable future 😄

What is HDR?

the final entropy-coded value for a VarDCT coefficient ends up being an integer, but since there's a good number of scale factors in a numerator and a denominator, as well as an inverse cosine transform, you will end up getting a value between 0.0 and 1.0, as a real number

basically you have integers and some reconstruction instructions to get them to sample values between 0.0 and 1.0

I’m more familiar with how AVIF tries to compute differences, & CfL means that rounding is less precise at 8 bits so 10 bit AVIF is a marked increase in quality over 8 bit even with an 8 bit source. I’m assuming JXL doesn’t have these oddities

how much precision you choose to use when decoding those instructions is really up to you

well, up to conformance margins 😛

yea

if you're too imprecise you end up breaking spec

but you could, say, use 32-bit floats, 16-bit floats, or even doubles

the spec defines tolerance

So internally, JXL doesn’t have fixed bit depth targets is what I’m understanding

yea, the bit depth tag is a suggestion

Damn, that’s cool

it's often used to represent the bit depth of the input

so you don't try to decode more information than was there originally

since decoding an 8-bit JXL to a 16-bit buffer doesn't really help you if it started as an 8-bit image

And the VarDCT doesn’t have integer frequency coefficients like JPEG does, it has fractions for more precision? Before entropy coding

do note, this is with lossy and XYB encoding

and VarDCT, specifically

Wait, so that option is set based on input? What if 1bpp works better- could someone design their encoder to automatically select the minimum number of bits?

in modular mode, the bit depth tag matters, as the integers are coded as-is

so 4096 just becomes 4096

you need to tag the image as 12-bit to map 4095 to "full white"

if you tag it as 16-bit, 4095 is 1/16 dark gray

cause in modular, in particular lossless modular, it's coded as-is

what do you mean?

What if the source bit depth is oversize for the information?

it depends, there's some internal modular coding tricks used to help in that case

for example, if you feed it an 8-bit image that only uses two colors, the libjxl encoder will detect that

and internally use a palette transform

oh nice

That’s super cool

8-bit YCbCr are about 4 million colors (the volume is kind of like a pyramid on top of an inverted pyramid, meaning about 3/4 of the space is empty). But JPEG uses more precision internally and you can decode it to higher precision YCbCr — this is what jpegli does but libjpeg-turbo doesn't.

Does mozjpeg do this?

mozjpeg is a libjpeg-turbo fork, it also uses 8-bit YCbCr

I though all of JPEG’s frequency coefficients were internally coded from -128 to 127 which is 8 bit

YCbCr and XYB are mathematically reversible if you do them on real numbers (so infinite precision). With finite precision (quantizing to integers each time) they're not reversible though. YCbCr is linear (just a matrix multiply) so the precision loss is uniform, XYB is nonlinear so precision loss is different in the darks than in the brights.

No, the coefficients are 12-bit

Oh, so why is JPEG limited to 8 bit color? Just how it was standardized?

This is why lossless recompression of jpeg is "more lossless" than converting a jpeg to a png

So losslessly transcoded JPEGs are 12-bit JXL?

I watched the Computerphile series to understand JPEG, that’s where my background knowledge comes from. I’m not sure how much they oversimplified

That's just what the popular implementations did. Also the original JPEG spec actually has a mode for 12-bit input which uses 16-bit coefficients internally. But that didn't become part of the "de facto" jpeg standard, just like lossless mode, arithmetic coding, hierarchical jpeg etc

So jpegli is *actual* 12 bit & standard decoders understand this? Or it can just operate with a >8 bit source

And I’m assuming this is RGB jpegli, not xyb

Or YCbCr jpegli probably

JXL in vardct mode has kind of arbitrary precision, quantized coefficients are always integers (12-bit ones in case of recompressed jpegs, but the spec doesn't care about that range, it just has to be representable in int32_t) but the dequant factors can be anything (in case of recompressed jpegs they just happen to be integers) so the actual precision can be almost arbitrary.

Oh wow. And that’s just a principle of the VarDCT generally I’m assuming, is inherently arbitrary precision that allows essentially any bit depth you want

Or at least before quantization

So you could have as much information you want distributed over any bit depth you want for the same size images?

It's producing 8-bit jpegs (which has 12-bit coefficients) so existing decoders can decode them (12-bit jpeg with 16-bit coeffs is basically dead letter in the 1992 spec). It just does everything with float precision instead of doing it the libjpeg-turbo way which is converting 8-bit rgb to 8-bit YCbCr and then doing DCT on that also in integer arithmetic so with rounding errors, and all the rounding errors are repeated on decode so there are 4 places where things get quantized to integers, with precision loss that can be avoided.

So the floats are stored in the quant table & the decoder rounds them to 8 bit ints?

In jpegli the rounding to ints is only done at the one place where it is unavoidable: when deciding which quantized coefficients to write in the bitstream.

As far as I understand, the bitstream is required to have 8 bit ints in the quant tables for each channel. Why would libjpeg go through what sounds like extra work for a worse implementation

In the usual jpeg implementations like libjpeg-turbo, rounding to ints happens in several intermediate stages too, which makes errors accumulate. For 8-bit and typical qualities that is fine, but if you want to get the most out of the old jpeg then you have to avoid that.

So it is in the intermediate stages, that makes sense. I still don’t understand why they’d do it that way though, jpegli sounds more sensible where you’re only rounding when necessary

The libjpeg-turbo way is faster on hardware which doesn't have good floating point support...

in the changelog for the recent beta versions of libjpeg-turbo it says this: ```"The accurate DCT/IDCT algorithms are now the default for both compression and decompression, since the "fast" algorithms are considered to be a legacy feature. (The "fast" algorithms do not pass the ISO compliance tests, and those algorithms are not any faster than the accurate algorithms on modern x86 CPUs.)"``` does this affect anything or is it unrelated?

I think they have 3 versions of DCT/IDCT in libjpeg-turbo: - float - fast inaccurate int - more accurate int (default)

it’s the ability to simultaneously reproduce very dark and very bright tones

it stands for “high dynamic range”

dynamic range being the range of signal levels that can be represented

For videogames there's different HDR: light is calculated in high dynamic, and then game simulates eye accomodation/camera exposure in SDR. (Well, regular HDR is also there, so sometimes things get confusing with two "HDR" terms)

it sounds like HDR tone mapped to SDR

HDR can be present or absent at various stages: capture (or synthesis), processing, physical reproduction

the term “HDR photography” has long been associated with just capture HDR which is then processed for SDR output, but hopefully, this will change now that it’s becoming increasingly practical to keep photos HDR throughout

https://iopscience.iop.org/book/mono/978-0-7503-2558-5 describes it in terms of the following “stages”: scene DR, ADC DR, raw DR, image DR, display DR

Starting with a normal raw photo, can you export it as HDR after editing? Is there a benefit to this or is there a specific HDR workflow for real HDR images?

you can, and there may well be a benefit

many cameras have more dynamic range than SDR can accomodate, even within a single shot without bracketing

both https://spider-mar.io/hdr/Weesen-P3-HLG.png, https://spider-mar.io/hdr/salon.jxl (/.avif) and https://spider-mar.io/hdr/piscine.jxl (/.avif) were made in that way

(or wait, maybe not salon, let me check)

What's the workflow like? I'm very interested, esp considering I always export my photos as jxl

I’m actually not fully sure, it seems I shot two frames but only used one of them?

slightly convoluted for now (export as EXR, for example using darktable, then grade the EXR in DaVinci Resolve, export as still in TIFF, and compress the TIFF to JXL/AVIF/whatever), but it may become more convenient once it’s implemented in Lightroom

https://helpx.adobe.com/camera-raw/using/hdr-output.html > Note: High Dynamic Range Output is a Technology Preview feature in Camera Raw. This means that the feature is still under development and not yet final. We are actively working on several areas, including support for Windows, and Lightroom, user interface improvements, and better interoperability with Photoshop and other apps.

I'm fully in with Darktable, right now I've been exporting as 16-bit PNG & compressing to `-d 1.0` jxl

I suspect it may be some time before darktable fully supports a self-contained HDR workflow, though I wouldn’t put it behind them to at least try

I expect that they will tweak the filmic module so that it can work for that

the challenge will likely be to actually get the monitor to display HDR (especially given that it’s impossible on X11 and may take a while in Wayland)

from what I remember, the macOS version of darktable can’t display colors outside of sRGB, because the stack of libraries that they use won’t let them signal another content colorspace to the OS

so it’s not impossible that Windows might be the first OS where darktable can do that (if the software stack doesn’t impose the same limitation as it does on macOS…)

which I think is also the situation with Krita?

It will probably still take a while, but I think Red Hat's desktop team are now actively working on HDR in wayland. https://lwn.net/ml/fedora-devel/20230601183054.12057.45907@mailman01.iad2.fedoraproject.org/

https://curia.europa.eu/jcms/upload/docs/application/pdf/2023-06/cp230110en.pdf

> According to Advocate General Medina European Harmonised Technical Standards must be freely available without charge because of their particular legal nature as acts that form part of EU law

Note that while the above is not legally binding, it is legal advice to the Court of Justice which is currently handling a case about technical standards and the law

while it is unlikely to to affect JPEG, if it goes the right way it is possible that quite a few ISO standards would be forced to be freely available.

That would be great

Lonnie: Adobe, Apryse and Foxit sponsored free publication of pdf ISO standard: https://pdfa.org/sponsored-standards/ So if JPEG XL finds similar-sized sponsors, it will be possible

https://github.com/rainerzufalldererste/hypersonic-rANS

Is he known as a user of this discord?

I invited him to come here and have a chat 🙂 https://github.com/rainerzufalldererste/hypersonic-rANS/issues/3

I don't think Discord should be called a forum

In the latin sense of the word, it is... But I'm not a native speaker and it was the first word that came to my fingers 😅

I love the title of the issue

https://github.com/libjpeg-turbo/libjpeg-turbo/releases/tag/3.0.0

fairly new and highest quality service for vectorization that I have seen so far, probably something similar would be useful for jxl splines, although they are much more limited for complex images https://vectorizer.ai/

Good. Works nicely

Tried using svg2jxl on the output but unfortunately the 800KB tree I got out was too much for jxl_from_tree to handle, only output a file once I truncated it down to 30KB or so

did it produce an error? I wonder why it cannot handle large amounts of splines on the encode side

It could be a problem with URL lenght

Tho now I see an optional output parameter so it can really convert to JXL on the device? Edit: Ah, it prints what can be pasted into the jxl art website. Nice to know.

No error, just silently failed with no output. svg2jxl exports the tree as a txt file (used the jxl extension, but it was just raw text inside)

Try a debug build, release builds are kind of silent...

Here's the text file with the tree inside. Not sure where to get a debug build, and looking at the code for the jxl_from_tree, it looked like there wasn't much error handling and no verbosity anyway

Turns out it wasn't silently erroring, it was completely crashing

what the f

Yea, I was a bit surprised too

Good thing I switched to codeberg (not affiliated with me) for my newer projects ( :

NO MORE fair use 1984

I try to like codeberg but the interface I really dont like, I wouldnt mind hosting my own gitlab but thats a little much

They use a gitea fork called forgejo (<https://blog.codeberg.org/codeberg-launches-forgejo.html>)

I wonder if there is an option to change the interface a bit (I haven't found one so far besides themes, language setting and hiding a few things). If such an option doesn't exist, an issue could be created: https://codeberg.org/Codeberg/Community/issues

Tho using a browser addon to apply some custom css is also an option

GitHub is social media

if I just wanted git I'd use nothing

That real?

Unfortunately so: https://github.blog/changelog/2023-06-07-code-search-now-requires-login/

In a few years they might forbid repo cloning without a GitHub account.

It would then kill itself in the process 😂

it's a massive shame since gitlab's search is aids, gitea doesnt really have one, and not to mention discoverability on github is excelent, a shame to see it die, oh well, time to make gitlab suck less I guess

i don't think it's dying any time soon

i imagine gitlab and alternatives would die before

i dunno, then been doing a lot of stupid stuff lately, if gitlab had better discoverability I would swap to it

that's what i mean by github is social media

gitlab simply won't

so it will stay sucky

honestly, I don't see the point of gitlab

its self hostable which is really nice

you dont need gitlab for git

its also a lot better for medium scale projects

so organizations would rather just have its own git instance

and for medium to large, gerrit also exists

I cant stand gerrit

Bliss wants to migrate to gerrit and I really dont want them to xD

it solves a ton of problems git has

I prefer good old cgit over gerrit

but gitlab solves a lot of problems, for one it actually has a good issue tracker

having a close tie to issue tracking and code PRs are such an underrated feature

prs are a mistake

it just scales really well for medium projects, something like the linux kernel would be stretching it tho

Having issue and pull/merge requests is a really nice workflow for most projects.

Git forges are basically industry standard by now, both in open source and closed source

I'm surprised it took them that long to block it off. Search functionality is one of the more memory & compute-intensive tasks for a website - especially when there are billions or trillions of entries to search through. Allowing public bots, scrapers, and non-logged in users to run searches tends to increase costs dramatically.

this is a single repository search.

Really no different - a single repository can have millions or tens of millions of records: files, commits, pull requests, issues, wiki pages, versions of any of these, etc.

They should enable search on forks and branches. People now fork locally then upload to new repository to circumvent this lame limitation.

search is VERY expensive

Really different - searching through a single repository has never been a computational problem. The reason why it was suddenly banned is complete bullshit. There are no excuses. Now, to search by code, I have to download the entire repository in zip and do a code search locally on my PC.

It sounds like there's no "single repository" search now. There's "global search, filter out repositories with names different than this"

search through a single respository is very expensive

I've worked in web app development for decades and I can tell you that it is expensive. Not as expensive as something like AI - though some search engines are now implementing AI parts. In fact, Github has a blog where many posts talk about their ongoing changes and improvements to their search infrastructure. Just because you become frustrated by a service making a change doesn't mean it was done with malicious intent. With every company out there reducing costs it was likely done as part of that change. I'd also appreciate it if you stopped acting as if you are an expert at everything while giving misinformation.

Of course I am an expert in everything, but who told you that? This is the first time I've seen people talk about me like that

Understand my argument. The developers of github did not redo the function, did not optimize it, they did not do research on how it would be possible to do code search in any other way, did not create new technologies. They just banned it, removed the function for ordinary people. What kind of work can we talk about here? A completely disgusting and incompetent decision. And of course I will strike it. Because for decades there has not been a hint that this Suddenly causes problems. And then at once they took and removed the function completely. Do you have any idea how many possible solutions there were besides that? I would even be willing to solve the captcha with every code search. But github's decision is unworthy of my respect.

> development for decades > is expensive This is your personal experience. Will there be real arguments?

just log in bro

fuck them.

?

it's clearly designed to mitigate bots and scrapers

To degrade the use of the platform to increase the number of registrations is an abomination

that's obviously not the goal

Soon it will become a universal excuse

don't slippery slope it

one day I went to github to search for repositories there by name. And she told me that you can't look for ANYTHING here without registration at all. Not a single word. - a garbage dump and not a platform

by the way 👆 gitlab now

They're just gotten bigger than they can reliably maintain. This is the first sign of the end.

how do I download all existing github repositories? where is the download button?

perhaps if github himself had made some 10,000 terabytes of torrent with all repositories at once, then there would have been no bots looking for something on the site. m?

I doubt that they'll permanently delete old repositories like gitlab did, but it's possible that they'll put them all in a cold storage, and you'll have to pay to retrieve it.

GItlab reversed their decision on that.

obvious why

this will never happen

Free Github Actions likely will be first to go.

Github Releases after that.

🥱

you guys are all gloom and doom about them trying to restrict their infrastructure to actual users rather than bots

Since actions can potentially use a lot of cycles depending on what they do, it could be scaled back or removed at some point if finances get worse. I highly doubt releases will get removed. That's the main way many know when new versions are available. There would likely be outrage and it doesn't look like it takes much to make a release; Mostly just zip the files from a given tag, run any associated actions, and produce a page with whatever info/changelog the manager sees fit.

I don't think people are doom and gloom. Taking away existing features that were, until recently, seen as a huge no-go and potentially a business killer. Now it's become common so people have trouble discerning how many changes would be made, largely as a result of economic shifts.

people are entitled

same thing with the whole YouTube ads/premium thing

~~for those who don't use adblock, anyway~~

~~or you can just live in a country where Google has disabled all advertising at all~~

I'm waiting for YouTube to fully block ad blockers so I have an excuse to get premium and also get the higher quality video

if youtube makes adblock stop working I'll just stop using youtube

it's completely unusable with ads and I'm poor

YouTube should have been nationalized.

You can't change my mind on this.

~~inb4 no politics~~

you mean a public service?

funded by taxes

ye, paid for by taxes

ye

that's an option, can't disagree

give it NASA budget

NASA's budget is 58B$, which is 350$ per year per taxpayer.

i guess a problem with that is the same as any other nationalized service being lack of competition

Oh yeah? And what does YouTube compete with?

uhh

tiktok

It's not the same kind of content.

But yeah, they do compete.

But the fact that they provide and *should* provide different services (hot take: YouTube shouldn't market shorts the way tiktok does) means that no innovation is happening in Video Content Creation and Streaming Service, only Shitpost Content Creation and Streaming.

And even then, all they do is copy each other.

NASA has a budget for RnD.

So a national content platform could, too.

They could pay all of their 100k subs content creators with half the budget.

But I guess that's where this falls apart.

Without a flexible income, it qould be harder to pay creators.

I guess that's what sponsors are doing already

Also maybe <#806898911091753051>

100k subs can make more than or up to 1800$/week, which times 306k (nunber of 100k subs creators) that's 28B$/year.

o

I remember back in the 2010 timeframe, YouTube used to tell it's creators that unique, longer-form (5-20 minutes), high quality did the best. Now with shorts its to produce as much low-quality content that they can. Granted, I also remember some asking if running more ads would get them more revenue. Their response was so little, it wouldn't be worth it. Now YT and other platforms have doubled and quadrupled the number of ads all while the creators see hardly any increase in revenue.

I don't think it is an issue for creators. That's one of the main reasons people move off of twitch and tiktok to youtube.

I'm pretty sure it's just the platform that isn't that profitable, hence the push to different model for monetization

ad supported platforms just aren't a thing anymore/weren't ever a thing

see twitter

it's probably more sustainable if they cut bandwidth (something like maxing out at 480p)

they are already trying that with the least friction (4k/higher bitrate for premium)

there's the tie in to make it somewhat on topic

Continues here :3 https://discord.com/channels/794206087879852103/806898911091753051/1127333609930502324

no

>:3 wot

Probably not with GitHub search, as the have an actually decent API, which includes search.

what's the current meta for converting XYB JPEG to RGB PNG

`ffmpeg -i xyb.jpg -vf libplacebo=color_primaries=bt709:color_trc=iec61966-2-1:format=gbrp rgb.png`

Thank you

(`color_trc=13` may be easier to remember)

When I decode with djxl into png, some software can't properly display colors and I get darker pictures like here. Original on the left, decoded on the right. Any way to fix it? I've tried different combinations of --color_space, but it's either the same or even worse. Currently I just decode to jpeg with -q 100 as temporal solution. https://i.imgur.com/30MfCCz.png

https://twitter.com/LukeGessler/status/1679211291292889100 lolol

would you have the original at hand? it sounds sort of related to https://github.com/libjxl/libjxl/issues/2289 but that should have been fixed

Seems I figured out a bit. This particular color problem is related to my default windows photo viewer, it distorts colors a lot, while other software like photoshop or xnview displays it correctly. I actually was dealing more with another situation. If we take a color like R50 G100 B50 and save as q100 jpeg, it becomes 50/100/49. If we losslessly transcode it into jxl and decode as png, it's going to turn it back into 50/100/50. Because I was always decoding into png without looking if it's transcoded jpeg or not, it was giving me ssimulacra score around 93 even when I was expecting lossless. The score is 100 if I decode it into jpeg.

I am getting `[libplacebo @ 0x562d041178c0] Failed opening ICC profile... ignoring` every time

git master libplacebo?

no, just using from the arch repos

should I build from source

you would have to, yea

alright trying that

now FFmpeg build fails with `undefined reference to 'pl_plane_data_from_comps'`

cloning from `git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg`

ye, although atm I think there's a bug in placebo's transfering

so I'd hold off for now

they recently did some work on git master that mighta messed with it

Ah :(

https://av1-community-contributors.github.io/av1-wiki.github.io/docs/images/jxl Hello, if anyone want so help us out & make an entry for JPEG-XL on the Codec Wiki, feel free. I'm gonna make an attempt otherwise, but I figure many of you are more well versed that I am

<https://av1-community-contributors.github.io/av1-wiki.github.io/docs/FAQ#why-not-use-mediawiki> > Not modern enough, the goal is to attract zoomers with fancy animations and eye candy.

🤮

uhh .... that's not my writing

this used to be a very different project, ignore that stuff 😭

I wouldn't want to participate in something like that

I guess they just made another guide.encode.moe

we're still in the process of cleaning it up. again, started as a lighthearted guide, became an actual wiki

the jpeg xl page is here : https://av1-community-contributors.github.io/av1-wiki.github.io/docs/images/jxl it doesn't cost much to put here a compilation of either the wikipedia article or the jpegxl.info website

ykw I'll just do it myself. I have a couple more entries to add before doing JXL. Also, the URL will be changed

Is it a real wiki, or git PR red tape nightmare?

.\cjpegli.exe --xyb ..\freddy.png ..\freddy2.jpg. The colors get weird and the image gets stretched. Is this a known issue?

Using the 64-bit build from 17th July from https://artifacts.lucaversari.it/libjxl/libjxl/latest/ btw

Yes I also had this issue

Seems to be a similar effect but only in dark areas for me

: /

Ohh of course, the preview on discord makes it green because of XYB, didn't realise it was almost black and white

open an issue on the tracker :)

it also has really weird moire effect when zooming in and out

at least on MacOS

is that `.jpg` file a JPEG or JPEG XL file?

jpegli

well, jpegli produces JPEG files

it's just an alternative encoder for the same format

i spent a day with an OLED HDR monitor (acer x27u) and can confidently say that HDR sucks and OLED sucks

yes

you can't calibrate hdr so using it for desktop will always suck and oled has worse viewing angles than IPS. response times only matter to gamers and deeper blacks are only noticable if you dont have lights

all that for $1000

doesn't suck on macOS

It's a jpeg. Turns out the png has an alpha channel which messes with jpegli when using xyb

oh yeah also the oled dimming thing is extremely annoying

$1000 sounds p cheap for OLED

>gamer monitor

yeah don't expect too much visual quality if it's marketed for gamers, these mostly optimize for response times

there are only 4 oled monitors on the market

they all cost the same

and they all have the same panel

I was able to calibrate it in sdr at a low brightness and the results are very good, but not impressive

but that's about it

if you've got a monitor in mind i'm up for buying and returning again

i've become one of those people

My phone's got an early OLED screen, over 6 years old now and still no burn in whatsoever, even with the android UI, frequent use of the same apps, ect

was thinking more in lines of LG 32EP950-B (and possibly PA32DCM in the future). idk availability wise but seems less refresh rate / response focused and more into color quality

I'm very happy with my phone's OLED, but the monitor oled (currently, LG WOLED) is completely different

it's not a year of OLED desktop yet

wait long enough and microled will become a thing

i guess my issue with oled is lg woled

but my issues with hdr (on windows, wayland still dead) still stand

ahh wayland, maybe we will finally have HDR in 2060

You can calibrate HDR. You just need capable equipment to do so. High contrast is very important. Even in my lit room, the difference between a VA panel and an IPS panel is noticeable, let alone an OLED. Response times doesn't only matter to gamers. Scrolling on a low response time display is sublime. Worse viewing angles is false: they either have as good or even better viewing angles. Besides, even an off-axis capable OLED display has better viewing characteristics than an IPS display 😂

how do you calibrate HDR

what software supports HDR profiling

how is it better if it I look at it and it's worse

higher contrast is noticeable, but you can't just change the tone curve to whatever you like. it's wrong

what do you mean by calibrating for HDR?

profiling the display

as in measure the capabilities of the display so software can use it to render things correctly

afaict, nothing can do it

theoretically, I should be able to have a 100 nit sRGB green next to a 100 nit P3 green next to a 1000 nit P3 green

but that's not possible

? calman can handle this fine

so does anyone want to buy me calman

you want to calibrate for `HDR 2084 P3/p3-st-2084` probably which is PQ in dci-p3

argyll cms and displaycal also have the option, but it's unusable outside of a madvr 3dlut tonemap in SDR mode

oh right calman just updates the tonemapping on the TV

oh yeah OLED also drifts like crazy

If I need calibratedish HDR I use my iPad 🙃

if I need calibrated HDR I just cry because I cant afford anything good and sold all of my semi-decent stuff lol

😔

gonna be waiting a long time friend lol

😭

i realized hdr effect is attainable by just tone mapping hdr signal to the correct value in sdr mode as long as the white level is at peak brightness

thats not a great time

I mean, it's fine if the HDR content is that much better than the SDR version of whatever you're consuming. but that's generally not the case

that being said, oooh boy are there exceptions

yeah hdr content is rarely better than the sdr version

and the wider gamut is ~~rarely~~ not even used

it's not even that, just, tonemapping SDR->HDR is significantly easier then doing HDR->SDR

I would argue that in a lot of cases, for legitimately good HDR->SDR it needs to be done on per content basis

i dont mean tonemapping hdr to sdr

i mean tonemapping hdr to a display in sdr mode

is that not the same thing?

tonemapping to the display by profiling

so like, the display produces this color at this amount of light when you send it this rgb value

sounds like normal ICC based tonemapping

yeah I just mean HDR is achieved like that through software

I guess i really didnt say anything if the only way to do that is with the madvr 3dlut anyway

well IIRC MPV can use ICC profiles for tonemapping, but you could also maybe do it with OCIO

can you do hdr with icc clut?

no idea, not something I ever cared to look into

there's probably a reason it has to use eecolor 3dlut

whatever i already returned the monitor for my $1600 back

I'm pretty sure it can, although that's more of a question for haasn

When I mask image with alpha channel and delete all unused pixels, sometimes I get smaller size after blurring this mask with 4x4-16x16 kernel (0 or 1, just expansion). It's quite interesting, because I store much more information and still get smaller size, so I suppose that masking some individual pixels just makes it harder to compress due to complex shapes. Would I get even smaller size if I mask based on image blocks, like ceiling odd indexes or using some kind of threshold, for example, if 256x256 contains more than some number of pixel values, then use whole block instead? If not, then look at smaller block and repeat.

> Intel Advanced Vector Extensions 10 (AVX10) is a new ISA that includes "all the richness" of AVX-512 and additional features/capabilities while being able to work for both P and E cores. Intel says AVX10 will be "the vector ISA of choice" moving into the future. > Part of making AVX10 suitable for both P and E cores is that the converged version has a maximum vector length of 256-bits and found with the E cores while P cores will have optional 512-bit vector use. > > Existing AVX2-based software could be able to be recompiled for Intel AVX10 in the future and that alone is enough to deliver performance gains. Intel will also be rolling out AVX10-optimized software moving forward.

> Part of making AVX10 suitable for both P and E cores is that the converged version has a maximum vector length of 256-bits and found with the E cores while P cores will have optional 512-bit vector use. would still make moving processes between cores annoying

Was trying this today & getting an error trying to encode .tif files exported from Darktable with cjxl. Does libjxl support TIFF input?

Hmmm. Wondering how to properly deal with HDR JXL

As in exporting an EXR from a Darktable HDR DNG & converting to JXL somehow while keeping HDR

this may be helpful: https://github.com/libjxl/libjxl/blob/main/tools/hdr/exr_to_pq.cc

if you build libjxl with devtools enabled, this will be built and found in `tools/`

(the `hdr` subdirectory is only in the source tree, it is not reproduced in the build directory)

my first HDR attempt :P don't have an HDR monitor so kinda shots in the dark

``` [19:23:27] Finished work at: Sat Jul 29 19:23:27 2023 M[u1x9i:n2g3:: 2t7h]i sl imbahyb :t awkoer ka wrheisluel.t. .= 0 Encode done! ``` I think HandBrakeCLI is trying to tell me something

Alien 👽 language

It encoded the output log 🙂

I think I can discern “Muxing” at the beginning

```Muxing: this may take awhile... [19:23:27] libhb: work result = 0```

the characters were just alternating in that line 🤔

How could that happen? Writing to stdout in two threads and somehow flushing at every character instead of at line breaks?

maybe it was stderr? iirc that's typically flushed immediately

You know what AVX-512, the instruction set that has never seen a complete implementation of needs? More fragmentation!

so i converted this picture with: -j 0 -d 2 -e 9, with 0.8.2, and the jxl one looks way more colourful, (imo it looks better) but it really shouldn't differ

it appears the original JPEG has a weird ICC profile

or rather

not weird but it's Adobe RGB 1998

which is not sRGB

cjxl with -j 0 is using an XYB transform and when you view the resulting JXL file you're viewing it in sRGB, clipped

keep in mind discord ignores color profiles

so whatever software you're using to view the JPEG is not properly color mananging

so anything with a wider gamut than srgb will appear incorrectly washed out in discord

yea, thats the problem

if i open this image in gimp, i can either convert to srgb or keep the embedded color profile, and either way it looks more saturated than the discord preview

and it looks identical to the jxl

well the problem is with qimgv in this case

interestingly, cjxl tags it with "unknown" primaries

so 2 bugs found with 1 image?

hm?

well, cjxl not knowing it, and qimgv not showing properly

what's the copyright on the image

no idea, downloaded it like a decade ago

https://commons.wikimedia.org/wiki/File:Defense.gov_News_Photo_080804-A-8725H-341.jpg

looks like it's US Gov't, which makes it public domain in the US

I don't like uploading copyrighted images to the issue tracker

i see

also just wondering is this a photo from some kind of war or something?

US War in Iraq

the file has a bunch of geo metadata, it was taken in Baghdad

AVX10 is the opposite of fragmentation on Intel SKUs.

I have an image in PNG that used to be in JPEG and has the corresponding artifacts. Is there a way, or a __*theoretical proof of the possibility of this*__, that would make it possible to make Jpeg lossless compression relative to an image already compressed once (that is, to make an identical structure of information)

I have reason to believe that this is possible due to the fact that in theory it is possible to iterate over all possible variants of DHT 8x8 until the variant with completely identical pixels is found. But I have doubts about the possibility due to the subsequent construction of a Huffman tree that cannot exactly be identical.

Anything is possible, so long as you have the time and resources ;P

Although if you're iterating over everything, at that point you might as well try to recrate the original image completely

I'm not going to do it, I'm just curious. Maybe in this chat, maybe even knows about whether this question was asked before me...

The dct coeffs you could in principle recreate from the pixels (not sure if the solution is unique). How they are entropy coded in a jpeg bitstream is not something you can know at all from pixels.

chroma subsampling might also make things trickier

not obvious to me how you figure out the quantization matrix in the general case, either

make a histogram of dequantized coeffs, look for peaks aligning with multiples

in general, the first thing I thought - why not create all possible variants of encoded dct 8x8 images ... until I realized that this is a mathematically impossible task due to the large number of options......... ...is to compress each 8x8 block separately on all quality parameters from q0 to q100 and choose the one that suits the most according to the corresponding pixels, and so separate for each block....it's just a pity that it can't work at all, because there is such a thing as re-compression degradation... in fact, of course, a big thing like reverse decoding: To be more precise, it's all similar to how the creation of sound and music works only according to the graphical image of the spectrogram What is used to clone the voice or in some projects to increase the duration of the sound while preserving the tone (which is a violation of physical laws, but the neural network can make it) It seems to me that only a neural network that will be trained on what data was in the original Jpeg, and what data was obtained in the image, will be able to recreate the image data approximately but as similar as possible...

I know after seeing this <https://vectorizer.ai/> I thought about NN spline encoding in JXL, but at that point you might as well make a full encoder, which JPEG is already working on

JPEG AI is a full AI-based codec, also the decoder uses AI. Doing an AI encoder into a non-AI codec is interesting but not something JPEG is doing — in general they don't really do encoders at all since what gets standardized is always only the decoder.

Ah right, and good point

Had some of his videos posted here before about image formats, nice to see JXL mentioned multiple times in the comments too https://youtu.be/68H1H6Z_gz4

Obviously I couldn't resist posting comments on a video like this.

Hi guys, I'd like to check the spec of JPEG XL, is buying this pdf from ISO/IEC 18181(https://www.iso.org/standard/77977.html) a must?

the theoretical answer is yes

in practice, some people might sometimes by mistake put a copy somewhere

Also I wouldn't buy the first edition, the second edition is more accurate

Even a draft of the 2nd edition is better than the published version of the 1st edition

we've been ironing out a lot of spec issues

see <#1021189485960114198> for an up to date draft

What would you recommend as metric for difference in pixelart? What I want is to assign weights to changed pixels and find more important to pixelart.

As quality metric or some kind of filesize influence? Last time I'm pondering about similar problem. Overall, I like ssimulacra 2.1, but it has some small problems for images with flat regions. Because such regions always have the same value, even after using extremely low quality settings for encoders like jxl q=40 or avif cq-level=40, we still get 90-100% score, because these values usually do not change. It means that quality scores for images with flat regions and without it are slightly different and can't be directly compared. Here is artificially created situation to demonstrate it. I take the same image and simply add regions of different sizes with a single black color value to see how it influences overall ssimu score. https://i.imgur.com/e0JnsWw.png Original encoding doesn't change and you can see how original 63 quality score becomes 79 once 93% of it are flat regions. Such situation is extremely unlikely to happen in real life, but when we talk about drawing, pixelart and similar images, it's actually quite common. Look at simple image like this one: https://www.boredpanda.com/blog/wp-content/uploads/2021/11/Hey-Pandas-Post-your-best-or-any-pixel-art-that-u-have-created-right-here-618b7194c5378__700.jpg It mostly consist of repeating colors. And even if we talk about significantly smaller areas like 1/3 of image, we can see some influence ~2 ssimu score. One of possible solutions would be to segment our image and see how much ssimulacra score varies and if it varies at all. If we count such segmented score, then it gives me original 63 quality for the segment with original image and 100 score for the rest. We can judge overall quality of the whole picture and the worst/best parts of it.

Another problem is that small color variation can include a lot of details that are lost after quantization. And this problem is trickier, because we need to judge if such quantization is perceivable or not and if it influences visual quality. For example, if we take white noise and change some values, visual quality doesn't change much, because it's still white noise. But if we take pixelart, like a tree with leaves that barely differ from remain colors, and after quantization this color difference disappers, we have a tree without leaves and huge visual lose. I'm not sure if such metric is even possible without AI. Currently I'm thinking about comparing the change, where variety to another variety conversion is punished less than variety to completely flat area. But such approach just makes sure that details remain, rather than what details are important and what aren't.

If you use a tool like ssimulacra2_rs, you will get additional data on what the scores are throughout the image: https://github.com/rust-av/ssimulacra2_bin https://github.com/rust-av/ssimulacra2

It's been a while since I've used mainline SSIMU2 and I'm on a very new clean system, so haven't installed it yet to check if it has scores other than the average.é

ssimulacra2 uses a mix of 1-norm and 4-norm. To be fully robust to adding 'easy flat background', you need inf-norm (max error). That does basically mean that the single worst region fully determines the score, which is good to have guarantees but not so good for correlation with subjective scores.

For comparison, the usual ssim (or ms-ssim) uses just the 1-norm (so just the average value of the heat map) so easy backgrounds bump up the score a lot. Ssimu2 is better in that respect, but not as good as something with a maxnorm like butteraugli-max

woah a gost

the best and fastest way to transcode 50 thousand jpeg to jxl images is to write a python/bash script? cjxl, like all similar programs, accepts only single files for input?

Yeah, cjxl can only do a single file at a time. I've attached my program to do it. It should work so long as you have python and cjxl on your PATH. By default it removes the original for successfully transcoded files. ## Usage ``` usage: jxl-convert [-h] [-d DISTANCE] [-r] [-q | -v] [target_directory] Bulk convert images to JPEG XL. positional arguments: target_directory Directory to convert. If omitted, defaults to the current directory. options: -h, --help show this help message and exit -d DISTANCE, --distance DISTANCE Max. butteraugli distance, where lower means higher quality. 0 is mathematically lossless (default), 1.0 is visually lossless. The recommended range is 0.5 to 3.0. -r, --recursive Recurse into subdirectories. -q, --quiet Suppress progress output. -v, --verbose Explain what is being done. ```

`for i in *.jpg; do cjxl $i $i.jxl; done`

``` parallel cjxl '{}' '{.}'.jxl ::: *.jpg ```

Optionally you can use https://pypi.org/project/imagecodecs/ . It partially supports jxl encoding/decoding. But probably won't be faster than subprocess or bash.

I would like to use specific executable cjxl files of the latest versions, so this not for me

<@604964375924834314><@794205442175402004>you have written very small scripts, apparently they are for linux. I will need to figure out the terminal language that is in CMD and do something similar. day after tomorrow

<:Windows:806135372298977342>

this code is too big.

<https://github.com/sharkdp/fd> `fd -e jpg -j 4 -x cjxl "{}" "{.}.jxl" --num_threads 0` `-j, --threads <num> Set number of threads to use for searching & executing (default: number of available CPU cores)`

`parallel` is a perl script https://www.gnu.org/software/parallel/

(although there are other implementations of the same interface)

(for example https://gitlab.redox-os.org/redox-os/parallel)

I personally use the old parallel from https://github.com/mmstick/parallel since the new parallel relies on ionshell so it;s incompatible with windows

<@184373105588699137> <@604964375924834314> Parallelization does not interest me at all. It's not difficult for me to move all my files to 4 folders and run 4 terminals on my 4 cores

Here is a demonstration of my current approach I'm working on. For jpeg images I count the difference between minimum and maximum values in pixel blocks of 8x8 size and count what % of a whole image contains values 0, <=3, <=5, <=8, <=30. The same for encoded images. All below examples are animated, if it doesn't work in discord, try to open original links. Here is a pixel art with % values 4, 4, 4, 5, 10. 90% of blocks for such image have a high color variety and it typically means that even significantly lower ssimulacra 2.1 scores would be visually appealing. There are some color changes, but details remain and this example is a difference between original and jxl q=82 with ssimulacra score 70.94 and % values 1, 3, 4, 5, 10. Such drop of 0 color difference for jxl/avif blocks is common, because it uses significantly bigger blocks and 8x8 segments can easily get slight 1-2 values variety. https://i.imgur.com/I6RJmlP.png Here is more extreme example with a dithered cat image. Original % values are 0, 0, 0, 0, 0. All blocks have differences above 30 RGB values. It's compared with jxl q=60 that produces 29.7 ssimu score and the same % values. Despite we have significant color changes, the overall shape remains the same and many people would still find it appealing enough. https://i.imgur.com/tVJBFTw.png

The opposite situation can be something like skin tone. This image has % values 1, 5, 11, 25, 80. Almost all blocks are within 30 pixel color difference, and 25% are within 8. It's compared with jxl q=53 that produces 70.9 ssimu score and jxl q=78 that produces 80 ssimu score. Both have significant block changes at 1, 17, 31, 48, 82 and 1, 10, 22, 41, 81. Such soft images typically require significantly higher ssimulacra score around 86-88 to retain all details, otherwise it's going to be flattened after quantization and it's easily perceivable. https://i.imgur.com/SMqfzXc.png Such approach isn't perfect, but it shows some potential to be able to judge if picture would be good at 70-80 or it would be awful and we need to push it up to 88. There is a huge filesize difference between the two. The higher threshold can be used to split soft/varied images and lower thresholds can be used to see how many blocks are flattened after quantization.

Fundamentally it's probably an idea to make ssim map output depend on image frequencies.

I found some strange nonamesite, but it has ready-made ffmpeg builds with extremely exotic codecs like svt-vp9, I think I worth sharing it https://jeremylee.sh/bins/

<@132637059327328256> oh its you

<:KekDog:884736660376535040>

as a result, my hands never got to that

I chose this solution. it works well for me, thanks for it

I forgot to mention that if you parallelise like that, you’ll have to balance the size of the various folders to really benefit

vs. something less ad hoc where the various threads pull from the same queue so that they don’t become idle while another thread still has several images to process

I just wrote small python script that traverses files in the tree, and can call arbitrary number of cjxl (also takes a limiter from args, to avoid ram overflow, because my cpu has 16 physical cores). So it uses 100% of CPU until number of still-not-compressed files is lower than core count

This is true.... But how could you forget that the jxl transcoding itself is parallelized. While working, I realized that I didn't need any parallelization <:JXL:805850130203934781>

For such work, a python script is too much. Like https://discord.com/channels/794206087879852103/794206087879852106/1140712391089332457

I was asking for purpose of comparing canvas on events like r/place with template and assigning damaged pixels weights or priority.

Buggy discord