JPEG XL

but rename to png and it gets processed, lol

but all actual image applications I work with support it now...

there's a lot of people who use outdated versions of stuff like Photoshop (mostly pirates) before native WebP support was added however this plugin exists and has existed for years https://developers.google.com/speed/webp/docs/webpshop but no if something doesn't work out of the box with 0% effort then the user is left slamming their head against the keyboard and blaming everything around them

people don't think stuff like "how can I get Photoshop to open this?" they think stuff like "Photoshop won't open this grrrrrrr"

apologizes I'm very tired at the moment

I'm gonna start a thread about this, but I'm experiencing some weirdness when encoding from EXR with cjxl that I don't experience with PNG

I made encoding vardct about 35 % slower at one phase -- to make it more robust for exotic images

exotic meaning like pixelized checker patterns and the like

I only write the best C++ code ```c++ for (int i = 0; (lol>>s, i++); ) cparams.delta_cols.back()[i] = stoi(s); ```

error handling, who needs it?

I don't think that makes the top 10 of the sketchiest code I have seen

😛

(also, you don't need the `i`, you can just `cparams.delta_cols.back().push_back(stoi(s));`

it's an std::array, not an std::vector

for manual palette colors, I added two `std::vector<std::array<int32_t, 3>>` to cparams

it'll look like this: ``` Palette[ 123,23,1 50,50,50 0,0,0 ] DeltaPalette[ Weighted -1,-1,2 1,1,1 -5,5,0 ] ```

you can have any number of triples and it'll just make your palette bigger

strtol based

https://ericportis.com/posts/2024/okay-color-spaces/

(very nice blogpost by my teammate <@702298678333014087>)

nice article. The fact that it's impossible to make a color space that does what we want (because color perception is non-euclidean) makes me wonder if for some applications, whether it'd be better to have rhe ability to just ad-hoc generate even gradients between any two given colors, rather than relying on a color space to do it for us. Since we have the ability to evaluate new color spaces without doing large trials, I assume the math of perception is well understood.

Parallelizing the encode process for multi-core systems.

I believe it is related, yes

What is good for gradients is also good for many other image manipulation operations, and image compression. For some image manipulation operations it makes sense to do things according to perception, for others it makes sense to do it according to the physics of light.

(but "perception" depends on several things, e.g. what is perceptually uniform depends on the visual angle: if you compare large patches of color, S cones play a bigger role than if you are comparing tiny, pixel-sized patches of color)

<https://github.com/libjxl/libjxl/commit/87ec5d0cdf0319f6694f7c957cc933fbfe43df08> <https://github.com/libjxl/libjxl/commit/744575425705ca2ad276c6f377acc12493a9e8f9>

is there a "default" web cmyk icc profile? like sRGB for rgb

Not really, no AFAIK. When I needed to answer that question, I decided on `U.S. Web Coated (SWOP) v2` as this was default in Photoshop for ages at the time (more than a decade ago).

yep, that one is also the one we (at cloudinary) assume to be the default for untagged CMYK images

but it's not defined by W3C and most browsers render CMYK images incorrectly even if they are tagged — they'll typically show the CMY interpreted as RGB in sRGB, and just ignore the K channel

(this kind of works since CMYK is stored with 0=full ink, so misinterpreting CMY as RGB is not a _horrible_ misinterpretation)

now I have to figure out how to make that with lcms2 programatically...

actually I'm just confused on how jpeg ycck and cmyk should be converted

libjpeg cmyk output is different for each

and I can't seem to get a normal RGB output with us swop to srgb

It's funky - with lcms2 you may have to explicitly select chocolate instead of vanilla

there's like nothing on ycck, whats up with that

inverted cmyk... looks like it's the adobe marker

now I also need to figure out how to create non inverted cmyk jpegs

i can see why nobody wants to implement cmyk

ycck is just ycbcr applied on the first 3 channels

but your jpeg decoder should already be able to undo do that

cmyk is always inverted in jpeg, that's how adobe defines cmyk, with 0=full ink, 1=no ink (which is `TYPE_CMYK_8_REV` in lcms2). At least that's how I remember it, I might be confusing things

adobe stuff tends to be very confusing, in the Photoshop format you'll find places where they use 0=full ink, 255=no ink, other places where it's 0=no ink, 100=full ink, yet others where 0=no ink, 255=full ink, etc.

same with alpha channels in PSD where sometimes 0 is opaque and sometimes 0 is transparent, and then it's sometimes premultiplied and sometimes not

<:dead:707868275366952981>

thanks for the help

as if cmyk wasn't annoying enough to deal with, they can't even be consistent? T.T

> // F: Flavor 0=MinIsBlack(Chocolate) 1=MinIsWhite(Vanilla) I had no idea what you meant

Hi! I am trying to add JXL support to dlib (a C++ library). Everything works except calling: `JxlEncoderSetFrameLossless` I've also tried just adding this to: https://github.com/libjxl/libjxl/blob/main/examples/encode_oneshot.cc ```c++ JxlEncoderSetFrameDistance(frame_settings, 0.f); if (JXL_ENC_SUCCESS != JxlEncoderSetFrameLossless(frame_settings, JXL_TRUE)) { fprintf(stderr, "JxlEncoderSetFrameLossless failed\n"); return false; } ``` But I am still getting the same error. I am using libjxl 0.9.2. Am I missing something? I've read in the docs that setting distance to 0 is not enough to achieve true lossless. How should I do it?

I figured out what I was doing wrong: I needed to set `basic_info.uses_original_profile` to true.

Thanks for sharing! Far too often do I see once someone asks a question that "Hey, I solved my problem!", then proceeds to never share how they did so... Sorry I couldn't help you fix the issue, though 😅

experimenting with some stuff

ugh discord strips

open that jpeg in chrome

when I load it, it's completely blank, until I switch to another tab and back, then it shows up the same as the it does here on discord

mpv worked

chrome on the left, safari on the right

(the screenshot doesn't really look correct, I guess apple does tone mapping when making screenshots of hdr stuff)

mpv

which one should have been displayed ?

maybe both

I guess it's some kind of hacky gain map

it's a sloppily handwritten UltraHDR image

Yes, I also hate it when that happens!

looks like mpv just shows the gain map image, maybe it thinks it's a two-frame video and plays it like that

so it seems

Noticed recently that Discord is 'progressively' loading images by requesting a 512 width version before the full size when clicked

that's correct, if you treat it as a series of concatenated jpegs and start it paused, it renders the first one until you framestep forward

`mpv --pause --demuxer-lavf-o=format=mjpeg test.jpg`

After 3 month of working very much

Demo Dragon ball xenoverse

https://www.facebook.com/share/v/ynLgRk7Zso7oB35z/

Oh no..

Me and damian101 we have a lot of Instagram datas

Instagram isn't very used in Amerixa but I don't care about

Trameptora you know BlueSworx

I find his comment very interesting

Also inherent to+93 jpeg xl

daym

glad to see zig in such a random place

Danyil

I'm having Strong av1 artifacts fidelity problem

I don't know if is av1 problem or JPEG XL can fix it

😌🤨

I don't even use Instagram <:av1_thinkies:895863009820414004>

Seems like amplifird signal

Fab this stuff is incomprehensible, either stay in off-topic or start making sense please. Or just stay quiet.

Ok,ritt

You post random screenshots with random text and nobody understands what you mean with it. Either explain better or stop.

Thanks in advance. I hope you understand, I am trying to keep this discord sane and not too spammy.

lots of Zig lovers around the AV1 servers, including myself :D

Zig? Is that the new C for millennials?

it's C that want's to be rust

[av1_omegalul](https://cdn.discordapp.com/emojis/885026577618980904.webp?size=48&quality=lossless&name=av1_omegalul)

anything better than C/C++ deserves to exist

quite the hot take ... lol

that's precisely Zig's philosophy. Rust is *different* from C/C++, so both are still used; Zig aims to be largely the same, but all around better. Zig can be used as a build system in existing C projects too, and integrates perfectly with existing C projects

hot or not doesn't make it not true lol

it isn't true imo

my personal take is that Rust is more or less a replacement for C++ while Zig is a replacement for C

yeah that makes sense

not to say that C++ and thus Rust is too complex or C and thus Zig is too simple

they share a roughly same design philosophy in each group, but newer ones do that job in a much better way

Rust is a replacement for C

Proof: Linux is adopting Rust

well Linux allowed a C++ code base for device drivers for a long time AFAIK

the *current* usage of Rust in Linux is an alternative language for writing drivers

Linus Torvalds has violently rejected C++ every time is has been suggested

in the future Rust can indeed spread to other subsystems of Linux, but that would rather prove that Linux was too complex to be written in C alone

Personally I don't understand C++ at all, but find Rust nice and easy

there are instances where you need to write unsafe Rust. it becomes hard to justify

again, only proves that C++ was not good for Linux, not something *better* than C++ would be also not good for Linux

Not really?

another similar example is GCC, which of course started as a C-only project, but now it's a C++ project albeit with a heavy restriction

yes, absolutely

Wow had to use unsafe two times in a code base, whelp guess C++ would've been a better choice /s

are you suggesting that Rust is better than what others suggest, or worse than that?

https://zackoverflow.dev/writing/unsafe-rust-vs-zig/

not what im implying

Well so yes Rust is more complex than C, but honestly it's not just Linux I've seen way more C stuff and C people move to Rust than anything C++. Rust is definitely less complex than C++.

I personally disagree to that article

Been a while since I read it but as I recall I found it to be a bit nonsensical.

I should mention that I have written Rust for more than a decade, including the current job

people pretend Rust is the universal catch-all which isn't necessarily the reality

*some*

yes, some people

The number of people who complain about people who exaggerate Rust is greater than the number of people who exaggerate Rust, in my experience.

and I absolutely think that Rust's memory safety is extremely valuable, even with a presence of `unsafe`

I don't really see this myself, well when you pile a bunch of crates on maybe

not at all

unsafe rust is **significantly** safer then C still since you still have to interact with it

because it gives you a modularized tool for managing an entire class of bugs which was never widely available before Rust

and zig is significantly safer than C

even in general its still safer then C

but it's also true that Rust's memory safety was a bit exaggerated by people who don't sufficiently understand that (both Rust users and outsiders)

it is still early to quantify whether unsafe Rust turns out to be safer than (always-unsafe) C or not btw

Zig is safer than C mostly by making "the right thing to do" more ergonomic, rather than anything fundamental. And that's good! Good language design is good.

mainly because C now has accumulated by quite a bit of tools like sanitizer, and hardware supports like memory tagging extension may change the situation in a near future (not guaranteed though)

even if you have to write *entirely* unsafe rust, which you pretty much never need to do, I would bet the rust tooling to make it still more ergonomic and "safe" then C

in comparison, safe Rust does grant more freedom to the compiler, which means that unsafe Rust has to much strictly follow rules

100%. even just `defer` makes a huge difference for memory deallocation

but that's currently a speculation and not fully quantified nor optimized from Rust

Rust for example is looking for the formal notion of pointer provenance, which would essentially make any code not following that provenance rule an instant UB

but it is also developing new APIs to make it far easier to follow

so depending on the success of that API, the unsafe Rust in the future may be better than today, or worse than today, or just as (un)safe as today if that proposal gets scrapped

Zig is conceptually simpler than Rust in that regard, and it *does* provide a good spatial memory safety (which is often all you want), while a temporal memory safety is left to users

while pretty much everyone agrees that memory safety is a significant cause of security problems, spatial vs. temporal divide was not really quantified mainly because C had none

so if Zig succeeds, it would be one of the first ever method to quantify an importance of spatial memory safety over temporal one

until then, I would reserve my opinion about the safety story of Zig

> good spatial memory safety (which is often all you want), is it? IIRC most memory bugs are temporal memory safety

I think many are actually a combination of both

that's another reason why that is hard to quantify

(and it's not like you couldn't add bounds checks in C++ by instrumenting `span` or stl containers and thereabouts)

oh absolutely

if we can significantly improve the memory safety story of C without any new language, that's also welcomed

but it's evident that ISO C didn't try too hard on that regard

All code not following pointer provenance in both C and Rust is already UB. We just don't really know what the rules are, for Rust and for C, so we don't know what is UB.

Rust currently doesn't have a fully defined notion, which means there are some Rust code that is unknown whether it's indeed an UB or not

practically most cases can be sorted though

you could do a lot to make c and c++ safe, but in the end, the effort has been put into rust and it shows. rust is rocketing in popularity for a reason afterall

I think C and C++ failed to stop something like Rust or Zig because they thought backward compatibility as something mechanical

IMO it's something more organic

you can remain backward compatible with an apparent breakage, provided that you can make the supposed breakage a non-issue or much less painful (e.g. `go fix`)

there are a spectrum of technical solutions as well, like language epochs/editions

none of them was properly considered from both parties

JavaScript, aka ECMAScript, fared much better

How to make C 100% memory safe: compile C to a GC'd language and ensure that every allocation is a separate object. Bam, you get both spatial and temporal safety at the object level.

that doesn't solve Heartbleed 😉

No amount of language level protections can save you when devs do really silly things

Alas

~~me allowing users to run commands defined in a text file~~

you can still partly attribute Heartbleed to C because OpenSSL tried to use its own memory allocator to avoid C's wild memory management...

Well, they wanted to avoid "slow" memory allocators

They had like, one platform, with a slow malloc

So they made a caching layer

Imagine if they used Zig

"general purpose allocator is too slow, let's make a caching layer"

what's good with C is that it is easy to write shitty code, but it is also easy to write clean one 😄

only for a very small one

This is not the fault of C tho

this is the fault of bad programming practices

you can write shitty code in any language

Rust but it’s all in an Unsafe block

putting code in an unsafe block doesn't actually do anything

anyone know any easy way to detect if an existing webp file is encoded with lossy or lossless?

i.e. encoded with vp8 or jyrki encoding

`file` reports it for this specific webp but idk if that's actually universal

or if there's a dedicated tool for it

I believe `webpinfo` should tell you

ah so there is a tool for that

ty

im identify also shows quality as 100 for lossless, but doesn't detect for lossy

someone sent this to me and asked if they should download 0.10 and start over

what the hell

Wtf

yes, 0.10 is faster

What’s -e 0 do?

Also what do the extra 4 “-v” s do

make me more verbose, but lazy

see, I'm not even going to correct that typo, but I'm going to point it out

I am a bit verbose but at least truly to put effort in and be correct but might not be the most efficient

has anyone considered making a standalone jpegli repo that can be compiled without the need to download all of libjxl? or is jpegli too dependent on libjxl to be decoupled like that? something like https://github.com/cloudinary/ssimulacra2

What does `-DCMAKE_BUILD_TYPE=None` do exactly? It's not Release or Debug, so, I don't understand when you would use this. Arch builds libjxl with `-DCMAKE_BUILD_TYPE:STRING='None'` so I'm sure there's a good reason for it.

I think it’s because they have their own set of optimisation flags and don’t want CMake’s interfering

https://wiki.archlinux.org/title/CMake_package_guidelines#CMake_can_automatically_override_the_default_compiler_optimization_flag

> Each build type causes CMake to automatically append a set of flags to `CFLAGS` and `CXXFLAGS`. When using the common `Release` build type, it automatically appends the `-O3`[1] compiler optimization flag, and this overrides the default Arch Linux flag which currently is `-O2` (defined in the makepkg configuration file). This is undesired, as it deviates from the Arch Linux targeted optimization level.

Does None generate the debug code?

it passes no specific flag, so no `-g` either

but also no `-DNDEBUG`

I mean like the libjxl debug macros

on its own, it would, but Arch adds `-DNDEBUG` (https://gitlab.archlinux.org/archlinux/packaging/packages/libjxl/-/blob/1158595804b57f70769d4fa7e3dc8baeedc3428f/PKGBUILD#L51-52) so it doesn’t

one of the authors of Pijul (pmeunier) seems a bit irritated in this thread https://www.reddit.com/r/programming/comments/1b98u8g/why_facebook_doesnt_use_git/

> No chance of catching up, even though Google is definitely trying to claim some of our innovations as theirs on that one. > > The main issue is, the sequential model of Git, Mercurial, SVN, CVS, Perforce and Fossil, is actually quite naive and does not take the complexity of collaboration into account, especially around conflicts. > > The authors of Jj are trying to bullshit their way around that, claiming to take the "best of Pijul" without understanding it. Ask them what their algorithms are.

What you think about extreme compression for tgcom24 or dmove?

Do you think I'm wualified enough to understand that?

just noticed some big sites like devianart seem to actually only have one real copy of images (the uploaded original) and they dynamically serve a recompressed version on the fly even if it was already lossy - change the url a bit and you get almost instantly served any image format at any quality level desired

no jxl but low qualities in lossy formats is a sight

i wonder if smugmug could do good on their past support and add jxl to their pipeline for deviantart and flickr

Those were crap q 32.1 about, I guess nobody would give indexs to the site

This quality is fine for what's there is on internet

Avif can scale even higher

HDblog is mondadori.it a book ublosjer but dday.it same thing perhaps

storage and bandwidth savings would be massive even if they used jxl for only the top series (recompressing from the png originals to maximize quality for the same filesize as the old badly compressed jpgs)

jpegli could do wonders there, stretching their use of jpg and generating higher quality jpgs they could eventually convert to jxl at extremely low ressource consumption effort (regenerating from pngs would guarantee higher quality/smaller files if available)

afaik jpegli compresses at a higher internal precision and bit depth than the final output and against ssimulacra, deviating less from the original image compared to other jpg encoders

on dex's numbers, they push 4gb/sec all the time and 25+ terabytes traffic everyday. chopping that in half overnight would be huge, even transitionally with an inefficient jpg->jxl conversion (would still come at 0 extra quality loss)

I just noticed that all the cached thumbnails on my Linux desktop are PNG, which seems rather silly.

Perhaps that makes sense for the very tiny ones, but some folders I have in large icon view, with 256 pixel wide or tall thumbnails

Those would surely be a good deal smaller if encoded as JPEGs instead of PNGs

pngs show transparency

the vast majority of these images have no transparency, though maybe they didn't want to have any logic, and just always do something safe

clearly they should be jxl instead!

> So dolphin created a lot of thumbnails over the last 2 years. They are all saved as png, so that makes some of them bigger than the jpg original. lololol

imo png thumbs is a legacy decision that just wasnt revisited. when desktops ran with sub-hd resolutions and thumbs had small sizes, jpg loses a lot of detail

not quite an issue nowadays with fast drives, highres and large monitors but linux is at a point its shaking off its old legacy compromises

afaik DEs compliant with freedesktop specs would read thumbs whatever the format theyre in even if its png generating by default

I tried replacing a png thumb with a jpg one and it just made a new png one >.>

gnome?

kde

Now I tried but with naming the jpg as .png, and it still overwrote it with a new png 😅

afaik thumbs may be linked to og file by hash not name so a swapped in jpg would be considered a corrupt thumb

I wonder where that association is stored, then 🤔

is png an issue btw?

not really, I just thought they were a bit large

but most of them are below 100 KiB so it isn't so bad anyway

at least those formats generate quickly. someone did a compare for generating thumbnails 2 months ago. everything generated in less than 1 second... except avif with 25seconds

lol

definitely not gonna be anyone's thumbnail format soon

and lets not get into the carbon footprint

jxl would be neat to not even need premade thumbnails in the first place (load as little as just 10% of a file to show a preview)

yeah cause jxl is better

o(≧▽≦)o

still want separate thumbnails, because the image files may be on spinning rust and the thumbnails on a fast SSD

well spec is inflexible, outside that itd depend on the de's approach

theres scripts to generate working hashes for uris to jpg thumbs but known to cause issues

do u have a link?

looked before, couldnt find it back

it was against current lib versions with large images so might be reproducible

I had Windows bluescreen the other day and delete the thumbnail cache, thanks to the thumbnail generator being single threaded it's faster to just scroll through the photos app than it is to wait for the thumbnails Dx (Especially for WebP or RAW files)

afaik fjxl was obsoleted in favour of low effort cjxl. effort 1 can be heavier though, e2 and e3 consistently guarantee lower size

wasn't cjxl effort 1 replaced with fjxl?

more like opposite iinm

system thumbnailers can be inflexible but apps like galleries can handle their own internal thumbs (using whatever format they allow but its usually either png or jpg)

definitely the opposite 😛

I mean, fjxl still exists, but cjxl -e 1 is pretty much equivalent

you can use fjxl if you want a single-file version, I guess

ideal for screenshots but still needs adoption (looks at sharex and steam)

can a jpeg file compressed losslessly to jxl be recompressed losslessly again to jxl but with a higher effort ?

could i guess but defo doesnt make a big diff filesize wise

pixel data doesnt change mostly the already small metadata

not directly reconverted but going back to jpg then higher effort jxl is still almost instant

that's a possibility indeed 😄 but if it makes no difference indeed, it may be useless

if specific filesize is needed and you got a lossless/hq original, recompressing to the same filesize would always be better quality than reusing an existing jpeg that mightve used a bad/old jpg encoder in the first place

not aware of any way to target a predetermined size other than trying efforts until you hit a close number

there are some differences in speed and compression when changing the effort of jpeg recompression, but generally default effort should be fast enough and higher effort than that will not save much

24x longer for a 0.5% reduction in the quick test I just did

e7 compared to e9

imo default effort should adapt to resolution rather than have the same for everything (slightly higher for small images, bit lower for large resolutions)

That's an interesting idea...

*Should* scale rather linearly regardless of content too

any substantial amount of new heuristic intelligence tends to be useful for some contexts and needs to be inversed in other contexts where it is harmful 🙂

Is there some tool out there for Linux that can analyze ICC profiles from PNG, and ideally also extract and inject them?

exiftool to extract/attach

And I use displaycal profile info on windows, don't know if it comes with it on linux

to transform to and attach profile, magick works

https://vxtwitter.com/JohnPhamous/status/1770509175791215001 <:NotLikeThis:805132742819053610>

I was confused for a moment, and then realized the problem...

Exiftool can actually display that too. There's also a very nice GUI for it, jExifToolGUI.

https://hackaday.com/2024/03/19/your-text-needs-more-jpeg/

Would be fun to do an AVIF variant of it. Way fewer obvious artifacts, but "smoothes" the text by replacing some words with plausible different words (same grammatical role, semantical distance not too far, and more common — e.g. "chameleon" could become "frog" or "animal", or "ponder" could become "think").

perhaps each MCU should correspond to each word's semantic vector, a la word2vec?

as far as I understand the JPEG DCT text lossifier straight up takes text and treats it as pixels and feeds it to a jpeg-style quantizer

and then inverts it

they could add AQ where they quantize the start and end of a word less

makes it more readable

there is an additional remapping based on glyph similarity AFAIK

which is referred as to "perceptual charmap"

You could use GPT2 to lossily find sementically similar text, and then use GPT2's predicted probabilities for entropy coding 😄

yea, but then you have to include GPT2's dictionary or model data in every decoder

probably not worth the overall size savings <:kek:857018203640561677>

amusingly though, the article says that the mangled text is still pretty interpretable by LLMs

Hah

much past the point that humans can read it

so you can use it like a reverse turing test

like this, a human would go "what?"

if it gives an intelligible response, probably an LLM

Lmao

GPT4 in every decoder!

easy

Minimum required specs:

24GB RAM

RTX 4070 or better

<:megapog:816773962884972565>

How did you generate this text

through the jpeg dct text lossifizer linked above

I didnt generate this, I grabbed it from the blog post

Ok

https://lcamtuf.substack.com/p/afternoon-project-jpeg-dct-text-lossifizer

here's the blog post from the creator

I wrote formulas to do "rgb to ycbcr to rgb" - in order to scale the color channel in advance so that when the picture was compressed, no extra color BLUR appeared in the telegram. Does anyone know where (and in general how to Google it) -- a more accurate numerical formula rgb to Ycbcr - because the formula that is written in the JPEG documentation (on wikipedia🧐 ) - gives a small error, Including, I finally understood why it is impossible to create loseless Jpeg

The default form is always lossy, YCoCg-R is the reversible version

No, wait, look, there are numbers in the ycbcr formula that have a certain precision.... why not calculate an accuracy so strong that there would be no artifacts even when working with 32 bits per pixel

i found T-REC-T.871-201105-I!!PDF-E.pdf

the error comes from storing the value as the same size integer, not precision in calculating

original, ycbcr, YCoCg

Yes, this is really a problem. the constants are just set in BT.601 and there's nothing I can do about it.... meh

I got mixed up between YCbCr and YCoCg, but my point still stands, YCoCg-R is the reversible form

it is possible to reverse recreate the function by iterating over the accuracy of the colors after conversion

<:PepeOK:805388754545934396> I'm too lazy, I'll leave it as it is, and I won't touch it

what about keeping in floating point

I'm a little tired.

I'm already doing it.

oh that's what you mean

algebra I guess

<:NotLikeThis:805132742819053610> oh. ok.

that's it, I solved the problem, hell know, instead of rounding the output FLOAT with ROUND, I used a damn INT

I wrote rgb to ycbcr manually, because Ycbcr convert in python pillow is bullshit

<:PepeOK:805388754545934396> <:PepeOK:805388754545934396> <:PepeOK:805388754545934396> <:PepeOK:805388754545934396>

the `--loop-file` issue should be fixed now

in git master

nice!

So annoyingly, when updating to Plasma 6 on Arch, I had to reinstall `kimageformats` and I only noticed since my screenshots were no longer in JXL and I couldn't open JXL files...

maybe because old kimageformats was renamed to kimageformats5...

Question for <@604964375924834314> about white points: when I take a screenshot in MacOS, the ICC profile it embeds is a DisplayP3 one, and it has the white point set at XYZ 0.95, 1, 1.089 which is D65. When encoding it to jxl and looking at the equivalent ICC profile libjxl generates, it is one with the same color primaries but with the white point set at a different value of XYZ 0.964, 1, 0.825, and there's a `chad` tag with a chromatic adaptation matrix that I guess maps that back to D65. Is there a reason for doing it like this as opposed to simply setting the white point to D65 and dropping the `chad` tag?

yes, that’s what ICC wants https://www.color.org/whyd50.xalter > In ICC v4, the requirement was introduced that the media white point of a Display class profile shall be equal to D50 (i.e. [96.42, 100, 82.49]). This is in fact implied in v2, but many profile creators did not grasp the rationale for it and as a result some incorrect profiles were created and distributed. ICCv2 spec: > The mediaWhitePointTag contains CIE 1931 XYZ colorimetry of the white point of the actual medium. If the viewer completely adapts to the white point of the medium (as is often the case with monitors) this tag should be set to Xi, Yi, Zi (the PCS white point). If chromatic adaptation is being applied to the PCS values, the adaptation should be applied to the mediaWhitePointTag values as well.

By the way, for some context: I want to make a simple tool that takes an ICC profile as input and returns a more compact, equivalent (or near-equivalent) ICC profile as output. Some of the ICC profiles in the wild contain lots of garbage like description strings in 20 different languages or tabled versions of transfer curves where a parametric one would suffice, so it would be useful for us if we can simplify them and e.g. turn the Apple's 4 KB version of DisplayP3 into a 500 byte version.

Oh, so the profile Apple is putting in its screenshots is technically incorrect, and jxl is silently fixing it. Makes sense, cool.

Are the ICC profiles libjxl synthesizes as compact as possible, or are there still bytes that can be saved in principle? I assume the description/copyright strings could be shortened or removed (or are they obligatory?), and I guess the `cicp` tag could also be dropped for SDR profiles, but other than that it's as compact as it gets?

I’m not sure that we save every last byte but they shouldn’t be _too_ inefficient

I assume they're brotli compressed too, which helps already

Or at least, the metadata contained within... Maybe...

> 7.2.3 Preferred CMM type field (bytes 4 to 7) > This field may be used to identify the preferred CMM to be used. If used, it shall match a CMM type signature registered in the ICC Tag Registry (see Clause 5). If no preferred CMM is identified, this field shall be set to zero (00000000h). All CMMs should be registered with ICC.

https://www.color.org/registry/signature/TagRegistry-2021-03.pdf

We are writing `jxl ` in that field...

I don't really get what the purpose is of that field, is it just a way to say "this profile was generated by this software"?

> 7.2.19 Reserved field (bytes 100 to 127) > This field of the profile header is reserved for future ICC definition and shall be set to zero. I can see the ICC is taking a different approach than we had in the JXL design when it comes to header overhead 🙂

you can fit an entire <#824000991891554375> image in that obligatory padding in ICC profiles 🙂

(well you can't because those bits *shall* be set to zero, but I bet most implementations will just ignore those bits, not check that they're zeroes)

because 100 bytes is not round enough 2**7 - 1 is much more pleasing... I guess

Ok i finished my review, i'm not aware of more search compression space

Good

I don't know if i infriged Copyright

In the papers jon said i'm not sensitive to long distance banding and the kakadu Media set

I'm big fan of Berlusconi is normal i'm not sensitive to mediaset

Study at high quality I prefer berlusconi AVIFs

Jon immediately diagnosed what could be the problem

How i vote is different from JPEG neurotypicals

Because with Giuseppe Conte i like so much jpeg

Jpeg xl seems to be bad for me

https://www.openwall.com/lists/oss-security/2024/03/29/4 oops

<:monkaMega:809252622900789269><:monkaMega:809252622900789269><:monkaMega:809252622900789269><:monkaMega:809252622900789269><:monkaMega:809252622900789269>

thats a big yikes

there’s a Phoronix article as well https://www.phoronix.com/news/XZ-CVE-2024-3094

After the attack on the supply chain of pypi... Bad week for the open source

oh my

that exploit is really crazy, the attacker in question was in fact a release manager of xz for 2 full years!

so it is very likely that one intentionally did all the contributions (which should be now scrutinized after the fact) for this one exploit

there are also some evidences for related attackers who tried to get the affected version of xz more widespread

In terms of impact, the exploit wouldn't have gotten running on many production systems. Only debian sid and fedora 40 beta actually shipped the versions of the package with the backdoor. Its still not good though.

https://github.com/tukaani-project/xz/ oh well the repository has been disabled for now (as of 2024-03-30T01:23Z)

seems to be related to the suspension of the attacker's account at 01:00Z according to other articles

I think you should at least update that library to a known good version (latest). That attacker has been around for a long time

for now, yes, but in the long term I think xz should be avoided at least for the reason that the attacker has controlled the repository for too long time

it might be also possible to revert back to the old enough version to not include attacker's contributions and selectively cherry-pick good commits instead

the attacker seems to be very patient and sophiscated this time, so that level of caution is justifiable

(I personally didn't buy the reason to make xz in the first place, but this concern is orthogonal)

(and that doesn't make lzip a good alternative, as lzip author tried to claim: https://www.nongnu.org/lzip/xz_inadequate.html, either)

indeed, and that eventually led to their exposure

but that can't discount 1.5--2 years of the apparent preparation

liblzma is affected? Libtiff, libXML depended on liblzma

yes, unfortunately

> After observing a few odd symptoms around liblzma (part of the xz package) on Debian sid installations over the last weeks (logins with ssh taking a lot of CPU, valgrind errors) I figured out the answer:

the current theory seems that the exploit is designed to link with openssh, which is normally not the case but many distros link sshd with libsystemd which depends on liblzma, so it should be liblzma problem

(apparently it abused ifunc in order to be executed much earlier than normal code)

was writing too long comment about this in the HN, but I don't think lzip is good either

the biggest red flag from the lzip format is that it thinks corrupted bits in the compressed data can be caught by the decompressor itself

lol

no it isn't, cause it means that your compressed data still has some redundancy to spare!

the author probably thinks that i) the decompressor is robust enough to the untrusted and/or corrupt input stream and ii) the uncompressed checksum should be able to catch any escape

which is slightly better than the actual statement, but also much harder to prove

and it doesn't seem wise to wait for the eventual corruption before the decompression of entire data

https://arstechnica.com/information-technology/2024/03/playboy-image-from-1972-gets-ban-from-ieee-computer-journals/

anybody have any idea who IEEE computer society is? im getting jpeg group vibes

good to have, but not necessary for the compression task proper.

so I actually think it is worthy to have some very approachable archive format that also gives a good encryption and recovery feature (that is, unlike ZIP's bad encryption algorithm)

but that shouldn't be the foremost concern

partly because compression format designers are rarely good cryptographic format designers 🙂

(for that reason, I once suggested that such format should just adopt something like `age` with a minimal change.)

Ah neat, so this is where the chat is at

https://www.reddit.com/r/programmingcirclejerk/comments/1br5ffb/package_xzutils_561really5451/kx77q7y/

well it's true that researchers did maliciously put code like this attacker, so...

to be clear, I believe they have shown a worthy proof of concept, but it's also not that unreasonable that they would have been criticized as well

indeed. there is no clear evidence about the nationality of the attacker(s).

most likely they used a VPN so it'll be hard to find them

people don't do this single-handedly, they're liked backed by a large criminal organization or a state

which means support and VPNs and stuff

probably some 13 year old in kansas

Just pay them money, problem solved. Instead they tried to twist it into some vague something for sake of empty PR.

https://cyberplace.social/@GossiTheDog/112191802878969983

ah, it’s only meant to support zstd when it’s a tar inside?

I guess that explains this weird error I got when trying to extract a bare zst from the explorer:

I like the unterminated error message (“To close this Assistant, click on” – on what?)

probably machine translated with zero QA

like this one? https://twitter.com/PR0GRAMMERHUM0R/status/1643961780027228160

lmao

Why not just remove the injected shit?

Why the need for debian to use such silly conventions

I believe they’re playing it safe in case there were other backdoors in that timeframe

The attacker was the primary maintainer for over a year, so yeah...

Ah

Remind me when I'm writing my own package manager, to rely on DATES rather than making assumptions about version numbers, when determining what needs to be updated

Even pacman has that same problem. I can't believe how bad the state of Linux is sometimes

pacman works around this with epochs

Yeah, I know

It sucks

Epochs are a bad workaround

idk how Alpine does it, but I think I've seen apk say "downgrading X" when I run `apk upgrade`

how so?

I mean, aesthetically, sure, I don’t love this leading `1:`

Why do you think debian tries to avoid using epochs with +really

but it doesn’t seem like that much of a deal, all things considered

Part of the problem for Debian I believe is that other projects were already depending on features introduced after that guy became maintainer, so rolling back to a version from 1+ year ago broke stuff.

Ideally there should not be any need for package managers to parse version strings for packages that have different rules for version names

That's a stupid proposition from the beginning

The package database should just use timestamps to determine if a system is up to date

Because that's obvious

you can have specific packages use timestamps as package versions if the upstream version numbers are silly enough not to parse as version numbers

Comparing timestamps between the local system and remote system makes it easy to tell

comparing package versions too

I’m not sure what timestamps solve

And just have the version string be whatever upstream calls it

There's no need to parse the version string at all

It's just informative

it seems that you trade that for a need to either specify timestamps manually or generate them automatically (and therefore not have reproducible builds)

The package database doesn't have anything to do with reproducible builds

The database can be a plain filesystem

ah, and have additional information that is not contained in the packages themselves?

And if the remote server has a newer timestamp than you then that means updates are available

Yes

When I say database, I imply information outside of the package contents

I still think there ought to be checks that the same exact version string isn't re-used twice

But the only way to tell if one is newer is... look at the time stamp

is it timestamp of “when the package was added to the database”?

The database should be responsible for making sure the timestamps are working as intended. Whether it's a database file or the filesystem itself

Ideally just an ordinary filesystem and just use the modification time

But it can be more complex if it actually needs to be

Most package managers use a database file

Because they hate simplicity

And they need to sell certifications and training courses to use their crappy Linux server OS

They probably use a file because POSIX file systems have basically zero guarantees about anything

If you are in control of the update server you write and ensure your own guarantees

I mean, you have to get the database to the clients somehow

Since it's your filesystem

What

File systems have certain guarantees about the available operations, and it is incredibly difficult to actually implement anything reasonable on top of them

You're adding tons of extra complexity if you want to do it right

(also using tons of files would be way slower, especially on older hardware and HDDs)

Im not sure I follow the benefits of using time over version. you are just replacing one check with another check, there is no net gain

In practice a database file might be simpler. Or not.

Time is at least expected to be consistent. Not different from project to project and changing arbitrarily over time

If someone decides to make a version scheme that goes backwards instead of forwards. That's their prerogative too. Maybe I'll start doing that just to spite crappy linux distros.

You're replacing an inconsistent check that's complicated and prone to fail. With a consistent one.

the vulnerability is strongly connected to ELF `GNU_IFUNC` feature, which doesn't apply to Windows.

is there a way to deliberately corrupt images in a predictable way? curious as to how error resilience and correction works for some formats, notably jpg (mozjpg and jpegli), webp, aivif and jxl

someone did some tests back in 2021 https://www.ctrl.blog/entry/bitrot-avif-jxl-comparison.html

something important to consider is this isn't just about the bitstreams/formats themselves but what decoder implementations decide to do

I think jxl-oxide has an option to decode even if there are errors