JPEG XL

for cjxl proper, that's a major blocker

anyway, for the original image: that particular code concerns with HF context modelling, and probably triggered by a higher effort which tries to mix LF coefficients into the context.

well gist does support pretty large file, so probably yes

if libraries like this https://github.com/mackron/dr_libs/blob/master/dr_flac.h work, then j40 would surely work

(dr_flac.h by the way is to my knowledge the largest single-file library I know of, without a help of amalgamation)

ah, yes, gist shows multiple diffs in one page and that overwhelms its server

wait for, say, 1--20 seconds and try again, it will show up

(it seems that new diffs gets eventually cached)

haha

I'll set up a new repo soon for that reason

yes, it is just a git repo with a flat structure

(no directories displayed)

yeah, just like ordinary github repos

today in j40: I gave up with tight memory packing for epf and decided to go an easier route

Probably a good idea. I think epf, especially at higher strength settings (more iters), is mostly only useful for low-fidelity encoding, which I think is not really relevant in practice except for testing how low you can go

for most convolutional codes I've used alternating rows as a scratch, but this was too complex to even design in epf

the easier route is to compute intermediate bitmaps at once, which needs more memory to compute but is much easier to design and understand

yeah I strongly suggest you do it that way, at least to begin with

solving the "minimize buffers" problem for jxl's filters is... complicated

see https://github.com/libjxl/libjxl/blob/main/lib/jxl/render_pipeline/low_memory_render_pipeline.cc

first make something that works, then optimize — or even keep that for a separate fork, because I think there's also value in having a simpler-but-slower implementation

one of the reason I tried to do row-wise computation was that there is an early exit condition (sigma < 0.3) which is better suited for row-wise mode. unfortunately row-wise mode was not suitable for everything else 😉

I thought I caught this earlier

is it the same bug? I remember you have observed a bug while going through libjxl test cases and I'm yet to check that (mainly because I have tons of unimplemented features anyway)

That would be kind of premature imo, when there are still many known limitations

<@268284145820631040> https://gist.github.com/lifthrasiir/137a3bf550f5a8408d4d9450d026101f#file-j40-h-L464 I think that's supposed to be `J40__INTN_MIN`

oh, thank you! I think I would have to separately test fallbacks (I do have tests for them, but in my environment fallbacks aren't enabled)

MSVC considered it an error, made it easy to find 😉

okay, initial epf implementation done (not yet tested). another 300 lines of code.

yes, I'll publish a public repo this month (projected, if my plan goes not too bad)

my current milestone is to be able to decompress most images compressed by the default cjxl options, thus no progressive nor animation, but YCbCr would be required.

today in j40: epf is not in place, but I've implemented RAW quantization table

and I've got a scary seagull... which image for some reason I can't upload and I'm actually scared 😮

restarting discord solved the problem

obviously this is wrong, but probably only because it's YCbCr reinterpreted as XYB

for epf support I have an important question: which part of the image does restoration filters and (possibly) image features operate on? group? LF group? entire image?

entire image

it was a considerable pain to make sure gaborish and epf work across group boundaries

oh that's unexpected

since, for example, sharpness information is clearly in the LF group

yeah the signaling is split in groups, but it's important that the filter gets applied across group boundaries

that was another reason I initially went for row-wise epf implementation

my current implementation can use up to 36 times the input memory size, which is very concerning if it has to operate on the entire image

yeah it's kind of a pain, I'm sure <@179701849576833024> can describe the pain in more detail 🙂

it is *a lot* of pain, indeed

you can do it in a memory-efficient way but it requires stuff like saving strips of pixels around the group boundaries, as far as I understand

I'm okay as long as the memory usage is proportional to width, not width *times* height

the whole point of epf is to remove ringing and blockiness artifacts, including the macro-blockiness that could originate from using 256x256 groups

the libjxl version uses something like 13 * (256+16) * 3 floats of extra memory in the worst case for EPF IIRC (well, plus O(image * 16 / 256) memory for buffering group borders, but that *can* be reduced in most cases in *theory*)

is the group splitting (even in epf) mainly for parallel decoding then?

if you do not care too much about the order in which you actually decode groups, you can decode one row of groups, then EPF the top 243 or so rows, then decode the next row of groups and EPF the next 256 rows, ...

I suppose if you don't care about producing incremental epf-processed results during decode, [what veluca just said]

yeah doing it in parallel makes everything *a lot more* painful

ah, good point, yeah epf can take much time

as well as designing things such that you use O(group size * threads) memory

are you aiming at parallel decode or is single-threaded OK?

I did design for a possibility to parallel decoding in the future, but my design only reflects what I knew at that time, which is short of restoration filters

by the way, why would you need 36 times the input memory size?

for many use cases, single-threaded decode is good enough (since you can do images in parallel instead), and it does simplify things a lot of you don't aim at multithreaded

even a naive implementation would only really need ~2x the size, no?

I'm essentially computing all DistanceStep* results at once and continue, and multiplying by per-channel scale at the very end

so I have 3 bitmaps per each kernel coordinate pair

ah, I see

I guess I should make it just a single bitmap (12 total)

36 bitmaps do seem too many given this information

couldn't you just compute the output pixel directly?

you need to compute `input_pixels` values of weights, then normalize the weights and do a scalar product with the input pixels, no?

where weights are something like f(weighted L1 norm of pixel-by-pixel differences of two patches)

yeah, it should be possible but that greatly complicates an already deeply nested loop

https://gist.github.com/lifthrasiir/77bd64bd259e553c8c1ea0b45b735eac this is a glimpse of what I wrote for now

I rewrote that part of code 3 or 4 times, and in hindsight the current code is easier to turn into more efficient row-wise implementation for distances than older versions

ah, I see, the way libjxl does it is very very different indeed

Different is good for crosschecking the spec

I can't remember who was responsible, but I really want to thank whoever added jxl support to GIMP. I was doing some normal image composition earlier, completely unrelated to codecs and whatnot, and dragged in an image. It just worked so well, that it was only later I realised that it was a jpeg xl. This kind of seamless workflow is amazing, and really gives me hope for the future of the format. ❤️

While I am eager to see jxl supported by default in browsers, it is also nice if broader support is available first, so we get less of the "i hate webp! I can do nothing with these files!" sentiment...

Problem is, even with editor/viewer support, many people don't use them. They just expect the os to support it without installing anything. At least be able to see previews in Explorer, open in the image viewer, and see/modify metadata in the right click menu. The most common OS being Windows, but it will take them a LONG time to add it. I am hoping there will be an easy to install plugin that will allow those 3 main functions that I can point people to that become frustrated seeing a JXL for the first time.

I am hoping that Microsoft will be pushed by Intel (who wants HDR to 'just work', considering they're selling HDR-capable graphics chips for a while now and you can't keep selling hardware if software doesn't follow) to make JXL work out of the box asap.

For Linux, the situation is a bit ambiguous: basic (SDR) JXL support is already available in a few distros out of the box, and I expect more to follow when we do a new libjxl release, but HDR in Linux seems like it's not going to work well in general in the near future.

hdr in wayland looks like it should just work

esp. compared to how windows is

problem is people dont understand that hdr is color management <:calibrate:930792680282259497>

And of course on Mac you never know what Apple is up to until they announce it, but I'm kind of hoping that Adobe will push them to make JXL work (also considering that Apple is ahead in terms of HDR support in hardware but they still don't really have a good way to use it for still images, they seem to be stuck at some kind of semi-HDR thing where they encode a tone-mapped HEIC with some extra channel for HDR, if I understand correctly it's basically RGBE)

iirc wayland HDR stuff is still very heavily work in progress

yeah just sending raw 10-bit stuff to a monitor might work in wayland and even X, but I mean properly handling the color management so it actually makes sense

30bit color stuff only started recently on swaywm for me and bigger things like gnome don't do that yet, but for HDR you'd need more than that

iirc its something something floating point buffers

yeah, i think the idea is that hdr comes almost free if you have the basic color management done correctly, which wayland has been working on

Pretty sure Microsoft has given up on their "competitor" format JPEG XR, they only use it for HDR stuff because there's no better alternative (And the encoder is a part of the core OS)

Add a JXL WIC codec and that gets you full pipeline HDR support, as long as devs actually use the provided frameworks

JPEG XR activity is dead inside the JPEG committee, if that's any indication. JPEG 2000 is still active, for comparison (mostly for HTJ2K), and even the old JPEG-1 still has some minimal activity (mostly just updating reference implementations but still).

https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/14 & https://gitlab.freedesktop.org/wayland/weston/-/issues/467 seems main things to track for wayland HDR r n & other compositors like swaywm will possibly come later & i suspect gnome even later

I got the feeling XR died pretty early on

I remember at one point Microsoft was working with Mozilla on a implementation for Firefox, and then it just stopped ever being updated

Microsoft used to be quite slow with new technologies. After all, it took several years before MS offered 32bit and then 64bit OS to consumers.

The positioning of JXR as being somewhere in between JPEG and JPEG 2000 in terms of both computational complexity and compression to me means that it's basically even more "meh" than JPEG 2000 for widespread adoption, while it lacks most of the rich functionality and existing deployments that keeps J2K alive in niches like cinema and medical.

I've only ever seen JXR used for HDR screenshots with Xbox and Windows and by Nvidia as a replacement for OpenEXR, outside of that it's non-existent

I've seen more interest in QOI than I've ever seen for JXR 😆

If HDR would have become more mainstream a bit earlier, then I can imagine JXR would have won against WebP/

hdr is scary

i dont want to scroll down a page and it suddenly flash bangs me

i believe u can actually sorta limit it but that depends on config i g

at the same time i believe there shouldnt be a control to limit it

since that goes against the whole idea of color being a function of light and it would make all images look WRONG

i mean some displays just cant go over like 400

kinda half hdr

I dunno - intensity is rarely absolute anyway, people do want to be able to adjust the dimming of their screen

so it should be just scaling, and that would also make everything else dimmer

oh but i guess it doesnt need to be everything else if done correctly

Yeah I'm using just a regular monitor but I set it to 60 brightness because it was too bright. I suppose it just regulates the power of LED backlight.

yeah this wont work with lcd backlight

my monitor has this calibrated srgb preset that doesn't allow setting brightness

I've seen OLEDs are kinda self regulated as the maximum power consumption of the panel is limited.

its actually not too bright tho so all good

srgb is 80 nits and i also wonder why there are "srgb" presets in monitors that dont come with sensors

precalibrating is much cheaper than adding sensors

displays drift fairly quickly though so it seems like a bait to me

I'll probably get sensor at some point but so far things look kinda ok so whatever

I got a second hand one, has worked out great

one with a glass filter, like an i1Display or a ColorMunki Display from X-Rite, is an excellent choice as they will last many years

whereas the organic filters used by Datacolor’s Spyder5 colorimeters tend to degrade with time

I don’t remember whether the recent SpyderX changed this

but it’s still considered sort of “meh” by the author of ArgyllCMS https://www.avsforum.com/threads/datacolor-spyderx.3050976/#post-57785616

> **Conclusion:** > > Cheap and cheerful. > It doesn't seem like the i1 Display Pro is at all threatened in a technical sense.

Speaking of color profiles, I came across a few pictures encoded using Display P3 color profile. Despite that, I preferred the rendering using sRGB. In Display P3, the colors were too oversaturated. In sRGB, the pictures looked much better.

Another interesting observations is about LCD screens using CCFL backlight. These screens are prone to yellowing over time. Not sure if it's the tube itself or if it's a degradation of the optical materials. The whole image just shifts to yellow tones.

HDR and color gamut are kind of orthogonal things — you can have grayscale-only HDR, while you can in principle have a very wide color gamut but a dynamic range that is very limited

i like to think of hdr as "i want it this color this bright physically"

(just blabber) say you have a "400 nit" monitor, you still need to profile it so your pc actually knows, at what level creates what brightness, (max level being ~400 nits) same with color. Given the information, color management kicks in and creates the correct EOTF - signal to the display to produce the correct brightness/color

the other option is to send the data straight to the monitor but we know how that ends up (windows hdr mode)

in SDR, you could get away with assuming rgb(1.0,1.0,1.0) is just whatever the brightest white is that the monitor can produce (or wants to produce, if the user is dimming the display) in HDR, the peak brightness of displays can vary a lot, so it doesn't really make sense anymore to encode images without saying how bright things are supposed to be, or to just display them by simply equating the maxval to the max brightness of the display — that would make images created in SDR just completely blinding when shown like that on a HDR display

with that said, HDR does not necessarily imply absolute luminance, e.g. HLG is relative

but yeah, (1, 1, 1) is not diffuse white

this is an excellent document on “what HDR is about”: https://www.itu.int/pub/R-REP-BT.2390-10-2021

in particular the first section (“Introduction and design goals for HDR television”)

That's what it does in SDR mode, in HDR mode it uses BT.2100 and converts everything to that

> In traditional imaging, the range allocated to these highlights was fairly low and the majority of the > image range was allocated to the diffuse reflective regions of objects. For example, in hardcopy print > the highlights would be 1.1x higher luminance than the diffuse white maximum. In traditional video, > the highlights were generally set to be no higher than 1.25x the diffuse white. Of the various display > applications, cinema allocated the highest range to the highlights, up to 2.7x the diffuse white.

One side effect of HDR being more common will be hopefully the end of 6 bit panels.

As far as phones, it is what it is. If you have an old phone, it's not going to support newer software. I can't tell you how much of a pain it was when Safari was in-between adding some of the CSS3 features and so many people had old iPhones that didn't support much of it... but there's nothing you can do about it. Just have to try to polyfill for older phones and wait until the majority upgrade.

for whoever has Chrome 105+: HDR PNG is now supported https://old.lucaversari.it/hdr.png

Whoa that image looks quite different in various previews/viewers

indeed

Android Chrome 105

Slack preview:

and while I was at it, I modified fpnge: https://github.com/veluca93/fpnge/commit/48020025e197318ab3c695903a4577e3d9565ddb

now if you call `fpnge -pq some_image_in_pq.png x.png`, then `x.png` will display as HDR

Is Android Chrome 105 supposed to render it correctly? It looks like a bandy mess

that's probably the fault of Android's HDR rendering

In Firefox, it's a darkness.

yeah FF doesn't support HDR

I'm on Android 10, my phone display should be (somewhat) hdr capable but I suppose Android doesn't do things correctly?

more precisely, Chrome on android does weird things with HDR

FWIW, Chrome 105 on Android 13.

nice, so it depends on Android version, not Chrome version

Chrome Canary 107 on Android 10 also looks like a bandy mess on my phone

Kiwi browser bandy mess

mhhh, it also looks like a bandy mess on an up-to-date pixel 5

(I'd say in the same way)

android 12 here & same as what wb is seeing

It works on Chrome. Not Firefox. However, on both there is very visible bands. Ouch.

Which one is the more correct? I assume the dark one is wrong, but given what it does with the XYB image in <#803663417881395200> I thought EoG was better at colour management than the preview.

Or is this just a case of HDR being broken on linux?

right, there is presently no HDR on Linux AFAIK

there is supposed to be some work going on to have it in Wayland

it is plausible that X11 will never get it

(not saying that it’s strictly impossible that it will ever get it, but it seems somewhat unlikely)

that's to be expected. in fact there is an yet-to-be-explained slowdown in Windows...

which compiler did you use by the way? J40 doesn't directly use SIMD so its performance greatly depends on compiler's ability to autovectorize.

in my experience recent enough versions of both gcc and clang are doing great jobs (in fact, I have both compilers in both Windows and WSL and the Windows slowdown appears with both compilers so I think it is not a compiler problem)

no `-O3` or similar?

that's very interesting, it should be equivalent to `-O0` and it would be very much slow.

Oof.

Wait wait Jurume.

You are currently developing J40 in Rust, correct? Do you think you can make a crate that does RGB <> XYB and YCbCr <> XYB transforms with input and output pixel formats? That would actually be quite nice to have, and if you have enough time, you can even add in metadata support since that will be needed.

uh, no? it's in C, with a parallel Rust version planned.

by the way the algorithm itself is not hard to make, I think I can just write it in a snippet

```rust struct RgbImage { width: usize, height: usize, r: Vec<f32>, g: Vec<f32>, b: Vec<f32> } struct XybImage { width: usize, height: usize, x: Vec<f32>, y: Vec<f32>, b: Vec<f32> } enum ConversionError { TooLarge } // XYB to linear RGB, assumes max intensity of 255 nits fn xyb_to_linear_rgb(xyb: XybImage) -> Result<RgbImage, ConversionError> { const OPSIN_BIAS: f32 = -0.0037930732552754493; const CBRT_OPSIN_BIAS: f32 = -0.15595420054924863; const OPSIN_INV_MAT: [[f32; 3]; 3] = [ [11.031566901960783, -9.866943921568629, -0.16462299647058826], [-3.254147380392157, 4.418770392156863, -0.16462299647058826], [-3.6588512862745097, 2.7129230470588235, 1.9459282392156863], ]; let npixels = xyb.width.checked_mul(xyb.height).ok_or(ConversionError::TooLarge)?; assert!(xyb.x.len() == npixels); assert!(xyb.y.len() == npixels); assert!(xyb.b.len() == npixels); let outr = Vec::<f32>::with_capacity(npixels); let outg = Vec::<f32>::with_capacity(npixels); let outb = Vec::<f32>::with_capacity(npixels); for (x, (y, b)) in xyb.x.into_iter().zip(xyb.y.into_iter().zip(xyb.b.into_iter())) { let l = (y + x) - CBRT_OPSIN_BIAS; let m = (y - x) - CBRT_OPSIN_BIAS; let s = b - CBRT_OPSIN_BIAS; let lmix = l * l * l + OPSIN_BIAS; let mmix = m * m * m + OPSIN_BIAS; let smix = s * s * s + OPSIN_BIAS; outr.push(OPSIN_INV_MAT[0][0] * lmix + OPSIN_INV_MAT[0][1] * mmix + OPSIN_INV_MAT[0][2] * smix); outg.push(OPSIN_INV_MAT[1][0] * lmix + OPSIN_INV_MAT[1][1] * mmix + OPSIN_INV_MAT[1][2] * smix); outb.push(OPSIN_INV_MAT[2][0] * lmix + OPSIN_INV_MAT[2][1] * mmix + OPSIN_INV_MAT[2][2] * smix); } Ok(RgbImage { width: xyb.width, height: xyb.height, r: outr, g: outg, b: outb }) } ```

something like this? I haven't tested it but the logic should be clear <@321486891079696385>

Ahh OK OK. I though you made your decoder in Rust, so I thought you could make the colorapace conversion code into a crate 😂

Sorry for bothering you in that regard :)

ah, I guess that part of code (if I would make a crate) will be private anyway 😉

random guess: that mips cpu has no float support 😉

Looks like it, it apparently uses the MIPS 1004Kc arch, but 1004Kf is the one with float support

I'm actually surprised that softfloat can be *that* fast

yeah it's not bad at all

I'm trying to compile j40 on Debian testing and it doesn't work. What I am doing wrong? I have gcc and build-essential installed. I have both j40.c and j40.h in the directory. gcc -lm j40.c It's still complaining about missing powf, hypotf and cbrtf.

perhaps `gcc j40.c -lm` may work, I see no other reason for the error

The order matters. It worked. Thank you! I thought it's something simple.

yeah, C/C++ link order is something almost magical

on arch any order always worked for me and on ubuntu it was particularly picky (only satisfying dependencies if you include things asking for symbols first & only including libs satisfying them later)

i never went deep on figuring out why but it's been like that for years

maybe ubuntu compile their stuff with different options

Where is the source code for j40. I cannot find it <:SadCat:805389277247701002>

on some github gist iirc

ctrl+f gist i g

thanks

-O3 240kb and -O2 168kb

huh, I thought I fixed it

older versions had a wrong stride in the PNG writer, but it looks like another bug

ah, found it, I forgot to update `j40__combine_modular_from_pass_group` to use the plane interface

and my working copy did fix that

so, short answer: replace that function with the following: ```c J40_STATIC void j40__combine_modular_from_pass_group( j40__st *st, int32_t num_gm_channels, int32_t gy, int32_t gx, int32_t minshift, int32_t maxshift, const j40__modular_t *gm, j40__modular_t *m ) { int32_t gcidx, cidx, y, gx0, gy0; (void) st; for (gcidx = num_gm_channels, cidx = 0; gcidx < gm->num_channels; ++gcidx) { j40__plane *gc = &gm->channel[gcidx], *c = &m->channel[cidx]; J40__ASSERT(gc->type == c->type); if (gc->hshift < 3 || gc->vshift < 3) { size_t pixel_size = (size_t) J40__PLANE_PIXEL_SIZE(gc); size_t gc_stride = (size_t) gc->stride_bytes, c_stride = (size_t) c->stride_bytes; (void) minshift; (void) maxshift; // TODO check minshift/maxshift!!! J40__ASSERT(gc->hshift == c->hshift && gc->vshift == c->vshift); gx0 = gx >> gc->hshift; gy0 = gy >> gc->vshift; J40__ASSERT(gx0 + c->width <= gc->width && gy0 + c->height <= gc->height); for (y = 0; y < c->height; ++y) { memcpy( (void*) (gc->pixels + gc_stride * (size_t) (gy0 + y) + pixel_size * (size_t) gx0), (void*) (c->pixels + c_stride * (size_t) y), pixel_size * (size_t) c->width); } printf("combined channel %d with w=%d h=%d to channel %d with w=%d h=%d gx0=%d gy0=%d\n", cidx, c->width, c->height, gcidx, gc->width, gc->height, gx0, gy0); fflush(stdout); ++cidx; } } J40__ASSERT(cidx == m->num_channels); } ```

long answer is that I'm in the way of another restructuring so I can't post the working copy to the gist as it won't compile yet 😦

this time the restructuring is to do with supporting permuted TOC, which requires random access to some extent

uh, no `gc`? I think it's in the line 2784 in the currently published version.

replaced the diff with a full code

and matched a function signature

...again

(one of the pending updates is that I avoid using `_t` suffix for types, as it is reserved)

oh well.

```c #define J40__PLANE_PIXEL_SIZE(plane) (1 << ((plane)->type & 31)) ```

that's why I'm yet to publish the full repo, it is still rapidly changing...

at the top of my task list there has been "implement `j40_advance`" for literally weeks, everything else is its subtask or sub-subtask or something like that

- I need to implement API instead of a hastily written PNG writer - most API functions depend on `j40_advance` which tries to parse as much as possible or to the certain requested point (e.g. the first frame) - `j40_advance` can run multiple times, so its state should be retained (it is possible to write a coroutine in pure C but it is really annoying to write it by hand, this time though it is easy to maintain so it's gonna be a coroutine at least in `j40_advance` and some key functions) - `j40_advance` internally calls `j40__frame` which should also be a coroutine - `j40__frame` had a long-standing bug that it neglects TOC permutation so I've got to fix this at last - TOC splits a frame codestream into multiple sections which should not overlap; it is best to define a nested context for each section (like libjxl `BitReader`), and there is already a support for that - aaand I realized that it is impossible to jump to a certain codestream offset without a mapping from codestream offset to file offset, which can only constructed by full scanning (aargh) - so I rewrote `j40__container` to construct that mapping (<- right now)

essentially, yes! everything else would extract some data from its output.

how does this work? I was under the impression that the cube root was taken after the bias, not before, when converting *to* XYB

so you should be cubing it and then adding the bias

I'm looking at CBRT_OPSIN_BIAS specifically

I think it's a faithful translation of the following pseudocode in the spec:

also note that I've moved `- cbrt(...)` into previous statements

I see, so that's straight up the cube root of the previous things

this has to be able to be done more cleanly

I avoided using `lgamma` etc. for that reason, was my translation not clear enough?

no, I meant, there's gotta be some simplification of the above using algebra

In particular, if you let `B = (Lgamma - A)` then `B^3 + A^3` factors into `(B + A) * (B^2 - BA + A^2)`

you can then write `(B^2 - BA + A^2) = (Lgamma - A)^2 + (Lgamma - A) * A + A^2 = (Lgamma - A)^2 + A Lgamma`

and you can write this entire thing as `Lgamma * ((Lgamma - A)^2 + A Lgamma)`

and `(L gamma - A) ^ 2 + A Lgamma = L gamma ^2 - 2 A Lgamma + A^2 + A Lgamma = Lgamma^2 - A Lgamma + A^2`

so you end up with `Lgamma * (Lgamma^2 - A Lgamma + A^2)`

which I suppose you'd get anyway when you just expanded the cubic?

maybe this doesn't help in this specific case but there has to be some sort of way to bake some of this stuff in

In any case it's just a few muladds

in particular, if you look at the spec, `itscale` can be baked into the opsin inverse matrix since it's just a scalar multiple

once you do that, you get `Mix = (something) + Bias`, and so when you multiplye `Opsin * Mix` you can break that into `Opsin * ((something) + Bias)` and get `Opsin * (something) + Opsin * Bias` and `Opsin * Bias` can be baked into it

sure, but I feel like it *could* be optimized, especially if this is done in fixed-point

yeah, for J40 it doesn't make much sense because it exclusively uses fp, but fixed point may benefit from that

although it sounds like this particular colorspace conversion is probably not the most computationally intensive part

in the case of J40 the most computationally intensive code is linear to sRGB conversion

which is very slow at the moment, because it just calls powf and I think it's not vectorized

linear to sRGB involves a branch, right? value^2.4 if the value is bigger than a certain amount, right?

yes, but that can be made branchless

the real problem is that one of those branches is disproportionally expensive

weird question but what if you did `expf(2.4 * logf(value))`

would that be faster than `powf`

it is hard to say, I think most powf implementations internally do that, but range reduction will happen once in powf but twice in expf + logf

in libjxl there's some *black magic* to speed up the sRGB tf (well, specifically x**2.4)

(also only in the u8 output path on arm)

yeah, I think it approximates the whole branch with dedicated code

https://github.com/libjxl/libjxl/blob/main/lib/jxl/dec_xyb-inl.h#L104

ah, sorry, apparently to approximate the other direction

basically: write x as 2^e * f with f in [0.5, 1), poly approx for `f**2.4` and lookup table for `(2^e)**2.4`

I thought this: https://github.com/libjxl/libjxl/blob/35f187f/lib/jxl/transfer_functions-inl.h#L344

hah I didn't remember I wrote that one first

well if you want to approximate `x^2.4` you could let `f(x) = x^2.4` and let `f'(x) = 2.4 x^1.4`

then you have `f(x)/f'(x) = x/2.4`

etc. etc. newton's method

actually wait newton's method is for approximating the zeroes, ignore me im being dum

yeah no that doesn't work, you could Newton on something like `y^5 - x^12` (which has `x^2.4` as a root) but that's still quite expensive

(you need `(y - x^12/y^4)*0.2` I believe)

is there anywhere you can just download a recent butteraugli and/or ssimulacra2 exe?

With x^(1/3) we are approximating log(x + C), not x^(1/2.4)

I was referring to approximating the sRGB transfer itself

ah, thanks

the idea was to do it faster than via powf

I'd have to think a bit harder about that though

this was an interesting read about the mathematics of sRGB

http://entropymine.com/imageworsener/srgbformula/

jxl larger than png at effort 8 too. 😦

Can you send the image?

https://www.deviantart.com/dpcdpc11/art/Isolation-Wallpaper-Pack-5120x2880px-836086303

Thx

Pictures with low number of colors sometimes require some tweaking of cjxl parametters. Might be a good candidate for making use of color palette.

Yeah I know. But I thought issues like this were fixed in recent builds.

cjxl isolation_v02_5120x2880.png -d 0 isolation_v02_5120x2880.png.jxl -g 0 JPEG XL encoder v0.7.0 [AVX2,SSE4,SSSE3,Scalar] Read 5120x2880 image, 4204461 bytes, 188.6 MP/s Encoding [Modular, lossless, effort: 7], Compressed to 4042055 bytes (2.193 bpp). 5120 x 2880, 0.81 MP/s [0.81, 0.81], 1 reps, 4 threads.

cjxl isolation_v02_5120x2880.png -d 0 isolation_v02_5120x2880.png.jxl -g 0 -X 0 -Y 0 -e 9 --patches 0 JPEG XL encoder v0.7.0 [AVX2,SSE4,SSSE3,Scalar] Read 5120x2880 image, 4204461 bytes, 199.2 MP/s Encoding [Modular, lossless, effort: 9], Compressed to 3790813 bytes (2.057 bpp). 5120 x 2880, 0.14 MP/s [0.14, 0.14], 1 reps, 4 threads.

so not impossible to beat png but kind of annoying that it doesn't by default

today in j40: still working on TOC permutation and backing buffer management version... 10?

there is a bug in gtk jxl loader plugin or bug in image viewer. it cannot properly decode images encoded with -d 25.

top -d 12, bottom -d 25

Is it normal to encode images slower at high distance? For me, it takes more time to encode image at d25 compared to -d1.

`❯ cjxl bb.png d1.jxl -d 1 JPEG XL encoder v0.8.0 1c19bb48 [AVX2,SSE4,SSSE3,Unknown] Read 1920x1200 image, 4040483 bytes, 64.1 MP/s Encoding [VarDCT, d1.000, effort: 7], Compressed to 402830 bytes (1.399 bpp). 1920 x 1200, **5.01 MP/s** [5.01, 5.01], 1 reps, 4 threads. ❯ cjxl bb.png d25.jxl -d 25 JPEG XL encoder v0.8.0 1c19bb48 [AVX2,SSE4,SSSE3,Unknown] Read 1920x1200 image, 4040483 bytes, 67.4 MP/s Encoding [VarDCT, d25.000, effort: 7], Compressed to 3255 bytes (0.011 bpp). 1920 x 1200, **0.27 MP/s** [0.27, 0.27], 1 reps, 4 threads. `

In https://github.com/libjxl/libjxl/blob/main/lib/include/jxl/encode.h#L1084 there is written that distance range is 0 .. 15 and recommended 0.5 .. 3.0 How is it possible to use 25 distance?

I was just testing. I actually never use above d1.5 for my photos.

I've noticed that in very high distance, vardct just gives up and lossy modular is used instead. It also uses downsampling, although downsampled pictures are often decoded incorrectly.

oh that's why it takes more time!!!

Old versions of libjxl had a bug in some decoding code paths where upsampling is needed - I think those bugs should be fixed now

At some point if you use very high distance, libjxl will just downsample the whole image and encode that instead. Obviously that causes things to be blurry.

resampling <:YEP:808828808127971399>

file a bug on libjxl ?

I spent earlier some time improving the performance at distance 128 (to be less embarrassing in somewhat unnecessary low quality benchmarks), now, thinking about utility and actual use, I consider it time wasted -- should have only been looking at d4 and better 🙂

but in any case, it shouldn't fail or crash at high distances -- that is just unnecessary excitement

what is the maximum reasonably possible distance in butteraugli? at that point it can just average all pixels and emit a single-color image...

in the very beginning I calibrated between 0.5 and 2.0, the decision boundaries were 0.6, 0.8, 1.0 and 1.3 or so

the training material fell into those categories

later I made it work less wildly outside of this area by other kinds of tweaks, possibly up to d8 or d16 or so

however, the aggregation of the data is often with a higher p-norm, whereas a poor quality image would benefit from a low p-norm (such as 2-norm)

libjxl depends on highway (for SIMD operations). (Un)fortunately, highway is very sensitive to compiler-related bugs. Those issues are being discovered when someone wants to try libjxl on a rare architecture. Recently I have got a report about https://bugs.gentoo.org/869077 and we are searching for the solution.

you know more about it than I do

consider filing an issue and then we can together start to build clarity on what is going on -- we don't need to know all scenarios beforehand to file an issue

I'm not aware of a change that breaks backward compatibility for cjxl/djxl -- and I don't think we took such risk intentionally after freezing the format in end of 2020

we did some backward incompatible changes in the format definition, but we were rather sure that those were not paths that were exercised in any cjxl version we had written

AFAIK we don't have a lot of tests for backward compatibility yet, like no newer encoders being decoded on old decoders, so we can mess up perhaps too easily

Thank you 🙂

Please attach the new-20.jxl to the issue too.

Nothing was done that breaks backwards compatibility. Old versions of djxl do have some bugs though, e.g. in upsampling

consider retweeting about xyb jpeg: https://twitter.com/jyzg/status/1569249920556888066

haha, here it got greenwashed

xyb needs to find more uses, it seems

how does xyb-jpeg work?

I thought JPEG forced a specific YCbCr matrix

through ICC v4 profile

JPEG without a color profile is commonly interpreted with a special YCbCr matrix or is stored directly in RGB or CMYK

about half the savings are from expressing the image in XYB

the other half is from using JPEG XL's adaptive quantization heuristics through variable dead zone quantization (no real adaptive quantizaiton possible in old JPEG)

the same trick that we used originally in guetzli

so as long as the ICC Profile is respected it will be viewed properly

unfortunately firefox greenwashes it

and ignores the ICCP

yes, we have some ideas on how to make it less disruptive when there is no color management

ICC v4 is available in firefox, but default off

.... why?

and how do you turn it on?

Should make it more disruptive, squeaky wheel gets the grease :p

Set `gfx.color_management.enablev4` to true in about:config

looks like that worked, thanks

And `gfx.color_management.mode` to 1 for good measure if you haven't done that, by default it only does colour management for images with embedded profiles, page colours are left as-is

https://bugzilla.mozilla.org/show_bug.cgi?id=488800 has discussion

could this v4 profile theoretically be detected automatically by cjxl the way that sRGB and other common profiles are auto-converted to enums? and if so, could you make it so XYB-jpeg is auto-reconstructed as XYB VarDCT losslessly?

or would there really be minimal reason to do that

XYB can be expressed in ICC v4 by using a 600 bytes ICC profile description

sometimes people encode once -- not for a particular client

are JPEG ICCs compressed?

they can be, if HDR or wide gamut they need to be

or mozjpeg?

the one in libjxl

comparison to mozjpeg is the fairest comparison IMO

does libjxl have a JPEG encoder?

I thought it only has a JPEG decoder

yes, sort of

https://github.com/libjxl/libjxl/pull/1735

hm

https://storage.googleapis.com/jxl-quality/eval/61297ec/index.jpeg_libjxl_d1.0_nr_q80.html vs https://storage.googleapis.com/jxl-quality/eval/61297ec/index.jpeg_q80.html

i wonder what's blocking that firefox bug about iccv4/ preventing then from enabling it by default and mode 1

all the open blocking bugs are seemingly fixed and i would say even works better than chrome right now

last (private) sprint for J40 in progress, I've completed j40_advance (finally!) and am now testing it

this took an eon

what were the prefixes before it was named j40

jxsml

oh...

I agree, and I am worried about this equally

we are looking for mitigations and have an idea that will very likely make the previews to look ok, possibly even improves the accuracy a bit

basically it would be to use tye YUV instead of RGB as a basic mode, and map Y=Y, U = B, V = X -- instead of R=X, G=Y, B=B today

it can feel a bit scary, but I think it will largely fix the problem we have today, and then there will only be a minor shift between the colors

That's a nice difference. Works quite well when the encoder is actually tuned to utilizing XYB internally.

https://twitter.com/Foone/status/1569352203575652352 is jxl? <:Thonk:805904896879493180>

isn't palettizing always a lossy operation?

(except if the input already fits in a small palette, of course)

yeah, I don't see how *lossless* paletted image is possible in general

https://twitter.com/Foone/status/1569353108723216384 this seems the actual question

<@604964375924834314> any idea whether we could make a XYB ICCv2 profile?

(although admittedly, that bug should be fixed after *13* years...)

from what I remember, when we looked into it, we decided to make it an ICCv4 profile because we specifically needed a certain feature

at least to make it compact

otherwise I guess we could always use a dense LUT

(and not just the 2×2×2 “LUT” we use to emulate a matrix product)

(if I remember right?)

IIRC v2 doesn't do parametric trC

yeah, that’s probably what we were missing

but maybe the LUT for those isn't too bad

q1 is whether we can use the same LUT for all 3

q2 is how big the LUT needs to be to get good quality

if it’s the same TRC then we can use the same entry in the file, if that’s what you mean

right

but we have biases

i.e. the TRCs are `(x-a)^3 + a^3` or something like it

and the `a` are not all the same

ouch

now, the first -a we can (probably) get rid of by changing the 3d LUT

(as I believe that can express any affine transform)

but the +a^3 I don't think we can?

maybe by (ab)using the white/black points?

i think they mean lossy representation of the palette indices

well given a later tweet I think the author actually wants a fixed palette

Yeah, pointed him to pngquant

those 256 colors are pretty surely not configurable

and probably 2-3-2 or similar, forgot which one has 3 bits

Green probably

(off-topic: and I searched Wikipedia and accidentally typed 256 *bits*, wonder how 256-bit colors would look like...)

anyway, R3-G3-B2 is a norm

The "safe for web" palette is also interesting, iirc it's something like a 6x6x6 color cube, so not integer bits for rgb

6x6x6 plus shades of gray I think

00 33 66 99 cc ff

It left some room for "OS defined colors", iirc

256-bit colors make my wildest dream, can we actually encode a spectral approximation lol

8 channels of 32-bit samples

Not too wild

just 8 color model then?

CMYK+OGV+Alpha

That is plausibly something a fancy application aiming at print might want to use

Of course 32-bit is a bit insane

I've seen non-CMYK four-color model back when I looked at the mixbox paper https://scrtwpns.com/mixbox.pdf

which is by the way a really visually appealing work, see also https://scrtwpns.com/mixbox/

author uses they/them

Ah oops

how does using Y,B,X instead of YCbCr actually improve the compression? is the improved transfer function really that impressive?

I keep hearing that XYB as a coding tool improves visual quality at the same bpp, but how does it manage to accomplish this?

more perceptually relevant and uniform means loss is also in the perceptual "least significant bits"

Considering the requirements, it seems the pictures are going to be displayed using basic VGA/MCGA hardware.

The default 256 color VGA palette is neither 3:3:2 or 6x6x6, it may be redefined though.

heh, indeed, it seems that the palette itself is actually an 18-bit color.

Yes, VGA can do only 6 bits per channel.

Later video cards could do 8 bpc but the legacy VGA modes were still limited to 6 bpc.

pngquant looks pretty good. Small paletted PNGs are manageable even for very old PCs.

I'm looking for the final API tweak before initial release of J40 (which is hopefully later today)

basically, ```c j40_frame frame; if (j40_next_frame(&image, &frame) == J40_OK) { // or while(...) for multiple frames // do something with frame } ```

I don't like this, for the reason that this would be the only place that `J40_OK` has to be used.

*normally*, people will then try the following design: ```c j40_frame *frame = j40_next_frame(&image); if (frame) { ... } ```

this is reasonable, except that a lifetime management is way harder or even impossible.

I want that, I don't need to free `j40_frame` manually, and yet want to be notified when I use `j40_frame` past its intended lifetime (for example, next `j40_next_frame` call may invalidate it)

pointer gives a clear interface but unless you allocate a tiny bit of memory every frame I don't see how the second goal can be achieved

in comparison, a struct may have a sort of counter or similar to detect the change to the internal frame structure, without actually allocating anything

if you meant jxl-to-jpeg reconstruction by "JPEG transcode" it's not a scope of J40 at all

patches and YCbCr are of high priority but the release was delayed for a long time so I will omit them in the initial public release (and put a big warning sign)

after all, I have this code: ```c #ifndef J40_CONFIRM_THAT_THIS_IS_EXPERIMENTAL_AND_POTENTIALLY_UNSAFE #error "Please #define J40_CONFIRM_THAT_THIS_IS_EXPERIMENTAL_AND_POTENTIALLY_UNSAFE to use J40. Proceed at your own risk." #endif ```

back to where I have left: so my current design is as follows: ```c if (j40_next_frame(&image)) { // note no frame argument j40_frame frame = j40_current_frame(&image); // will put a placeholder j40_frame on failure ... } ```

this split is clear in design, I'm less sure that this is actually safe

I'm almost sure that if `j40_current_frame` is called without prior `j40_next_frame` it will have to call `j40_next_frame` automatically

how about other edge cases though? I'd appreciate any other possible misuses possible in this design.

I actually haven't tested this, I guess it's not too many because in the current conformance tests, each image tests a distinct feature and not its specifics; J40 still lacks quite many features when you count them naively

say, "modular" and "VarDCT" are the biggest features and may require thousands of tests for full testing, but if naively counted they are just two out of many features

the conformance repo mostly contains bitstreams that combine lots of things and that are "hard" in general to decode correctly — passing conformance is intended to mean that it's quite likely that everything was implemented and implemented in a correct way.

probably a better way to indicate what is supported is to say what kind of encoder output you can handle: I would guess that J40 can currently decode fjxl output and anything produced by cjxl -e 1-6 (both lossless and lossy) with png input and without extra flags, and recompressed RGB 4:4:4 jpegs (decoding only to pixels of course, not to reconstructed jpeg)

with patches and YCbCr that brings it up to any speed setting of cjxl and any recompressed jpeg

then I guess there's the multi-layer / multi-frame thing that requires blending to be implemented properly (needed for e.g. dealing with recompressed apng animations), and then there's dealing with progressive passes, and then I think it will decode anything libjxl can produce

splines are not currently produced by libjxl, the only way to produce jxl files with splines is using jxl_from_tree atm. I'd still add it to J40 of course, but that's probably the lowest priority thing to add to it.

yeah, after the public release the next priority will be multi-frame support for patches

(and I'll put a proper issue tracker there)

thank you for this summary, I guess I'll steal it (with an addition of `--patches 0`, I think it is enabled by default?)

ah and `--dots 0`, this is also enabled by default

iirc patches and dots are disabled by default at e<7, but best to check the code

maybe it's still doing patches at somewhat lower effort in modular mode

it is

SpeedTier has the following docs: ``` // Turns on dots, patches, and spline detection by default, as well as full // context clustering. Default. kSquirrel = 3, ```

so I guess up to `-e 2` are fine but above are possibly not

more accurately, libjxl seems to enable patches for `-e 3` and above and dots for `-d 3` and above

so... `-e 1 -d 3` can actually enable dots as I can tell?

-e 3 is not kSquirrel though xD

(confusing? yeeeeeep)

waaaait

huh indeed

yeah, SpeedTier n is equivalent to `-e (10-n)`

so it's actually `-e 7` and above, well

and dots are not enabled in the same condition, `FindDotDictionary` has its own condition and the caller has another

yeah but the comment lies -- for lossless that's *not* the speed they get enabled at

IIRC it's somewhere in enc_modular.cc

I see this code, which is the only occurrence of `FindDotDictionary` ```cpp if (info.empty() && ApplyOverride( state->cparams.dots, state->cparams.speed_tier <= SpeedTier::kSquirrel && state->cparams.butteraugli_distance >= kMinButteraugliForDots)) { info = FindDotDictionary(state->cparams, opsin, state->shared.cmap, pool); } ```

oh I meant for patches

isn't this the sufficient condition?

ah

```cpp if (do_color && metadata.bit_depth.bits_per_sample <= 16 && cparams_.speed_tier < SpeedTier::kCheetah && cparams_.decoding_speed_tier < 2) { FindBestPatchDictionary(*color, enc_state, cms, nullptr, aux_out, cparams_.color_transform == ColorTransform::kXYB); PatchDictionaryEncoder::SubtractFrom( enc_state->shared.image_features.patches, color); } ```

this must be it then

no no that's tied to --faster_decoding

not to `-e`

this moment in j40: hahahahahaha

enlarged, this is a placeholder image returned by a frame when the error has occurred

I even considered to dynamically generate a placeholder image depending on the current error message, possibly in the thread-local storage, but I can always do that later

anyway, yeah, the full end-to-end testing with a new CLI works for my main test images!

https://gist.github.com/lifthrasiir/137a3bf550f5a8408d4d9450d026101f has been updated and this would be the last gist-hosted version

Error images is a nice touch, can be convenient for applications that don't want to deal with errors

I've done initial testing and a list is miserable lol https://github.com/lifthrasiir/j40/issues/1

amazingly enough this doesn't matter much for actual images encoded by cjxl

pretty bad, it broke malloc

yes exactly, if I knew J40 would be this large I'd used Rust instead

this will be a huge problem if you use it as a library

if you use it as an executable, it might still be a problem because memory corruption can be potentially converted to remote code execution

there may be reasons not to use Rust, but if it's just an annoyance as opposed to the actual problem (e.g. logistics) then it doesn't seem reasonable

found a bug; `malloc` size was off during `j40__read_dq_matrix`

ngl writing it in Rust would likely have been much easier 😄

some things deserve C for ideological reasons

Rust may have prevented accidental bugs, but architectural issues would be same regardless of languages 😉

having written J40 in C already, porting it to Rust should be much, much easier

not because Rust is a better language but because I've done most of works already

i personally see minimalist C implementations as breath of fresh air in otherwise bloated and overcomplicated softwarelands

recently there were lots of lightweight system languages, here lightweight means "no strong protection against temporal memory safety"

most of them still provide spatial memory safety (automatic free, to name a few) so they are still measurably safer than C

I once considered writing J40 in Zig for that reason

honestly something like zig implementation would be cool too. but rust implementation would probably have more popularity because rust is more buzz word and lesser likelihood for bugs

yeah, writing in C was only for maximal portability (which I considered important for the future of JPEG XL)

not being faster is very expected and unlikely will ever change

otherwise it is a subpar choice

it's a good benchmark of how complicated a standard is

i mean the specification i guess

looks like multi-threading enabled things versus single-threaded thing, also expected

yes, J40 currently doesn't support multi-threaded decoding (but most infrastructure is there, just not prioritized)

should I just enable asan for dj40 builds 🙂

enable all the sans

for now, yes, it supports only a tiny subset of what it should support in the future

you can see that even `J40_RGB` is commented out 😉