|
lonjil
|
2022-11-08 12:13:14
|
remind me, what viewing conditions, pixel size, and distance from the screen is d1 supposed to be tuned after? (or however it works)
|
|
|
Jyrki Alakuijala
|
2022-11-08 12:13:28
|
it is like q90 libjpeg turbo
|
|
2022-11-08 12:13:53
|
you might be able to type q85 etc. in its place
|
|
2022-11-08 12:14:07
|
this encoder is magical only in the higher quality
|
|
2022-11-08 12:14:15
|
I suspect that at q70 it gets really boring
|
|
2022-11-08 12:14:32
|
but at q85 is should be already magical
|
|
2022-11-08 12:14:39
|
and above q90 it is 'legendary'
|
|
2022-11-08 12:15:20
|
(sorry about the poor humor, it is late here and has been a long week already)
|
|
|
|
lonjil
|
2022-11-08 12:18:56
|
Just wondering because I know d1 is supposed to look transparentish as long as you don't look too close, but my first test image with the jpeg thing has quite obvious artefacts
|
|
2022-11-08 12:19:15
|
currently trying to remember how to convert the original png into a format that cjpeg will accept...
|
|
|
|
Jyrki Alakuijala
|
2022-11-08 12:19:46
|
for one corpus I get 1.9 BPP with d1, and 2.9 BPP with libjpeg-turbo q90, with similar scores
|
|
2022-11-08 12:20:10
|
if I generate two versions with 1.9 BPP, the one from libjpeg-turbo look hideous
|
|
|
|
lonjil
|
2022-11-08 12:29:03
|
I need to go to bed, so I will leave it at that one image for tonight, will try more tomorrow. cjpeg -quality 83 made an image about the same size as the benchmark command above. they seem to have about the same amount of artefacting when zoomed in, but the libjxl one has smoother artefacts that change the color more, while the libjpeg one has harsher artefacts that keep the color better. not zoomed in, the change in color is more noticeable. this is a flat color drawing. I will test some photographs tomorrow.
|
|
|
|
Jyrki Alakuijala
|
2022-11-08 12:37:47
|
thank you
|
|
|
The_Decryptor
|
2022-11-08 12:55:24
|
I've only really got video game screenshots to play with, first one I tried got really good results
|
|
2022-11-08 12:55:56
|
Filesize equivalent to GIMP's JPEG output at quality 80, but much higher quality with clearer edges and less artifacts
|
|
2022-11-08 01:01:45
|
This bit stood out to me, output from benchmark_xl on the left, gimp on the right
|
|
|
Traneptora
|
|
lonjil
currently trying to remember how to convert the original png into a format that cjpeg will accept...
|
|
2022-11-08 01:09:06
|
mozjpeg accepts PNG input
|
|
2022-11-08 01:09:18
|
otherwise, TGA, `ffmpeg -i input.png output.tga`
|
|
2022-11-08 03:17:37
|
still very confused, this is the spline I'm getting
|
|
|
_wb_
|
|
Traneptora
is this a standin for channels[0], channels[1], channels[2]?
|
|
2022-11-08 05:45:51
|
Yes, the spec should say that somewhere, that any mentions of "X,Y,B" should be interpreted as channels 0,1,2 also in the RGB and YCbCr cases
|
|
|
Traneptora
still very confused, this is the spline I'm getting
|
|
2022-11-08 05:51:12
|
That doesn't look like the right shape, so maybe the spline control points aren't decoded correctly?
|
|
|
|
Traneptora
|
2022-11-08 06:05:33
|
it's also not the right color
|
|
2022-11-08 06:05:48
|
the code in libjxl for determining the color differs pretty significantly from the spec as well
|
|
2022-11-08 06:05:54
|
so it's gonna take some figuring out what to do
|
|
|
|
_wb_
|
2022-11-08 06:23:13
|
Yes, there could very well be undiscovered spec bugs in splines, since you're the first one to independently implement it
|
|
|
|
Traneptora
|
2022-11-08 06:49:42
|
it's not a typo though, the whole structure is wildly different
|
|
2022-11-08 07:28:26
|
well I fixed the color issue, shape is still wrong though
|
|
|
|
dds
|
|
_wb_
Yes, there could very well be undiscovered spec bugs in splines, since you're the first one to independently implement it
|
|
2022-11-08 07:52:26
|
the spec doesn't prohibit identical adjacent points (causing divide-by-zero errors) AFAICT. This is checked for in libjxl though.
|
|
|
diskorduser
|
|
Traneptora
well I fixed the color issue, shape is still wrong though
|
|
2022-11-08 07:52:33
|
Looks nice
|
|
|
|
Traneptora
|
2022-11-08 08:50:01
|
and there we go
|
|
2022-11-08 08:50:02
|
|
|
2022-11-08 08:52:47
|
the libjxl version has varying thickness
|
|
2022-11-08 08:52:48
|
|
|
2022-11-08 08:52:55
|
I have to figure out the difference there
|
|
|
spider-mario
|
2022-11-08 08:55:41
|
the varying thickness should be a result of interpolating σ throughout the spline with an IDCT
|
|
|
|
Traneptora
|
2022-11-08 08:56:37
|
Yea, I'm applying the same ICT to sigma as I do to the colors
|
|
2022-11-08 08:59:28
|
the formula in the spec is totally wrong, so I had to inspect code from libjxl
|
|
|
|
dds
|
2022-11-08 09:04:11
|
that's not how it's meant to work
|
|
2022-11-08 09:04:24
|
best practice is research code -> spec -> reference implementation
|
|
2022-11-08 09:04:28
|
from everything I've seen, JXL looks like research code -> reference implementation -> spec
|
|
|
|
spider-mario
|
|
Traneptora
the formula in the spec is totally wrong, so I had to inspect code from libjxl
|
|
2022-11-08 09:07:15
|
what was the discrepancy like between the two?
|
|
|
|
Traneptora
|
2022-11-08 09:07:35
|
you're right that I'm getting nearly constant sigma. that's something I'll have to figure out
|
|
2022-11-08 09:07:44
|
but given the correct sigma:
|
|
2022-11-08 09:07:57
|
This is what I have, which matches what's in libjxl
|
|
2022-11-08 09:09:06
|
```
double diffX = x - arcX;
double diffY = y - arcY;
double distance = sqrt(diffX * diffX + diffY * diffY);
double factor = erf(0.5D * (distance + SQRT_H) * inverseSigma);
factor -= erf(0.5D * (distance - SQRT_H) * inverseSigma);
buffer[c][y][x] += 0.25D * values[c] * intensity * sigma * factor * factor;
```
|
|
2022-11-08 09:09:27
|
`SQRT_H` is half the square root of two
|
|
2022-11-08 09:10:42
|
the spec has *this*
```c
s2s = sqrt(2) * sigma * sigma;
(value * intensity * sigma / 4) *
(erf((p.x − x + 0.5) / s2s) − erf((p.x − x − 0.5) / s2s)) *
(erf((p.y − y + 0.5) / s2s) − erf((p.y − y − 0.5) / s2s));
```
|
|
2022-11-08 09:10:50
|
I renamed some variables so they're similar
|
|
2022-11-08 09:10:55
|
but otherwise it's the same
|
|
2022-11-08 09:11:21
|
the spec says you take a difference of erfs in the x direction and the y direction, and adjust the input by a factor of sigma^2
|
|
2022-11-08 09:11:44
|
whereas libjxl calculates the hypot from diffX and diffY
|
|
2022-11-08 09:11:57
|
and does erf in one dimenion, that direction, and squares it
|
|
2022-11-08 09:15:29
|
and, I just figured out the bug with my sigma problems
|
|
2022-11-08 09:15:30
|
|
|
|
|
spider-mario
|
2022-11-08 09:19:19
|
I’m pretty sure libjxl used to do what is in the spec, I wonder when we changed it
|
|
|
|
Traneptora
|
2022-11-08 09:19:44
|
sigma issues were a result of the spec
|
|
2022-11-08 09:19:53
|
> `arclength_from_start = min(1.0, i / arclength);`
> `Sigma = ContinuousIDCT(dctSigma, 31 * arclength_from_start / arclength);`
|
|
2022-11-08 09:19:55
|
this is wrong
|
|
2022-11-08 09:20:30
|
`arclength_from_start` is a fraction in the range of `[0, 1]`
|
|
|
|
spider-mario
|
2022-11-08 09:20:57
|
oh, right, there’s an extraneous `/ arclength`, isn’t there
|
|
|
|
Traneptora
|
|
|
spider-mario
|
2022-11-08 09:21:03
|
thanks
|
|
|
|
Traneptora
|
2022-11-08 09:22:35
|
one of the nice things about doing this is it lets us iron out these spec issues
|
|
2022-11-08 09:22:53
|
although I should sleep at this point
|
|
2022-11-08 09:23:00
|
I'm in a US timezone
|
|
|
|
spider-mario
|
2022-11-08 09:23:24
|
ah, the erf change was in https://github.com/libjxl/libjxl/pull/510
|
|
|
|
Traneptora
|
2022-11-08 09:24:44
|
I do find this a little odd
|
|
2022-11-08 09:24:45
|
> `const auto one_over_2s2 = Set(df, 0.353553391f);`
|
|
2022-11-08 09:24:52
|
it's still in the code and it's not used
|
|
2022-11-08 09:24:56
|
this is just 1/4 of the square root of 2
|
|
2022-11-08 09:25:13
|
(I factored 0.5 out of it and used half of the square root of 2, but otherwise it's the same)
|
|
|
spider-mario
ah, the erf change was in https://github.com/libjxl/libjxl/pull/510
|
|
2022-11-08 09:26:11
|
not so sure, it looks like it was already there
|
|
2022-11-08 09:26:16
|
https://github.com/libjxl/libjxl/pull/510/commits/50e48b96378a0a1387c5cf4da3adb34deddcbd23#diff-a05cd804c62dacf5d95cb0ab752a6d1bd06794ced46273710ba1921af93232bdL123
|
|
2022-11-08 09:26:41
|
search FastErff
|
|
|
|
spider-mario
|
2022-11-08 09:27:03
|
oh, yeah, I misread
|
|
2022-11-08 09:27:07
|
the search continues
|
|
|
|
dds
|
2022-11-08 09:43:28
|
<@604964375924834314> did you have any luck finding those hand-crafted PoC images that showcase the compression benefit of splines?
|
|
|
fab
|
2022-11-08 10:14:45
|
benchmark_xl --input C:\Users\Use\Pictures\q\a.png --output_dir . --codec=jpeg:libjxl:d1.0 --save_compressed
|
|
2022-11-08 10:16:12
|
|
|
2022-11-08 10:16:14
|
it didn't woked
|
|
2022-11-08 10:20:54
|
ah debug build
|
|
|
|
Jyrki Alakuijala
|
|
dds
best practice is research code -> spec -> reference implementation
|
|
2022-11-08 10:21:11
|
we tried things out in code rather than thinking about it -- intuition does not always work with compression systems 🙂
|
|
|
|
fab
|
2022-11-08 10:21:38
|
how to do it
|
|
2022-11-08 10:21:41
|
windows seven
|
|
2022-11-08 10:21:49
|
no waste of time
|
|
2022-11-08 10:22:04
|
i downloaded release build deploy
|
|
2022-11-08 10:25:31
|
if you upload the right build i can test
|
|
2022-11-08 10:31:38
|
|
|
|
|
The_Decryptor
|
2022-11-08 10:33:05
|
ci.sh won't work on Windows, and don't include the leading $
|
|
|
|
fab
|
2022-11-08 10:33:59
|
so how to convert the image
|
|
2022-11-08 10:34:16
|
debug build is needed but i'm on windows seven
|
|
2022-11-08 10:35:10
|
it converted but it did error
|
|
2022-11-08 10:35:48
|
with
|
|
2022-11-08 10:35:50
|
benchmark_xl --input C:\Users\Use\Pictures\q\a.png --output_dir . --codec=jpeg:libjxl:d1.0 --save_compressed
|
|
2022-11-08 10:35:58
|
how to specify thread
|
|
2022-11-08 10:36:10
|
i don't know why it fails
|
|
2022-11-08 10:36:17
|
should it fails on windows seven
|
|
2022-11-08 10:36:24
|
i have release build deploy
|
|
2022-11-08 10:36:28
|
and not debug build
|
|
|
DZgas Ж
|
2022-11-08 10:36:31
|
this is a command completely on linux, I don't understand why no one will write a command for windows, or some kind of instruction
|
|
2022-11-08 10:37:53
|
I don't even want to try to compile a build for windows, it's hard. Where can I get ready-made builds of the latest commit???
|
|
|
|
Jyrki Alakuijala
|
2022-11-08 10:41:24
|
DZgas, would you be open to try the new jpeg encoder? You have good eyes for artefacts 🙂
|
|
2022-11-08 10:41:51
|
$ ./ci.sh opt
$ ./build/tools/benchmark_xl --input input.png --output_dir . --codec=jpeg:libjxl:d1.0 --save_compressed
|
|
|
|
DZgas Ж
|
|
Jyrki Alakuijala
DZgas, would you be open to try the new jpeg encoder? You have good eyes for artefacts 🙂
|
|
2022-11-08 10:43:57
|
NEW means that the code has just been updated on github, it also means that I need to download the latest version from github and build it? No, I'm on windows
|
|
|
|
lonjil
|
2022-11-08 10:45:05
|
So far it definitely seems that for flat color digital art, libjxl creates more noticeable artefacts, because the libjpeg artefacts are mostly high frequency, keeping the average color in the artefacted regions the same as in the original image, while libjxl create lower frequency artefacts with the wrong color. So -d0.5 is worse than q90, and bigger.
|
|
|
|
DZgas Ж
|
|
Jyrki Alakuijala
$ ./ci.sh opt
$ ./build/tools/benchmark_xl --input input.png --output_dir . --codec=jpeg:libjxl:d1.0 --save_compressed
|
|
2022-11-08 10:45:48
|
if you don't have the windows build, or you can't make one, then I won't be able to participate in anything
|
|
2022-11-08 10:51:27
|
because it does not compress the file FROM JPEG, it does transcoding
|
|
|
|
Jyrki Alakuijala
|
|
DZgas Ж
if you don't have the windows build, or you can't make one, then I won't be able to participate in anything
|
|
2022-11-08 10:52:02
|
Sorry -- I don't know how to compile for windows or where to get the binaries -- let's wait until they are available
|
|
|
|
DZgas Ж
|
|
Jyrki Alakuijala
Sorry -- I don't know how to compile for windows or where to get the binaries -- let's wait until they are available
|
|
2022-11-08 10:52:46
|
month? until a new version appears on github?
|
|
|
|
fab
|
2022-11-08 10:53:59
|
Jyrki alakuijala has said you should build a debug build
|
|
|
|
DZgas Ж
|
2022-11-08 10:54:07
|
whole year has passed since versions 0.6.1 and 0.7.0
|
|
|
|
fab
|
|
Jyrki Alakuijala
Sorry -- I don't know how to compile for windows or where to get the binaries -- let's wait until they are available
|
|
2022-11-08 10:54:45
|
Ok
|
|
|
|
DZgas Ж
|
2022-11-08 10:55:37
|
I don't believe that 100% of developers are on linux, someone needs to be on windows and build for windows, I just need to find this someone and ask him to drop the files here
|
|
|
|
dds
|
|
Jyrki Alakuijala
we tried things out in code rather than thinking about it -- intuition does not always work with compression systems 🙂
|
|
2022-11-08 10:56:22
|
I get that you want to be able to do things in parallel. However it was written, I claim there's - demonstrably - a lot of technical debt in the spec. It comes across as a best-efforts description of how the libjxl decoder works rather than a rigourous standard that has survived scrutiny. So where there is conflict, the (semi-public) libjxl codebase always seems to take precedence over the spec.
|
|
|
|
Jyrki Alakuijala
|
2022-11-08 11:00:15
|
good points -- perhaps 5th edition in 2033 will make scrutiny people happy
|
|
|
|
DZgas Ж
|
2022-11-08 11:00:46
|
huh, I AM Found
|
|
2022-11-08 11:00:48
|
https://artifacts.lucaversari.it/libjxl/libjxl/
|
|
|
|
Jyrki Alakuijala
|
2022-11-08 11:00:49
|
what makes libjxl semi-public?
|
|
|
|
DZgas Ж
|
|
DZgas Ж
https://artifacts.lucaversari.it/libjxl/libjxl/
|
|
2022-11-08 11:01:31
|
<@532010383041363969>did you know about this site?
|
|
|
|
Jyrki Alakuijala
|
2022-11-08 11:01:45
|
I thought they are somewhere, but I didn't know where
|
|
2022-11-08 11:02:02
|
I was hoping in github somehow
|
|
|
|
DZgas Ж
|
2022-11-08 11:02:37
|
BRUH
|
|
2022-11-08 11:03:14
|
<@532010383041363969> you must add it to the main jxl github page https://artifacts.lucaversari.it/libjxl/libjxl/latest/
|
|
2022-11-08 11:05:00
|
why I have to find a site where Builds for each commit, on all platforms, Not on the official site
|
|
2022-11-08 11:06:33
|
seriously?
|
|
2022-11-08 11:07:57
|
<@794205442175402004> why is this link not be on Github?
|
|
|
|
veluca
|
2022-11-08 11:08:31
|
You can just get the artefacts from GitHub directly
|
|
2022-11-08 11:08:49
|
Those files are just the GitHub artefacts, downloaded
|
|
|
|
DZgas Ж
|
2022-11-08 11:09:35
|
I want to be Able to download the latest builds.
|
|
|
veluca
You can just get the artefacts from GitHub directly
|
|
2022-11-08 11:10:50
|
post the link and sign its. the main thing is that the link would be at all
|
|
2022-11-08 11:12:31
|
it turns out there is a place where you can download builds for all commits - well, how was I supposed to find out?
|
|
|
|
veluca
|
2022-11-08 11:13:13
|
https://github.com/libjxl/libjxl/actions/workflows/release.yaml
|
|
|
|
DZgas Ж
|
|
veluca
https://github.com/libjxl/libjxl/actions/workflows/release.yaml
|
|
2022-11-08 11:15:43
|
what? I don't understand. and where are the files? what's the use of it?
|
|
|
|
dds
|
|
Jyrki Alakuijala
what makes libjxl semi-public?
|
|
2022-11-08 11:16:27
|
I mean semi-public in the sense that libjxl is open to public contributions (which is fine) yet it's authoritative over the spec (which is weird).
|
|
|
|
DZgas Ж
|
2022-11-08 11:16:55
|
I know that there are sites that check whether the build was done correctly, but they are absolutely useless because they do not place the files themselves
|
|
|
|
dds
|
|
Jyrki Alakuijala
good points -- perhaps 5th edition in 2033 will make scrutiny people happy
|
|
2022-11-08 11:17:24
|
That sounds a bit sarcastic. One of the main issues with the spec (following a previous discussion with _wb_) is that bad inputs appear to be defined implicitly rather than explicitly, i.e. if you can find an inconsistent set of options or something that causes the decoder code to break, overflow or not make sense then your JXL file is invalid. I don't think this is unfair. IMO it's better to state explicitly what is good and what is bad - and better still for bad inputs to be unexpressable where this is practical.
|
|
2022-11-08 11:18:35
|
Scrutiny shouldn't be an annoyance - if you're trying to write a universal standard, a rigourous spec allows for compatibility between implementations. There are many interesting side-alleys in the JXL spec, and problems arise if different implementations don't agree on decoding behaviour.
|
|
|
|
DZgas Ж
|
2022-11-08 11:19:30
|
I see it, I don't understand how I can get a ready build from what I see
|
|
|
|
afed
|
2022-11-08 11:20:08
|
<https://github.com/libjxl/libjxl/actions/runs/3414489033#artifacts>
|
|
|
|
DZgas Ж
|
|
afed
<https://github.com/libjxl/libjxl/actions/runs/3414489033#artifacts>
|
|
2022-11-08 11:20:51
|
But it's not links, it's not buttons, it's text just
|
|
2022-11-08 11:22:02
|
I probably don't want to have a github account
|
|
2022-11-08 11:24:23
|
well, let's go back to my claim - why is there no link to this site on github
|
|
|
|
veluca
|
2022-11-08 11:25:11
|
because it's hosted in my flat and I don't particularly want to make it official and/or promise it will stay around 😉
|
|
|
|
lonjil
|
2022-11-08 11:25:39
|
if anyone wanted to try it, here is the latest build for windows
|
|
|
|
DZgas Ж
|
|
lonjil
if anyone wanted to try it, here is the latest build for windows
|
|
2022-11-08 11:26:40
|
ok. it just https://artifacts.lucaversari.it/libjxl/libjxl/latest/
|
|
|
veluca
because it's hosted in my flat and I don't particularly want to make it official and/or promise it will stay around 😉
|
|
2022-11-08 11:27:17
|
that sounds like an argument
|
|
|
Jyrki Alakuijala
$ ./ci.sh opt
$ ./build/tools/benchmark_xl --input input.png --output_dir . --codec=jpeg:libjxl:d1.0 --save_compressed
|
|
2022-11-08 11:29:40
|
so
|
|
2022-11-08 11:30:48
|
D:\a\libjxl\libjxl\tools\benchmark\benchmark_xl.cc:295: JXL_CHECK: WriteFile(compressed_str, compressed_fn)
|
|
2022-11-08 11:31:00
|
exactly the same bug with paths
|
|
2022-11-08 11:31:59
|
I have no idea what this 👉 D:\a\libjxl\libjxl\
|
|
2022-11-08 11:33:08
|
--save_compressed same bug
|
|
2022-11-08 11:34:25
|
I don't understand it. the program just doesn't work
|
|
2022-11-08 11:35:32
|
I put it in the root of the C drive, and I run it from there, and this is written , who and why prescribed absolute paths for their system in the code?
|
|
2022-11-08 11:38:25
|
And I can't output path
|
|
2022-11-08 11:38:43
|
maybe
|
|
|
|
The_Decryptor
|
2022-11-08 11:40:31
|
`benchmark_xl.exe --input input.png --codec=jpeg:libjxl:d1.0 --save_compressed --output_dir=.` This is the command I use and it works (With the github binaries)
|
|
|
|
DZgas Ж
|
2022-11-08 11:43:34
|
The error was in another, I found
|
|
2022-11-08 11:44:22
|
The program does not understand the Windows path at all
|
|
2022-11-08 11:44:33
|
like "C:\Users\a\Desktop\13.png"
|
|
2022-11-08 11:45:09
|
To make everything work, you need to use the terminal to go to the folder with the program, and put a photo in the same folder
|
|
2022-11-08 11:45:32
|
and only then --input 13.png will work
|
|
2022-11-08 11:46:46
|
O kay, what i see?
|
|
2022-11-08 11:47:20
|
|
|
2022-11-08 11:47:55
|
the incoming file was a white sheet
|
|
|
Jyrki Alakuijala
DZgas, would you be open to try the new jpeg encoder? You have good eyes for artefacts 🙂
|
|
2022-11-08 11:48:53
|
?
|
|
|
|
The_Decryptor
|
2022-11-08 11:49:32
|
It produces XYB JPEGs, you need working colour management to view them properly
|
|
|
|
DZgas Ж
|
|
|
_wb_
|
|
Jyrki Alakuijala
good points -- perhaps 5th edition in 2033 will make scrutiny people happy
|
|
2022-11-08 11:51:13
|
I think the 2nd edition should already be good enough to do an implementation from spec without consulting libjxl, even if some things might still a bit unclear in it and require test bitstreams to confirm things.
|
|
|
dds
That sounds a bit sarcastic. One of the main issues with the spec (following a previous discussion with _wb_) is that bad inputs appear to be defined implicitly rather than explicitly, i.e. if you can find an inconsistent set of options or something that causes the decoder code to break, overflow or not make sense then your JXL file is invalid. I don't think this is unfair. IMO it's better to state explicitly what is good and what is bad - and better still for bad inputs to be unexpressable where this is practical.
|
|
2022-11-08 11:55:49
|
It is explicit about what is invalid, it's just saying it by saying things like "this value is in that range" instead of explicitly saying that out-of-range values are invalid. The reason is that the spec only defines what a conforming decoder has to do with conforming bitstreams, it does not specify how to handle bad bitstreams — it can throw error, do a best-effort decode anyway, or explode, it is not specified.
|
|
|
|
DZgas Ж
|
|
DZgas Ж
|
|
2022-11-08 11:56:26
|
I also want to note that depending on what you are looking at the picture through, its color is different
|
|
2022-11-08 11:56:36
|
|
|
2022-11-08 11:56:46
|
|
|
2022-11-08 11:57:46
|
|
|
|
|
The_Decryptor
|
2022-11-08 12:00:53
|
Found a forum post from 2010 asking for colour management support in Paint.NET, answer was only professionals needed it, seems that opinion is unchanged
|
|
|
|
DZgas Ж
|
|
The_Decryptor
Found a forum post from 2010 asking for colour management support in Paint.NET, answer was only professionals needed it, seems that opinion is unchanged
|
|
2022-11-08 12:10:44
|
viewing windows photos same opinion And the Discord preview
|
|
2022-11-08 12:11:35
|
and I also don't understand why any color profiles should be stored inside the photo.
|
|
2022-11-08 12:12:07
|
it's good that jpeg XL will kill it
|
|
|
|
dds
|
|
_wb_
It is explicit about what is invalid, it's just saying it by saying things like "this value is in that range" instead of explicitly saying that out-of-range values are invalid. The reason is that the spec only defines what a conforming decoder has to do with conforming bitstreams, it does not specify how to handle bad bitstreams — it can throw error, do a best-effort decode anyway, or explode, it is not specified.
|
|
2022-11-08 12:19:53
|
We've had this conversation before - I don't think we're going to agree 😕 IMO the large number of specification issues people are finding is a strong argument for more rigour in the spec. e.g. the issue with behaviour of out-of-range properties that <@853026420792360980> found a couple of days ago. This kind of thing should be explicit IMO.
|
|
|
yurume
|
2022-11-08 12:27:45
|
I don't know if this is a huge burden for editors or not, but one thing I personally want is a clear label for all properties defined in the specification. as currently is the spec is full of prose definitions, which *is* okay by its own but harder to decompose into programmable chunks or individual tests.
|
|
2022-11-08 12:28:46
|
in some cases a single sentence defines multiple properties, not beknownst to even editors, and I think it is definitely bad
|
|
|
|
dds
|
2022-11-08 12:33:31
|
<@532010383041363969> <@794205442175402004> in case it needs to be said - I'm not here to hate on JXL. I really want it to succeed but I'm concerned that not enough people have tried throwing rocks at it. IMO it's much better that this happens sooner rather than later.
|
|
|
|
lonjil
|
2022-11-08 12:37:56
|
In the long term, the standard won't matter because as soon as JXL is even slightly popular, handing out illicit copies of the standard won't work anymore /s
|
|
|
|
fab
|
2022-11-08 01:05:19
|
done
|
|
2022-11-08 01:05:22
|
benchmark_xl --input a.png --output_dir . --codec=jpeg:libjxl:d1.0 --save_compressed
|
|
|
|
Jyrki Alakuijala
|
2022-11-08 01:16:21
|
Green is the New White.
|
|
|
dds
We've had this conversation before - I don't think we're going to agree 😕 IMO the large number of specification issues people are finding is a strong argument for more rigour in the spec. e.g. the issue with behaviour of out-of-range properties that <@853026420792360980> found a couple of days ago. This kind of thing should be explicit IMO.
|
|
2022-11-08 01:20:51
|
what is your recommendation now with the spec? for our team libjxl 1.0 and wasm are the highest priority
|
|
|
|
DZgas Ж
|
|
DZgas Ж
|
|
2022-11-08 02:32:57
|
ffmpeg -i 13.png.d1.0.libjxl.jpeg out.jpeg
|
|
2022-11-08 02:34:01
|
very funny
|
|
|
|
fab
|
|
|
_wb_
|
|
dds
<@532010383041363969> <@794205442175402004> in case it needs to be said - I'm not here to hate on JXL. I really want it to succeed but I'm concerned that not enough people have tried throwing rocks at it. IMO it's much better that this happens sooner rather than later.
|
|
2022-11-08 02:55:50
|
I agree the spec needs as much scrutiny as possible so we can make the 2nd edition as correct and clear as possible. But it is also a reality that libjxl will define the de facto standard and even e.g. hardware implementors will probably in the first place cross-check against libjxl and use the spec more as documentation than as an authoritive thing, even if that's not how things are supposed to be. With jpeg-1, the same thing happened, with the key difference that ijg-libjpeg didn't even implement much of the spec.
|
|
|
|
dds
|
2022-11-08 03:46:42
|
what is your recommendation now with the
|
|
|
|
Jyrki Alakuijala
|
|
DZgas Ж
?
|
|
2022-11-08 04:09:01
|
I think you pointed out quite a few JPEG XL artefacts on encode.su -- some I had missed previously
Even today we could do some parts better, related to high intensity HF masking of colors too much
|
|
|
|
DZgas Ж
|
|
Jyrki Alakuijala
I think you pointed out quite a few JPEG XL artefacts on encode.su -- some I had missed previously
Even today we could do some parts better, related to high intensity HF masking of colors too much
|
|
2022-11-08 04:10:05
|
a, thank you! I really like Jpeg XL and I want it to be better
|
|
|
|
Jyrki Alakuijala
|
|
dds
That sounds a bit sarcastic. One of the main issues with the spec (following a previous discussion with _wb_) is that bad inputs appear to be defined implicitly rather than explicitly, i.e. if you can find an inconsistent set of options or something that causes the decoder code to break, overflow or not make sense then your JXL file is invalid. I don't think this is unfair. IMO it's better to state explicitly what is good and what is bad - and better still for bad inputs to be unexpressable where this is practical.
|
|
2022-11-08 04:12:38
|
It has been a delicate balance of engineering, spec writing, testing setups, security engineering, inventions, and research -- in our effort spec writing was not initially the number one priority but we are slowly recovering from that
|
|
|
|
DZgas Ж
|
|
Jyrki Alakuijala
I think you pointed out quite a few JPEG XL artefacts on encode.su -- some I had missed previously
Even today we could do some parts better, related to high intensity HF masking of colors too much
|
|
2022-11-08 04:14:46
|
As of today, I still don't understand how I can see the boundaries of the VarDCT blocks
|
|
2022-11-08 04:28:14
|
<@416586441058025472>How __screenshot of powershell__ can allow me to see the VarDCT
|
|
|
|
fab
|
2022-11-08 04:29:33
|
https://discord.com/channels/794206087879852103/794206087879852106/1038859866032312331
|
|
2022-11-08 04:30:20
|
Try same procedure but copy all exes in the folder with images
|
|
2022-11-08 04:30:29
|
All 21 files from actions
|
|
|
|
DZgas Ж
|
|
fab
Try same procedure but copy all exes in the folder with images
|
|
2022-11-08 04:32:21
|
the picture is already in the folder with all the files
|
|
|
|
fab
|
2022-11-08 04:33:29
|
You should have ImageMagick
|
|
2022-11-08 04:33:39
|
To do the ppm image
|
|
2022-11-08 04:34:06
|
Don't know if it works with release deploy on Windows 10
|
|
|
|
DZgas Ж
|
2022-11-08 04:37:48
|
```convert $tmp/orig \( $tmp/orig.jxl:d$i.dbg/ac_strategy.png -alpha set -channel A -evaluate set 66% \) -composite $tmp/blockselections.ppm```
|
|
2022-11-08 04:37:58
|
linux command
|
|
|
fab
You should have ImageMagick
|
|
2022-11-08 05:04:26
|
I have it, what's next?
|
|
|
DZgas Ж
```convert $tmp/orig \( $tmp/orig.jxl:d$i.dbg/ac_strategy.png -alpha set -channel A -evaluate set 66% \) -composite $tmp/blockselections.ppm```
|
|
2022-11-08 05:05:22
|
it's completely unclear what, I don't understand what and where is going on here, what are the input and output parameters
|
|
2022-11-08 05:39:01
|
<:AngryCry:805396146322145301>
|
|
|
|
Traneptora
|
|
dds
We've had this conversation before - I don't think we're going to agree 😕 IMO the large number of specification issues people are finding is a strong argument for more rigour in the spec. e.g. the issue with behaviour of out-of-range properties that <@853026420792360980> found a couple of days ago. This kind of thing should be explicit IMO.
|
|
2022-11-08 05:52:49
|
yea, it is a strong argument for more rigor in the spec, but that's what we're doing :)
|
|
2022-11-08 05:53:13
|
problem with coming up with a spec before a library is that there's a very big difference between trying to review code and trying to execute it
|
|
|
|
DZgas Ж
|
|
DZgas Ж
|
|
2022-11-08 05:59:49
|
|
|
|
DZgas Ж
|
|
2022-11-08 06:41:54
|
|
|
|
|
lonjil
|
2022-11-08 07:39:19
|
Interesting, my copy of Firefox decodes it as green.
|
|
|
daniilmaks
|
|
lonjil
Interesting, my copy of Firefox decodes it as green.
|
|
2022-11-08 07:43:25
|
color management in firefox was under a flag at some point then I think became default, but I can't recall if that was for v2 or v4 profiles
|
|
|
|
lonjil
|
2022-11-08 07:45:50
|
So wait, it might be decoding every image with an attached profile as whatever random color space it thinks that that format is always used with? <:raritywhy:590982864523362335>
|
|
|
|
daniilmaks
|
2022-11-08 07:46:20
|
<@226977230121598977> xyb jpegs use v4 right?
|
|
|
|
lonjil
|
2022-11-08 07:46:51
|
I believe XYB requires v4, yeah.
|
|
|
|
DZgas Ж
|
2022-11-08 07:46:59
|
|
|
2022-11-08 07:47:16
|
_Per?
|
|
|
|
daniilmaks
|
|
lonjil
I believe XYB requires v4, yeah.
|
|
2022-11-08 07:47:23
|
that might not be default yet, it should be under a flag
|
|
|
lonjil
So wait, it might be decoding every image with an attached profile as whatever random color space it thinks that that format is always used with? <:raritywhy:590982864523362335>
|
|
2022-11-08 07:49:21
|
if the browser doesn't have color management it literally cannot see or use color profiles.
|
|
|
|
lonjil
|
|
|
daniilmaks
|
2022-11-08 07:50:43
|
oh I see what you mean
|
|
2022-11-08 07:51:13
|
it's worse actually
|
|
2022-11-08 07:53:10
|
without color management the browser doesn't even assume a color space for the image, it just maps them 1:1 to whatever color space the monitor is using
i.e. it will look different in different gamut displays
|
|
|
|
lonjil
|
2022-11-08 07:53:48
|
oh. yeah that makes sense.
|
|
|
|
daniilmaks
|
2022-11-08 07:54:59
|
if you load an srgb image in a non-managed browser on a bt2020 display the image will show severely oversaturated and hue shifted for example
|
|
|
|
lonjil
|
2022-11-08 07:58:17
|
Unrelated, but does anyone have an up to date copy of the spec?
|
|
2022-11-08 07:58:43
|
I want to torture myself by trying to create a third alternative implementation.
|
|
|
|
_wb_
|
2022-11-08 08:08:21
|
<:BlobYay:806132268186861619>
|
|
2022-11-08 08:09:29
|
At this rate, soon we'll have more jxl decoders than av1 ones 😅
|
|
2022-11-08 08:10:12
|
There is a relatively recent draft if you scroll up enough in <#1021189485960114198>
|
|
2022-11-08 08:11:35
|
But we fixed a few more things since, I will perhaps post an updated version there later tonight
|
|
|
|
lonjil
|
2022-11-08 08:58:17
|
does anyone have any nice and simple sample images for me to poke at?
|
|
|
|
_wb_
|
2022-11-08 09:14:48
|
you can make simple modular images using jxlart
|
|
2022-11-08 09:15:14
|
for vardct the simplest ones are -e 3 encoded ones
|
|
2022-11-08 09:16:00
|
current draft spec, btw:
|
|
|
|
lonjil
|
2022-11-08 09:17:26
|
nice
|
|
2022-11-08 09:17:28
|
Is that available in pdf?
|
|
|
|
_wb_
|
2022-11-08 09:29:47
|
the pdf backend of my metanorma setup seems to not work properly, I can produce a Word file though and print it to PDF, I guess...
|
|
|
|
lonjil
|
2022-11-08 09:30:16
|
ah, don't bother then 😄
|
|
|
|
_wb_
|
2022-11-08 09:34:16
|
I guess you can also just print the html to pdf
|
|
2022-11-08 09:35:29
|
|
|
2022-11-08 09:42:45
|
this is the pdf I get by printing the word version:
|
|
|
|
Traneptora
|
|
lonjil
does anyone have any nice and simple sample images for me to poke at?
|
|
2022-11-08 09:49:00
|
I tested by grabbing images from `jxl-art` to start out with
|
|
2022-11-08 09:49:05
|
since they're all guaranteed to be modular mode
|
|
|
Shuga
|
2022-11-08 11:36:25
|
Hi, I've been trying to find out how the structure of a prediction-tree-only JXL works, and more notably, what each command looks like when 'compiled' into the image. Is there documentation on what bytes do what and/or how the prediction tree is stored inside the image?
|
|
|
|
_wb_
|
2022-11-08 11:40:56
|
This is described in the spec in H.4.2
|
|
2022-11-09 12:12:04
|
here is a concrete example:
|
|
|
|
Shuga
|
|
_wb_
here is a concrete example:
|
|
2022-11-09 12:51:42
|
Thank you.
|
|
|
|
lonjil
|
2022-11-09 01:26:09
|
Aw yeah, I am now officially reading the signature correctly
|
|
2022-11-09 01:53:44
|
width and height have been conquered...
|
|
2022-11-09 01:54:02
|
but now I should probably go to bed
|
|
|
|
DZgas Ж
|
|
afed
```Error in jxl:d1 codec
tools\benchmark\benchmark_xl.cc:143: JXL_CHECK: speed_stats.GetSummary(&summary)```
```lib\jxl\dec_external_image.cc:260: JXL_CHECK: float_out ? bits_per_sample == 16 || bits_per_sample == 32 : bits_per_sample > 0 && bits_per_sample <= 16```
|
|
2022-11-09 11:50:59
|
all right
|
|
|
|
Traneptora
|
2022-11-09 12:21:51
|
apparently I hit 6.5k LOC in jxlatte without even noticing
|
|
2022-11-09 12:21:58
|
I'm spending way too much time on this <:KEKW:643601031040729099>
|
|
|
|
lonjil
|
|
|
Traneptora
|
2022-11-09 01:29:04
|
I also updated my readme to keep track of features
|
|
2022-11-09 01:29:04
|
https://github.com/thebombzen/jxlatte/blob/main/README.md
|
|
|
|
spider-mario
|
2022-11-09 02:11:53
|
I hadn’t realised that meson could do java
|
|
|
|
Traneptora
|
2022-11-09 02:15:01
|
it's apparently "experimental support"
|
|
2022-11-09 02:15:09
|
but for a very simple project like mine it works fine
|
|
2022-11-09 02:15:23
|
I have no dependencies
|
|
|
|
lonjil
|
2022-11-09 02:15:48
|
damn I'm like 20 lines of code into mine and I already have a dep
|
|
|
|
Traneptora
|
2022-11-09 02:15:57
|
what are you writing it in?
|
|
|
|
lonjil
|
|
|
Traneptora
|
2022-11-09 02:16:10
|
ah, and what dependency did you introduce?
|
|
|
|
lonjil
|
2022-11-09 02:16:37
|
f16 library
|
|
|
|
Traneptora
|
2022-11-09 02:16:52
|
I just homerolled f16 reader
|
|
2022-11-09 02:17:01
|
f16 -> f32
|
|
2022-11-09 02:17:28
|
if it's lightweight I don't see the issue
|
|
2022-11-09 02:17:39
|
but idk if you need to pull in an entire f16 crate
|
|
|
|
lonjil
|
2022-11-09 02:17:49
|
yeah idk
|
|
|
|
Traneptora
|
2022-11-09 02:17:53
|
unless you're planning on doing f16 arithmetic as well
|
|
|
|
lonjil
|
2022-11-09 02:18:00
|
I'll look at it closer once I'm actually reading f16s
|
|
|
|
Traneptora
|
2022-11-09 02:18:29
|
I actually do a lot of arithmetic in double cause java stdlib doesn't have float versions of many functions like pow()
|
|
|
|
lonjil
|
|
|
Traneptora
|
2022-11-09 02:18:52
|
so like I have to do gamma correction in double precision for example
|
|
2022-11-09 02:19:01
|
which is unnecessary, but I don't want to brew my own pow function
|
|
|
Jim
|
2022-11-09 02:23:57
|
https://tenor.com/view/pow-bam-mic-drop-truth-onomatopoeia-gif-14656419
|
|
|
|
Traneptora
|
2022-11-09 02:24:36
|
I already had to brew my own erf via the art of google "erf implementation"
|
|
2022-11-09 06:50:11
|
well that's odd
|
|
2022-11-09 06:51:06
|
Input:
|
|
2022-11-09 06:51:26
|
djxl output:
|
|
2022-11-09 06:51:37
|
libjxl appears to be working fine though
|
|
2022-11-09 06:51:49
|
i.e. the pixbuf loader, ffmpeg, and other software that links to libjxl reads the image normally
|
|
2022-11-09 06:52:40
|
jxlinfo output
|
|
2022-11-09 06:52:42
|
```
$ jxlinfo -v samples/wb-rainbow.jxl
JPEG XL image, 2048x1152, (possibly) lossless, 9-bit RGBA
num_color_channels: 3
num_extra_channels: 1
extra channel 0:
type: Alpha
bits_per_sample: 9
alpha_premultiplied: 0 (Non-premultiplied)
have_preview: 0
have_animation: 0
Intrinsic dimensions: 2048x1152
Orientation: 1 (Normal)
Color space: RGB, D65, sRGB primaries, sRGB transfer function, rendering intent: Relative
layer: full image size
layer: 1024x576 at position (512,164)
layer: 1024x576 at position (512,640)
layer: 1024x576 at position (512,164)
layer: 1024x576 at position (1860,950)
```
|
|
2022-11-09 06:53:21
|
I wonder if djxl is getting extremely confused because `bits_per_sample=9` is signalled in the header
|
|
2022-11-09 06:53:50
|
the image decodes fine when 16 bits is requested in libjxl
|
|
2022-11-09 06:53:56
|
and JXLatte can also decode this image and it looks like it should
|
|
|
|
_wb_
|
2022-11-09 06:54:25
|
Yeah djxl bug in the png writing very likely
|
|
|
|
Traneptora
|
2022-11-09 06:55:35
|
my own PNG writer is very primitive
|
|
2022-11-09 06:56:01
|
just takes the samples, multiplies them by 255, clamps between [0, 255]
|
|
2022-11-09 06:56:16
|
only supports filter 0 i.e. no filtering
|
|
|
|
lonjil
|
|
lonjil
I'll look at it closer once I'm actually reading f16s
|
|
2022-11-09 07:21:28
|
first use of it now, and I think the values look kinda bogus
|
|
2022-11-09 07:21:51
|
Intensity targets usually wouldn't be negative, right?
|
|
2022-11-09 07:23:13
|
ah wait no might've read too many bits in an earlier function...
|
|
|
|
Traneptora
|
|
lonjil
Intensity targets usually wouldn't be negative, right?
|
|
2022-11-09 07:39:48
|
that's possibly an alignment issue
|
|
2022-11-09 07:40:05
|
I discovered that values being wrong often means you read the wrong number of bits much earlier
|
|
|
|
lonjil
|
|
|
Traneptora
|
2022-11-09 07:40:53
|
for example, in this case most images have AllDefault set for their tone mapping bundle
|
|
|
|
lonjil
|
2022-11-09 07:41:00
|
at the correct alignment, all_default read true, so I won't get to yet know whether the crate works right ^\_^
|
|
2022-11-09 07:41:15
|
damn u ninja'd me
|
|
|
|
Traneptora
|
2022-11-09 07:41:45
|
once you decode an XYB modular image you'll figure it out
|
|
2022-11-09 07:42:03
|
since there's LFQuant bundle which is frequently present for modular lossy images created with cjxl
|
|
|
|
lonjil
|
2022-11-09 08:02:17
|
🥳 ```
jxl-rs % ./target/debug/rdjxl art.jxl
2815
h: 1080, w: 1920
all_default: false
extra_fields: true
orientation: LeftTop
have_intr_size: false
have_preview: false
have_animation: false
bit depth: Int { bits: 10 }
modular 16 bit buffers: true
num_extra_channels: 0
xyz_encoded: false
ColorEncoding, all_default: true
color_encoding: Some(ColorEncoding { want_icc: false, color_space: KRGB, white_point: KD65, white: None, primaries: KSRGB, custom_rgb: None, tf: TF(KSRGB), rendering_intent: KRelative })
ToneMapping, all_default: true
tone_mapping: Some(ToneMapping { intensity_target: 255.0, min_nits: 0.0, relative_to_max_display: false, linear_below: 0.0 })
extensions: 0
default_m: true
cw_mask: 0
```
|
|
2022-11-09 08:19:46
|
when a field whose type is a bundle has a condition, should I interpret the false condition case as "this bundle will not be needed" or "please synthesize this bundle from default values for later use"?
|
|
|
|
veluca
|
2022-11-09 08:27:25
|
I think the second one is the safest choice
|
|
2022-11-09 08:27:56
|
(btw, <@167023260574154752> are you aware of https://github.com/libjxl/jxl-rs ? feel free to take that repo over and/or copy stuff from there)
|
|
|
|
lonjil
|
2022-11-09 08:28:15
|
I was not aware of that, no.
|
|
|
|
veluca
|
2022-11-09 08:28:39
|
it's something I started a while back and then left there
|
|
2022-11-09 08:30:06
|
probably the most interesting thing in it is the proc macro magic to write headers in a declarative way
|
|
2022-11-09 08:30:36
|
but IIRC I managed to get started on entropy decoding too
|
|
|
|
lonjil
|
2022-11-09 08:32:15
|
I have a few macro_rules! macros that lets me at least read non-bundle fields with code that looks identical to the field definitions, but it isn't really declarative.
|
|
2022-11-09 08:34:46
|
proc macro to make it all declarative sounds nice
|
|
2022-11-09 08:35:53
|
yeah, that looks complicated 🤔
|
|
2022-11-09 08:36:51
|
though, apparently with a bit more functionality than I would've implemented
> fn texify(
|
|
|
|
Traneptora
|
2022-11-10 12:03:53
|
problem with noise synthesis I'm finding is that I'm not sure if I'm generating the noise properly
|
|
2022-11-10 12:03:58
|
but it's really hard to subtract and compare
|
|
2022-11-10 12:04:03
|
cause I see noise
|
|
2022-11-10 12:04:04
|
and it's like
|
|
2022-11-10 12:04:06
|
ok ._.
|
|
2022-11-10 12:06:53
|
I'm within the tolerance, since the image is tagged as 9bpp and I'm getting `-10.28261246` mse
|
|
2022-11-10 12:06:56
|
in the log scale
|
|
2022-11-10 12:08:29
|
oh wait I lied it's rmse tolerance, not mse, I'm not in it at all <:kek:857018203640561677>
|
|
|
lonjil
when a field whose type is a bundle has a condition, should I interpret the false condition case as "this bundle will not be needed" or "please synthesize this bundle from default values for later use"?
|
|
2022-11-10 12:10:05
|
frequently the default values are used
one such example is the LFChannelCorrelation bundle, which is not present if a frame is encoded with modular
but it's used for spline and noise rendering, which are permitted in modular
|
|
2022-11-10 12:10:19
|
in that case its default values are used
|
|
|
|
lonjil
|
2022-11-10 12:10:30
|
I see I see
|
|
2022-11-10 12:11:08
|
In that case I will do the simple thing for now and simply generate default structs for everything and use the condition to control whether to update the values away from the defaults.
|
|
|
|
veluca
|
|
lonjil
though, apparently with a bit more functionality than I would've implemented
> fn texify(
|
|
2022-11-10 12:59:41
|
yeah you don't really need that 😄
|
|
2022-11-10 01:00:02
|
(that was a spin-off from me trying to use the rust impl for a new edition of the spec)
|
|
|
|
DZgas Ж
|
2022-11-10 04:59:36
|
I have a big question.
|
|
2022-11-10 05:00:30
|
cheetach IS NOT -e 4 ???
|
|
2022-11-10 05:02:22
|
when I do tests through benchmark_xl I have blocks in ac_strategy.png, but when I compress the cjxl image I have everything 8x8
|
|
2022-11-10 05:03:12
|
<@179701849576833024>
|
|
|
|
veluca
|
2022-11-10 05:10:37
|
I don't remember how speed animals and effort numbers are mapped, but rest assured it's *not* how you'd expect
|
|
|
|
DZgas Ж
|
2022-11-10 05:14:33
|
<:SadCheems:890866831047417898>
|
|
|
veluca
I don't remember how speed animals and effort numbers are mapped, but rest assured it's *not* how you'd expect
|
|
2022-11-10 05:32:49
|
Well how ./tools/benchmark/benchmark_codec.cc:52: JXL_ABORT: Invalid parameter e4
|
|
|
|
ayumi
|
|
DZgas Ж
Well how ./tools/benchmark/benchmark_codec.cc:52: JXL_ABORT: Invalid parameter e4
|
|
2022-11-10 05:50:27
|
AFAIK `benchmark_xl` supports only a subset of `cjxl` flags and `-e` is not in this subset.
|
|
|
|
DZgas Ж
|
|
ayumi
AFAIK `benchmark_xl` supports only a subset of `cjxl` flags and `-e` is not in this subset.
|
|
2022-11-10 05:54:51
|
but I can't use cheetah instead of number
|
|
2022-11-10 05:54:57
|
in cjxl
|
|
|
|
ayumi
|
2022-11-10 06:08:55
|
Yes, `cjxl` only understands the effort numbers. According to the `libjxl` encoder API documentation this is the mapping between effort numbers and speed names:
|
|
2022-11-10 06:08:58
|
|
|
|
|
Traneptora
|
2022-11-10 06:08:58
|
ideally you just use a number instead
|
|
2022-11-10 06:09:17
|
also noise synthesis is implemented in jxlatte, which now is all the Features done :)
|
|
2022-11-10 06:09:31
|
although it doesn't match either the spec or libjxl as both have bugs, hoping both get fixed
|
|
|
|
DZgas Ж
|
|
DZgas Ж
I have a big question.
|
|
2022-11-10 06:10:23
|
I have a problem that the names are written in the tests, the digit is written in the encoding, they seem to be identical, but I get a DIFFERENT result
|
|
|
|
Traneptora
|
2022-11-10 06:17:19
|
I noticed that I'm getting rmse error in the order of `1.7e-5` for lossy modular
|
|
2022-11-10 06:17:30
|
this error is evenly distributed across the whole image
|
|
2022-11-10 06:17:38
|
is this possibly error in the linear -> sRGB transfer?
|
|
2022-11-10 06:19:14
|
libjxl has > `// Linear to sRGB conversion with error of at most 1.2e-4.`
|
|
2022-11-10 06:19:45
|
so if I'm using formulaic double-precision linear->sRGB, would that be a source of the error?
|
|
2022-11-10 06:21:03
|
imagemagick shows that the error is almost uniformly distributed across all color channels and image locations
|
|
2022-11-10 06:26:51
|
I'm well within the tolerance, I'm just trying to figure out where it's coming from
|
|
|
|
_wb_
|
|
DZgas Ж
Well how ./tools/benchmark/benchmark_codec.cc:52: JXL_ABORT: Invalid parameter e4
|
|
2022-11-10 07:38:57
|
You have to write just 4 without the e
|
|
|
|
DZgas Ж
|
|
|
_wb_
|
|
Traneptora
so if I'm using formulaic double-precision linear->sRGB, would that be a source of the error?
|
|
2022-11-10 07:40:24
|
Yes, very likely. That's why we have tolerances 🙂
|
|
|
|
DZgas Ж
|
|
_wb_
You have to write just 4 without the e
|
|
2022-11-10 07:45:05
|
<:AngryCry:805396146322145301>
|
|
2022-11-10 07:45:18
|
you have a bug
|
|
2022-11-10 07:45:40
|
name cheetach is e 5 in benchmark_xl
|
|
|
|
Traneptora
|
|
_wb_
Yes, very likely. That's why we have tolerances 🙂
|
|
2022-11-10 07:46:56
|
although after quantizing to 0-255 8bit, wouldn't that in theory be coarse enough that it should go away?
|
|
|
|
_wb_
|
2022-11-10 07:47:49
|
What's the peak abs error?
|
|
|
|
Traneptora
|
2022-11-10 07:53:33
|
imagemagick says 257, which is very odd considering it's an 8-bit PNG
|
|
2022-11-10 07:54:59
|
doing absolute error shows there's 6 pixels different in the red channel, 3 in the green channel, and 6 in the blue channel
|
|
2022-11-10 07:55:07
|
but I'm trying to figure out why imagemagick is saying 257 for pae
|
|
2022-11-10 07:56:05
|
although that might be 257 in 16-bit space
|
|
2022-11-10 07:56:14
|
which corresponds to a difference of 1 pixel in 8-bit space
|
|
|
|
_wb_
|
2022-11-10 08:02:08
|
Imagemagick works in 16-bit in default compiles, so 257 is an off-by-one in 8-bit
|
|
|
|
Traneptora
|
2022-11-10 08:03:00
|
right, cause `255 * 257 = 65535`
|
|
|
|
_wb_
|
2022-11-10 08:03:31
|
Which is the kind of error I would expect from doing the xyb conversion not super accurately in libjxl
|
|
|
|
Traneptora
|
2022-11-10 08:03:58
|
I see, so that's well within the tolerance and not matching libjxl doesn't actually mean it's incorrect
|
|
|
|
_wb_
|
2022-11-10 08:04:17
|
Yep, in this case you are more correct than libjxl
|
|
|
|
Traneptora
|
2022-11-10 08:04:55
|
I'm also about 7 times slower
|
|
2022-11-10 08:05:03
|
acceptible trade-off tbh on libjxl's part
|
|
2022-11-10 08:05:45
|
the image in question has one group, so I can't effectively parallelize either
|
|
|
|
_wb_
|
2022-11-10 08:05:49
|
It's exactly because we want to allow such trade-offs that we define the spec only with mathematical real numbers and have tolerances
|
|
|
|
Traneptora
|
2022-11-10 08:06:10
|
whereas libjxl can still highway fine
|
|
|
|
_wb_
|
2022-11-10 08:06:23
|
For lossless RGB the tolerances are low enough to ensure it is actually lossless up to the bit depth needed
|
|
|
|
Traneptora
|
2022-11-10 08:06:32
|
I should parallelize inverse squeeze tbh
|
|
2022-11-10 08:07:11
|
the screenshot you linked from 18181-3 doesn't provide tolerance either way for lossy modular
|
|
2022-11-10 08:07:26
|
it looks like just lossless X-bit RGB modular, and then VarDCT
|
|
|
|
_wb_
|
2022-11-10 08:09:05
|
That's just an overview table for the main cases, I suppose we do have a lossy xyb modular case in the actual conformance set. The table is only informative, the json files have the normative tolerances per case.
|
|
|
|
Traneptora
|
2022-11-10 08:09:49
|
I'd have to see 18181-3 fully but I don't have it atm, either way, I'll just assume that if it looks the same in XYB space it's correct 😅
|
|
|
|
_wb_
|
2022-11-10 08:18:29
|
For modular there are two things that can cause inaccuracy: xyb and frame/patch blending. Libjxl does all blending with floats, if you do it with doubles it will be very slightly different (not enough to change the integer values after rounding, but enough to produce numbers that are not exact integers)
|
|
|
|
Traneptora
|
|
_wb_
Yeah djxl bug in the png writing very likely
|
|
2022-11-10 08:20:27
|
I opened an issue about this, btw
https://github.com/libjxl/libjxl/issues/1889
|
|
|
|
ayumi
|
|
DZgas Ж
name cheetach is e 5 in benchmark_xl
|
|
2022-11-10 08:35:48
|
So `benchmark_xl` seems to use (https://github.com/libjxl/libjxl/blob/main/tools/benchmark/benchmark_codec_jxl.cc#L142) `jxl::ParseSpeedTier(const std::string&, SpeedTier*)` (https://github.com/libjxl/libjxl/blob/main/lib/jxl/enc_params.h#L51) to parse speed/effort. If the first parameter is `"cheetach"` it will assign `SpeedTier::kCheetah` to the second parameter. If the first parameter is `"4"` it will convert it to an integer, subtract it from 10, check if the result (`6`) is between `SpeedTier::kLightning` and `SpeedTier::kTortoise` and, if it is, assign `SpeedTier(6)` to the second parameter. Since `SpeedTier(6)` is `SpeedTier:kCheetah` I do not see any obvious problems here.
|
|
|
|
DZgas Ж
|
|
ayumi
So `benchmark_xl` seems to use (https://github.com/libjxl/libjxl/blob/main/tools/benchmark/benchmark_codec_jxl.cc#L142) `jxl::ParseSpeedTier(const std::string&, SpeedTier*)` (https://github.com/libjxl/libjxl/blob/main/lib/jxl/enc_params.h#L51) to parse speed/effort. If the first parameter is `"cheetach"` it will assign `SpeedTier::kCheetah` to the second parameter. If the first parameter is `"4"` it will convert it to an integer, subtract it from 10, check if the result (`6`) is between `SpeedTier::kLightning` and `SpeedTier::kTortoise` and, if it is, assign `SpeedTier(6)` to the second parameter. Since `SpeedTier(6)` is `SpeedTier:kCheetah` I do not see any obvious problems here.
|
|
2022-11-10 09:40:18
|
I would have noticed at the very beginning, because the size of the debug is the same with the e5 mode
|
|
2022-11-10 09:42:07
|
oh no
|
|
2022-11-10 09:42:12
|
its e7
|
|
|
ayumi
So `benchmark_xl` seems to use (https://github.com/libjxl/libjxl/blob/main/tools/benchmark/benchmark_codec_jxl.cc#L142) `jxl::ParseSpeedTier(const std::string&, SpeedTier*)` (https://github.com/libjxl/libjxl/blob/main/lib/jxl/enc_params.h#L51) to parse speed/effort. If the first parameter is `"cheetach"` it will assign `SpeedTier::kCheetah` to the second parameter. If the first parameter is `"4"` it will convert it to an integer, subtract it from 10, check if the result (`6`) is between `SpeedTier::kLightning` and `SpeedTier::kTortoise` and, if it is, assign `SpeedTier(6)` to the second parameter. Since `SpeedTier(6)` is `SpeedTier:kCheetah` I do not see any obvious problems here.
|
|
2022-11-10 09:42:52
|
in the code, the SpeedTier is considered from Zero?
|
|
|
|
ayumi
|
2022-11-10 10:33:35
|
The lowest assigned value of the `SpeedTier` enumeration is `1`/~~`SpeedTier::kLightning`~~`SpeedTier::kTortoise`. Passing a `"0"` to `jxl::ParseSpeedTier` would fail the range check, causing the function to return `false`. On the `benchmark_xl` side this would be interpreted as lack of speed tier/effort assignment, in which case the default setting of `SpeedTier::kSquirrel` (effort 7) would be used.
|
|
|
|
DZgas Ж
|
|
ayumi
The lowest assigned value of the `SpeedTier` enumeration is `1`/~~`SpeedTier::kLightning`~~`SpeedTier::kTortoise`. Passing a `"0"` to `jxl::ParseSpeedTier` would fail the range check, causing the function to return `false`. On the `benchmark_xl` side this would be interpreted as lack of speed tier/effort assignment, in which case the default setting of `SpeedTier::kSquirrel` (effort 7) would be used.
|
|
2022-11-10 10:43:31
|
Shortly.. A bug in determining the speed. I don't understand why everything works like this? Why it impossible to fixed the numbers and words one time
|
|
|
|
ayumi
|
2022-11-10 10:51:38
|
So you are saying that it does not work correctly, because you get same files while using different effort/speed tiers, or did I misunderstood something?
|
|
|
|
DZgas Ж
|
|
ayumi
So you are saying that it does not work correctly, because you get same files while using different effort/speed tiers, or did I misunderstood something?
|
|
2022-11-11 01:02:25
|
Name cheetach is e 4. But bug, do e 7
|
|
|
|
Traneptora
|
2022-11-11 01:06:30
|
well I just fully implemented LF Groups and HF Metadata, and none of the modular sub-bitstreams are throwing any errors on any of the ANS final states or overrunning any buffers
|
|
2022-11-11 01:06:40
|
the first time I try to execute it
|
|
2022-11-11 01:07:06
|
I'm not sure if that's because I've been aggressively squashing modular bugs so much or because they're all using prefix codes <:KEKW:643601031040729099>
|
|
|
|
veluca
|
2022-11-11 01:07:23
|
congrats xD
|
|
2022-11-11 01:07:48
|
does it also look reasonable?
|
|
|
|
Traneptora
|
2022-11-11 01:08:10
|
well it errored out when I got to HF Global as this is still unimplemented
|
|
|
|
veluca
|
2022-11-11 01:08:15
|
fair
|
|
2022-11-11 01:08:46
|
what images are you trying this on? if you do some palette-ized or progressive modular images you should get ANSed data in LF groups
|
|
|
|
Traneptora
|
2022-11-11 01:09:03
|
VarDCT
|
|
|
|
veluca
|
2022-11-11 01:09:08
|
ah, fair
|
|
|
|
Traneptora
|
2022-11-11 01:09:08
|
I'm starting work on VarDCT
|
|
|
|
veluca
|
2022-11-11 01:09:19
|
there's no way that's not ANSed then
|
|
|
|
Traneptora
|
2022-11-11 01:09:29
|
these are things like HF Metadata
|
|
2022-11-11 01:09:36
|
which contain a modular subbitstream
|
|
|
|
veluca
|
2022-11-11 01:10:04
|
LF groups are for sure ANSed and contain a bunch of data
|
|
|
|
Traneptora
|
2022-11-11 01:10:18
|
the image is 1280x720 so there's only one LF Group
|
|
|
|
veluca
|
2022-11-11 01:10:26
|
if they decode without failing the final state, you almost certainly got it right
|
|
|
|
Traneptora
|
2022-11-11 01:10:49
|
noice, I also had no errors placing the DCT blocks when decoding HF metadata
|
|
2022-11-11 01:11:02
|
although that might be faulty block placement algorithms rather than anything else
|
|
2022-11-11 01:11:29
|
the spec said "place it topmost and leftmost" but this is not always well-defined
|
|
2022-11-11 01:11:47
|
I'm assuming if it can be placed in more than one location, lower y always takes precedence over lower x
|
|
2022-11-11 01:12:11
|
i.e. suppose the VarDCT block at (0, 0) is 32 wide, and 16 tall
|
|
|
|
veluca
|
2022-11-11 01:12:15
|
that's indeed correct IIRC
|
|
|
|
Traneptora
|
2022-11-11 01:12:35
|
I'm assuming I'm *supposed* to put it to the right of the 32-wide block, even though putting it at (0, 16) is "more toplefty"
|
|
2022-11-11 01:12:37
|
if you will
|
|
2022-11-11 01:12:47
|
but that was a guess
|
|
|
|
veluca
|
2022-11-11 01:13:17
|
another way to see it: view the image as 8x8 blocks, then encode each top-left corner of a coding block in the usual row-by-row order
|
|
2022-11-11 01:13:51
|
and skip things that are not top-left corners of a DCT block
|
|
|
|
Traneptora
|
2022-11-11 01:14:13
|
what I ended up doing was filling the whole DCTSelect array with null, and then iterated over the whole array
|
|
2022-11-11 01:14:18
|
y outer, x inner
|
|
2022-11-11 01:15:21
|
and then if I found a place where it can fit, I put the block there, and then fill the *entire* array rectangle that it occupies with its enum type
|
|
|
|
veluca
|
2022-11-11 01:15:33
|
yep, that's what libjxl does
|
|
|
|
Traneptora
|
2022-11-11 01:16:03
|
well, I start back at (0, 0) for the next block, which is necessary unless I get one more assumption
|
|
2022-11-11 01:16:34
|
that the encoder always places blocks info scanline order no matter what fits where
|
|
2022-11-11 01:16:42
|
for example, suppose I have a 256-wide group
|
|
|
|
veluca
|
2022-11-11 01:17:02
|
yeah, no, you're right, the spec is too loose
|
|
2022-11-11 01:17:23
|
it allows way more than what libjxl does, and the libjxl thing is way more reasonable
|
|
|
|
Traneptora
|
2022-11-11 01:17:56
|
and I've now placed only 32x32 blocks, so that there's 32 pixels remaining left in the group
|
|
2022-11-11 01:18:08
|
but the decoder then spits out a 64x64 block ID
|
|
2022-11-11 01:18:17
|
I can't put it there, so I put it at (0, 32)
|
|
2022-11-11 01:18:24
|
and then *after that* it gives me a 16x16 block
|
|
2022-11-11 01:18:30
|
which actually fits back up top
|
|
2022-11-11 01:18:42
|
spec permits this sort of thing
|
|
|
|
veluca
|
2022-11-11 01:18:53
|
yes, the spec would indeed tell you to do that, and that is *not* what was meant to be allowed
|
|
|
|
Traneptora
|
2022-11-11 01:19:04
|
I'm wondering if the spec *should* forbid this
|
|
|
|
veluca
|
2022-11-11 01:19:12
|
IMO 100% yes
|
|
|
|
Traneptora
|
2022-11-11 01:19:40
|
I can't imagine *why* an encoder would spit out blocks not in scanline order of topleft corner
|
|
2022-11-11 01:19:45
|
so it makes sense to just disallow it
|
|
|
|
veluca
|
|
|
_wb_
|
|
|
fab
|
2022-11-11 11:12:55
|
What I intend that libjxl hasn't a tuning for autistic vision
|
|
2022-11-11 11:13:34
|
|
|
2022-11-11 11:13:43
|
this with this command
|
|
2022-11-11 11:13:45
|
cjxl C:\Users\Use\Documents\ww\A.png C:\Users\Use\Documents\ww\A.jxl --intensity_target=273 -d 0.552 -e 7 --photon_noise=ISO494 --lossless_jpeg=0 -p --epf=3 --dots=0
|
|
2022-11-11 11:13:49
|
https://artifacts.lucaversari.it/libjxl/libjxl/2022-11-10T16%3A51%3A35Z_37bd58d3fff4e30a1b87352f30e35fcde7c4f5cb/
|
|
2022-11-11 11:14:02
|
|
|
2022-11-11 11:14:09
|
adding cjxl C:\Users\Use\Documents\ww\A.png C:\Users\Use\Documents\ww\A.jxl --intensity_target=273 -d 0.552 -e 7 --photon_noise=ISO494 --lossless_jpeg=0 -p --epf=3 --dots=0 -I 6
|
|
2022-11-11 11:14:14
|
then
|
|
2022-11-11 11:14:15
|
-I 6
|
|
2022-11-11 11:14:48
|
|
|
2022-11-11 11:14:54
|
Jon intention has always been about fidelity
|
|
2022-11-11 11:15:05
|
about a general use codec
|
|
2022-11-11 11:15:13
|
screenshots
|
|
2022-11-11 11:15:16
|
etc
|
|
2022-11-11 11:15:50
|
it doesn't use entropy more
|
|
2022-11-11 11:16:03
|
it does uses a MA coder
|
|
|
|
Traneptora
|
2022-11-11 11:48:04
|
isn't this what <#840831132009365514> is for
|
|
2022-11-11 01:06:01
|
Just to be clear, nested distributions never use LZ77, right?
|
|
|
|
veluca
|
2022-11-11 01:14:18
|
nested distributions?
|
|
|
|
Traneptora
|
2022-11-11 01:16:55
|
when reading the context map requires reading another distribution
|
|
|
|
veluca
|
2022-11-11 01:19:43
|
ah, they *could* actually IIRC
|
|
2022-11-11 01:20:08
|
but I think there's a limit on how many times you can LZ77 it (which I think is 2 total)
|
|
|
|
Traneptora
|
2022-11-11 01:20:13
|
I'm wondering because I thought they weren't permitted so I added a check for it
but now it's being tripped
|
|
|
|
veluca
|
2022-11-11 01:20:45
|
let me guess, the spec doesn't mention this
|
|
2022-11-11 01:21:25
|
lib/jxl/dec_context_map.cc:77
|
|
|
|
Traneptora
|
2022-11-11 01:21:42
|
I just checked libjxl, it disallows it for nested distributions if the num contexts you're reading is <= 2
|
|
2022-11-11 01:21:49
|
my check was for <= 1
|
|
2022-11-11 01:21:54
|
so that's why mine tripped but libjxl didn't
|
|
|
|
veluca
|
2022-11-11 01:22:16
|
uh, isn't your check more permissive?
|
|
|
|
Traneptora
|
2022-11-11 01:22:39
|
er wait, I just always had it disallowed
|
|
2022-11-11 01:22:40
|
nvm
|
|
2022-11-11 01:22:42
|
<:KEKW:643601031040729099>
|
|
2022-11-11 01:23:14
|
makes sense
|
|
2022-11-11 01:23:33
|
I changed it to numDists <= 2 and it's verifying now
|
|
2022-11-11 01:24:24
|
aight, HFPasses are now decoding properly :)
|
|
2022-11-11 01:24:30
|
I think
|
|
2022-11-11 01:24:41
|
I could be getting garbo
|
|
|
|
veluca
|
2022-11-11 01:35:26
|
nice
|
|
2022-11-11 01:35:40
|
so the fact that the spec doesn't say this is also a bug
|
|
|
|
Traneptora
|
2022-11-11 01:39:38
|
libjxl comments say that nested Lz77 distributions could be used to construct malicious bitstreams
|
|
2022-11-11 01:39:47
|
like explodo-streams
|
|
2022-11-11 01:39:51
|
(think zip bomb)
|
|
2022-11-11 01:40:02
|
so that's probably why it just refuses
|
|
|
|
veluca
|
2022-11-11 01:51:46
|
yes, but we should also write in the spec that bitstream that have too many nested lz77s are invalid
|
|
|
|
_wb_
|
2022-11-11 02:03:16
|
Is there no bound on the nesting? I didn't even realize there was something nested there...
|
|
|
|
Traneptora
|
2022-11-11 02:04:37
|
to read a complex cluster map requires you to read another entropy-coded stream
|
|
2022-11-11 02:06:07
|
see section C.2.2
|
|
|
|
_wb_
|
2022-11-11 02:30:22
|
Oh, and this can be done recursively without the number of distros getting necessarily smaller?
|
|
|
|
Traneptora
|
2022-11-11 02:30:45
|
no, the number of pre-clusted distributions on the nested stream is declaratively 1
|
|
2022-11-11 02:30:50
|
so it can't re-nest
(this is also explained in the spec)
|
|
2022-11-11 02:31:15
|
but lz77 could potentially cause issues, at least according to the libjxl code
|
|
|
|
veluca
|
2022-11-11 02:39:21
|
and so you could in principle ANS it
|
|
|
|
Traneptora
|
2022-11-11 02:51:21
|
Is the arrangement of the VarBlocks identical for each group within the LF Group?
|
|
2022-11-11 02:51:48
|
I'd assume not
|
|