|
Traneptora
|
2021-10-23 01:32:04
|
that's the issue here
|
|
2021-10-23 01:32:56
|
if I have to parse the entire header to determine if the file is legal, then I should probably just add a parser
|
|
2021-10-23 01:32:57
|
and call it
|
|
2021-10-23 01:33:10
|
that way it doesn't have to decode an entire frame to parse the file
|
|
2021-10-23 01:33:31
|
luckily, I can add that to avcodec
|
|
|
yurume
|
2021-10-23 01:33:59
|
yeah, that sounds the only viable option to me
|
|
|
|
Traneptora
|
2021-10-23 01:36:44
|
unfortunately I don't quite have access to the header spec
|
|
|
|
yurume
|
2021-10-23 01:38:01
|
well for example ImageMetadata fields are defined in https://github.com/libjxl/libjxl/blob/ad96c3d/lib/jxl/image_metadata.cc#L241-L314
|
|
|
|
Traneptora
|
2021-10-23 01:42:30
|
tbh, I'd rather just read a spec than try to reverse-engineer code
|
|
2021-10-23 01:42:45
|
jon said that there might be another draft
|
|
2021-10-23 01:42:50
|
I'm hoping for that
|
|
|
|
yurume
|
2021-10-23 01:44:05
|
do you have any chance to use (possibly a part of) libjxl directly?
|
|
|
|
Traneptora
|
2021-10-23 01:44:17
|
libjxl is written in C++
|
|
2021-10-23 01:44:19
|
so probably not
|
|
2021-10-23 01:44:20
|
this is C
|
|
2021-10-23 01:44:48
|
I also don't want to incur a libjxl dependency to *probe*
|
|
2021-10-23 01:45:00
|
only to actually decode
|
|
2021-10-23 01:46:59
|
in either case I could always look at the C++ code
|
|
2021-10-23 01:47:10
|
but avutil provides some useful utilities
|
|
2021-10-23 01:47:16
|
fore xample, I have a convenient macro in the prober already
|
|
2021-10-23 01:48:03
|
`AV_RL64(b)` returns a 64-bit little-endian integer at teh buffer pointer `b`
|
|
|
improver
|
2021-10-23 01:48:27
|
wasnt fdis on how was that site called uhh
|
|
2021-10-23 01:48:35
|
zlibrary
|
|
|
|
Traneptora
|
2021-10-23 01:53:23
|
so I can do something like
```c
size_t tbits = 0, bc;
#define get_bits(bc) (tbits += (bc), (AV_RL64(b + (tbits-(bc))/8) >> ((tbits-(bc)) % 8)) & ~(~(uint64_t)0 << (bc))
int c = get_bits(2);
int u32_bits = (c & 0x02 ? 0x12 : 0x09) | (c & 0x01 ? 0x0C : 0x00);
int u32 = get_bits(u32_bits):
```
|
|
2021-10-23 01:54:03
|
`(c & 0x02 ? 0x12 : 0x09) | (c & 0x01 ? 0x0C : 0x00)` is just a nonbranching weird hack for `U32(u(9), u(13), u(18), u(30))`
|
|
2021-10-23 01:54:07
|
used in the size header
|
|
2021-10-23 01:54:22
|
well kinda nonbranching
|
|
2021-10-23 01:54:26
|
idk if the ternary operator is branching
|
|
|
|
improver
|
2021-10-23 01:54:40
|
it CAN be done with CMOV
|
|
|
|
Traneptora
|
2021-10-23 01:54:49
|
I do know that `a > 0` is nonbranching
|
|
2021-10-23 01:54:52
|
so a trick I find is something like
|
|
2021-10-23 01:56:15
|
so I could do something like `9 * ((c & 0x02) >> 1) + 1)`
|
|
2021-10-23 01:56:34
|
etc
|
|
|
|
yurume
|
2021-10-23 01:56:38
|
isn't `((30 << 24) | (18 << 16) | (13 << 8) | 9) >> (c << 3) & 0xff` also non-branching
|
|
|
|
improver
|
2021-10-23 01:56:42
|
for non-branching stuff, one could always do bit magic if one really wanted to
|
|
|
|
Traneptora
|
2021-10-23 01:57:01
|
huh, that does work
|
|
|
|
improver
|
2021-10-23 01:57:04
|
eg even conditions can be expressed with bit magic
|
|
|
|
Traneptora
|
2021-10-23 01:57:04
|
and that's way more readable
|
|
2021-10-23 01:57:11
|
I'll use that instead, thanks
|
|
|
yurume
isn't `((30 << 24) | (18 << 16) | (13 << 8) | 9) >> (c << 3) & 0xff` also non-branching
|
|
2021-10-23 01:59:47
|
`(((30 << 24) | (18 << 16) | (13 << 8) | 9) >> (c << 3)) & 0xff`
|
|
2021-10-23 01:59:54
|
left shift is grabbier than bitwise and/or
|
|
|
|
yurume
|
2021-10-23 02:00:13
|
ah yeah
|
|
|
|
Traneptora
|
2021-10-23 02:00:18
|
`bits = ((30 << 24 | 18 << 16 | 13 << 8 | 9) >> (c << 3)) & 0xff;`
|
|
|
|
yurume
|
2021-10-23 02:00:25
|
anyway that's an idea
|
|
|
|
Traneptora
|
2021-10-23 02:00:26
|
left shift has an unnecessarily high precedence for some reason
|
|
2021-10-23 02:00:35
|
it's also *way* more readable to do it this way than my way
|
|
2021-10-23 02:00:52
|
youd' have to duck debug my method to figure out what it does
|
|
2021-10-23 02:02:32
|
this looks much more intuitive since I'm not bit-masking ordinary integers, *and* I'm guessing the compiler will easily optimize out that entire `(30 << 24 | 18 << 16 | 13 << 8 | 9)` expression anyway
|
|
2021-10-23 02:02:48
|
not that speed in parsing a header is that critical, I just wanted to avoid a big if tree
|
|
|
|
yurume
|
2021-10-23 02:05:27
|
I would just make it a macro though
|
|
|
_wb_
|
|
Traneptora
left shift is grabbier than bitwise and/or
|
|
2021-10-23 02:06:15
|
"grabbier" is a nice way to express operator precedence
|
|
|
|
Traneptora
|
2021-10-23 02:06:17
|
only issue with macroing it is that `30, 18, 13, 9` might not always be constants
|
|
|
|
veluca
|
2021-10-23 03:27:07
|
so if you're so inclined, you can compile jxl-rs in a way that will produce a PDF with a description of headers
|
|
2021-10-23 03:27:37
|
|
|
2021-10-23 03:28:23
|
this is just all the tables in alphabetic order, the "outermost" one is "Table 11: FileHeaders bundle."
|
|
|
Nova Aurora
|
|
_wb_
3 more days to go
|
|
2021-10-25 02:35:40
|
I probably missed it, but was there any noteworthy news
|
|
|
|
_wb_
|
2021-10-25 05:05:47
|
Nothing very exciting; part 3 (conformance testing) is moving to DIS, and part 1 is officially getting started with the 2nd edition (which will be editorial improvements and minor technical fixes/tweaks without adding anything new or breaking bitstreams).
|
|
|
|
improver
|
2021-10-25 07:25:59
|
most important q: will draft for 2nd ed be public
|
|
2021-10-25 07:27:57
|
or it was never intended to be like that. but it would be kinda helpful tbh
|
|
|
Jyrki Alakuijala
|
|
_wb_
|
|
2021-10-25 07:32:56
|
looks likeiIn Brotli the huffman tables are codified about 17x more efficiently than in jxl ... we could have paid more attention here (edit: I made this comment falsely assuming the JPEG1 stream was a JXL stream)
|
|
2021-10-25 07:34:14
|
Brotli has a code to codify the codes codifying the huffman tables 😛
|
|
|
|
_wb_
|
2021-10-25 07:34:25
|
We are using the Brotli nested huffman table encoding
|
|
|
|
Jyrki Alakuijala
|
2021-10-25 07:34:41
|
interesting
|
|
2021-10-25 07:34:48
|
we get quite a lot of zeros
|
|
2021-10-25 07:36:01
|
perhaps this is a more lose packing because it was not generated with normal tools?
|
|
|
|
_wb_
|
|
improver
most important q: will draft for 2nd ed be public
|
|
2021-10-25 07:36:46
|
The CD will be public, but the drafts before that not. Unfortunately. CD is planned for April.
|
|
|
Jyrki Alakuijala
perhaps this is a more lose packing because it was not generated with normal tools?
|
|
2021-10-25 07:37:23
|
There might be opportunity to do something denser and shave off a byte or two in jxl art...
|
|
|
|
Jyrki Alakuijala
|
2021-10-25 08:20:13
|
the brotli huffman code coding allows for rleing efficiently by repeating 17-code: https://datatracker.ietf.org/doc/html/rfc7932#section-3.5
|
|
|
|
veluca
|
2021-10-25 08:20:59
|
not sure what you mean, we use brotli's huffman tables in JXL
|
|
2021-10-25 08:21:07
|
that image is JPEG 1 AFAIU 😛
|
|
|
|
Jyrki Alakuijala
|
2021-10-25 08:21:31
|
ahahahaaaa
|
|
2021-10-25 08:21:40
|
that explains my confusion
|
|
2021-10-25 08:21:51
|
thank you Luca, now it makes perfect sense
|
|
|
fab
|
2021-10-25 12:44:07
|
139 MB (146.681.160 bytes) 589 files, 0 folders that's Instagram situation, is not too much
|
|
|
|
Nova Aurora
|
|
_wb_
The CD will be public, but the drafts before that not. Unfortunately. CD is planned for April.
|
|
2021-10-25 10:22:32
|
But will this CD contain all the stuff from that changed since the (public) last CD? <:Thonk:805904896879493180>
|
|
|
|
improver
|
2021-10-25 10:34:06
|
it should otherwise whats the point even
|
|
|
|
Nova Aurora
|
|
improver
it should otherwise whats the point even
|
|
2021-10-25 11:03:02
|
Just keep making tiny improvements committee drafts so that they are public <:galaxybrain:821831336372338729>
|
|
|
|
improver
|
2021-10-25 11:23:02
|
sounds like more effort than its worth
|
|
|
|
Nova Aurora
|
|
improver
sounds like more effort than its worth
|
|
2021-10-25 11:25:34
|
To keep the updated spec outside a paywall, I think it's genius
|
|
|
|
improver
|
2021-10-25 11:27:02
|
oh i didnt read ur stuff properly. yeh
|
|
2021-10-25 11:27:34
|
v3 soon to reveal all the hidden sauce of v2
|
|
|
|
veluca
|
2021-10-25 11:53:22
|
except the plan is to have nothing non-editorial in v2 post CD
|
|
2021-10-25 11:53:26
|
so whatevs 😛
|
|
|
|
_wb_
|
2021-10-26 02:13:13
|
|
|
2021-10-26 05:24:30
|
<@519159731529580554> started posting the videos from ImageReady v1.2: https://twitter.com/HenriHelvetica/status/1453048114114400270?t=352yBSbCtvzxqI9U1Y-96A&s=19
|
|
2021-10-26 05:45:58
|
https://youtu.be/YsbsKimR-Fw
|
|
|
Lastrosade
|
2021-10-26 07:27:25
|
Is there a tool that I could use to determine if a jxl file is a repacked jpeg?
|
|
|
|
_wb_
|
2021-10-26 07:31:39
|
jxlinfo might say that
|
|
2021-10-26 07:31:47
|
Not sure if it does atm
|
|
2021-10-26 07:32:07
|
Otherwise just grep the file for `jbrd` :)
|
|
|
Eugene Vert
|
2021-10-26 07:33:32
|
`jxlinfo {file} | grep jbrd` works)
|
|
|
|
Lastrosade
|
2021-10-26 07:34:13
|
jxlinfo doesn't out a jbrd
|
|
2021-10-26 07:34:31
|
What it has is a "have_container"
|
|
2021-10-26 07:34:42
|
but I'm not sure that this is what I want
|
|
|
|
Eugene Vert
|
2021-10-26 07:36:50
|
I think it's not in the release version yet, works after https://github.com/libjxl/libjxl/pull/693
|
|
|
|
_wb_
|
2021-10-26 07:38:27
|
have_container is not enough to know it can be reconstructed to jpeg
|
|
|
|
Eugene Vert
|
2021-10-26 07:39:41
|
Now jxlinfo prints something like
```
box: type: "JXL " size: 12
box: type: "ftyp" size: 20
box: type: "jbrd" size: 476
box: type: "jxlc" size: 150555
...
```
where `jbrd` is the box with jpeg reconstruction data
|
|
|
|
Lastrosade
|
2021-10-26 07:40:30
|
Ah, ok thanks
|
|
|
|
_wb_
|
2021-10-26 07:43:41
|
If anything else fails, you can also just literally grep the file for "jbrd", since that's a literal 4 bytes that occur with high likelihood only if there is a jbrd box :)
|
|
|
|
Lastrosade
|
|
_wb_
If anything else fails, you can also just literally grep the file for "jbrd", since that's a literal 4 bytes that occur with high likelihood only if there is a jbrd box :)
|
|
2021-10-26 07:44:22
|
yeah this works thanks
|
|
|
|
Nova Aurora
|
|
_wb_
|
|
2021-10-26 08:37:22
|
Cool, helpful for deciphering the header
|
|
|
|
_wb_
|
2021-10-26 08:39:52
|
It's of course only one example; every header is kind of unique since lots of fields are conditional and are only in the header if previous fields have certain values
|
|
2021-10-26 08:43:43
|
A small very default image can have 2 bytes signature, 2 bytes size SizeHeader + ImageHeader, 1 byte FrameHeader, 2 byte TOC, so 7 bytes of header only (much of it padding bits too)
|
|
|
Maiki3
|
2021-10-26 11:43:19
|
|
|
2021-10-26 11:43:35
|
⬆️ anyone think Disneyplus would benefit from JXL? lol
|
|
|
|
Nova Aurora
|
|
Maiki3
⬆️ anyone think Disneyplus would benefit from JXL? lol
|
|
2021-10-27 02:52:33
|
This is why Netflix has been so focused on AVIF
|
|
|
|
Jyrki Alakuijala
|
2021-10-27 05:31:03
|
we haven't been focusing on the mixed photo/text kind of images yet on jxl
|
|
2021-10-27 05:31:32
|
I suspect we can find another 15 % gains there
|
|
2021-10-27 05:33:04
|
more use of the dct2x2 transform helps for this kind of content
|
|
|
|
fab
|
2021-10-27 09:00:35
|
so speed 8 with new heuistics is patches 0?
|
|
2021-10-27 02:51:14
|
Jxl lossless is bigger than original dng
|
|
2021-10-27 02:51:24
|
Is it expected?
|
|
2021-10-27 02:54:17
|
I can send the dng
|
|
2021-10-27 02:54:30
|
With 4g+ and mega.nz mobile
|
|
|
Fox Wizard
|
2021-10-27 02:55:26
|
Can't say I've ever had that happen
|
|
|
|
_wb_
|
2021-10-27 03:35:04
|
Does the DNG contain CFA Bayer data?
|
|
|
|
fab
|
2021-10-27 03:35:23
|
ok i send
|
|
|
|
_wb_
|
2021-10-27 03:35:58
|
How are you compressing it with jxl? we don't currently have any tools that take DNG input or any other CFA Bayer data
|
|
2021-10-27 03:36:25
|
if you're compressing the debayered image, that's not quite the same thing and it's also 3 times as many samples
|
|
|
|
fab
|
2021-10-27 03:36:37
|
https://mega.nz/file/OVlGxKJB#JjLJpr-BNbJVu9IrcKETl6u0zjvzUANiQx3G-4Ol8_Q
|
|
2021-10-27 03:37:03
|
the png hee is it
|
|
|
|
_wb_
|
2021-10-27 03:38:48
|
what PNG?
|
|
|
|
fab
|
2021-10-27 03:38:54
|
https://mega.nz/file/bJkADajR#H_1RM0gRqY6Hwp1sjfhxKn7hJXkoyAnlwkCBlWo_DFk
|
|
2021-10-27 03:39:05
|
This
|
|
|
|
_wb_
|
2021-10-27 03:39:26
|
the PNG is not the same thing as the DNG
|
|
|
|
fab
|
2021-10-27 03:39:32
|
https://artifacts.lucaversari.it/libjxl/libjxl/2021-10-26T12%3A35%3A12Z_bbb4418c39f22e73e485bdb09c1f092774ad476f/
|
|
|
|
_wb_
|
2021-10-27 03:39:37
|
PNG is RGB
|
|
|
|
fab
|
2021-10-27 03:39:45
|
dng what is
|
|
|
|
_wb_
|
2021-10-27 03:39:53
|
DNG is RGGB
|
|
|
|
fab
|
2021-10-27 03:39:57
|
it says to me compessed jpg
|
|
2021-10-27 03:40:03
|
so is not lossless
|
|
2021-10-27 03:40:12
|
but jxl is lossless
|
|
2021-10-27 03:40:16
|
so what is the poblem
|
|
2021-10-27 03:40:23
|
i inceased numb of colos
|
|
2021-10-27 03:40:26
|
?+
|
|
2021-10-27 03:40:48
|
so not a bug?
|
|
2021-10-27 03:41:15
|
16,4 MB (17.290.511 byte)
|
|
2021-10-27 03:41:17
|
jxl
|
|
2021-10-27 03:41:33
|
s 7 q 100
|
|
|
|
_wb_
|
2021-10-27 03:41:58
|
File Type : DNG
File Type Extension : dng
MIME Type : image/x-adobe-dng
Exif Byte Order : Little-endian (Intel, II)
Subfile Type : Full-resolution image
Image Width : 4640
Image Height : 3472
Bits Per Sample : 16
Compression : JPEG
Photometric Interpretation : Color Filter Array
|
|
2021-10-27 03:42:19
|
no idea what this is
|
|
2021-10-27 03:42:29
|
16-bit JPEG?
|
|
2021-10-27 03:42:45
|
storing CFA data?
|
|
2021-10-27 03:44:09
|
anyway, it's the PNG size you should compare to, not the DNG size
|
|
2021-10-27 03:45:14
|
that is until we can have a way to input CFA data to cjxl, then we could try actually representing the DNG data
|
|
2021-10-27 03:45:38
|
somewhat surprising to me that this "raw" seems to be JPEG-compressed, I wonder what's up with that...
|
|
|
|
Fox Wizard
|
2021-10-27 03:48:38
|
Lossy DNG? <:ReeCat:806087208678588437>
|
|
2021-10-27 03:49:05
|
Also, inefficient PNG. 17.5% smaller after optimizing with ECT <a:ThinkingSphere:821038590091329557>
|
|
|
|
fab
|
|
_wb_
somewhat surprising to me that this "raw" seems to be JPEG-compressed, I wonder what's up with that...
|
|
2021-10-27 04:03:59
|
motion cam hdr
|
|
2021-10-27 04:04:34
|
i tend to trust those apps
|
|
2021-10-27 04:04:48
|
that they are not quality 100 jpgs
|
|
2021-10-27 04:05:22
|
anyway it could be that is a jpg inside a dng containe o not?
|
|
|
diskorduser
|
2021-10-27 04:08:10
|
What camera app is that? Camera apps on my phone take bayered DNG.
|
|
2021-10-27 04:08:59
|
Miui camera doesn't do any compression in DNG file. And Google camera does lossless zip like compression.
|
|
|
|
fab
|
2021-10-27 04:09:22
|
yes but they are poorly autofocused
|
|
2021-10-27 04:09:40
|
only the center is on focus
|
|
|
|
diskorduser
|
2021-10-27 04:10:30
|
Jxl lossless compresses dng very well. But emma compresses more.
|
|
|
|
Fox Wizard
|
2021-10-27 04:12:31
|
Rip, almost smaller than your DNG <:Cheems:884736660707901470>
|
|
2021-10-27 04:12:41
|
At least when I compress that PNG :p
|
|
|
|
fab
|
|
diskorduser
Jxl lossless compresses dng very well. But emma compresses more.
|
|
2021-10-27 04:13:17
|
i want it at 8 mb
|
|
|
Fox Wizard
Rip, almost smaller than your DNG <:Cheems:884736660707901470>
|
|
2021-10-27 04:13:43
|
embed didn't work, i think it needs max 8 mb
|
|
|
|
Fox Wizard
|
2021-10-27 04:14:18
|
You could download it I guess
|
|
2021-10-27 04:14:31
|
And rip. 8MB will only be possible with lossy for that <:Cheems:884736660707901470>
|
|
|
|
fab
|
2021-10-27 04:14:47
|
yes firefox now automatically opens jxl files
|
|
2021-10-27 04:14:52
|
and even other types
|
|
|
|
Fox Wizard
|
2021-10-27 04:19:17
|
Fire <:Poggers:805392625934663710>s
|
|
|
|
Traneptora
|
2021-10-27 09:22:27
|
it can't tho
|
|
2021-10-27 09:22:34
|
it just tries to and tells you that the file is corrupt
|
|
2021-10-27 09:22:50
|
|
|
|
nathanielcwm
|
|
Fox Wizard
Also, inefficient PNG. 17.5% smaller after optimizing with ECT <a:ThinkingSphere:821038590091329557>
|
|
2021-10-28 01:30:12
|
:carl
|
|
2021-10-28 01:30:22
|
fabian is a light theme user <:PepeHands:808829977608323112>
|
|
|
w
|
2021-10-28 01:38:40
|
im a light theme user
|
|
|
|
diskorduser
|
|
w
im a light theme user
|
|
2021-10-28 01:47:59
|
Discord with light theme?
|
|
|
|
w
|
|
|
Traneptora
|
2021-10-28 02:01:57
|
<:NotLikeMiya:731194408891973652>
|
|
|
|
diskorduser
|
2021-10-28 02:55:04
|
light theme is not that bad when using it on well lit environment.
|
|
|
|
fab
|
2021-10-28 08:05:04
|
this looks good
|
|
2021-10-28 08:05:05
|
https://old.lucaversari.it/jpeg_xl_data/bt_after/red-room.png.jxl_d4_tortoise.png
|
|
2021-10-28 08:07:50
|
https://old.lucaversari.it/jpeg_xl_data/bt_after/geranium2.png.jxl_d2_epf0_tortoise.heatmap.png
https://old.lucaversari.it/jpeg_xl_data/bt_after/pink-flower.png.jxl_d3_epf0.png
|
|
2021-10-28 08:09:38
|
how it looks at that quality
|
|
2021-10-28 08:09:39
|
https://discord.com/channels/794206087879852103/794206087879852106/902846589272481792
|
|
|
lithium
|
2021-10-28 03:46:55
|
PR790 also affect d0.5 quality?
> https://github.com/libjxl/libjxl/pull/790
|
|
|
|
_wb_
|
2021-10-28 03:48:15
|
Yes, but only at effort 8 or 9
|
|
|
|
lithium
|
2021-10-28 03:52:11
|
ok, I usually use e8. 🙂
|
|
2021-10-28 03:52:34
|
Thank you for your work
|
|
|
lithium
ok, I usually use e8. 🙂
|
|
2021-10-28 04:14:43
|
Sometime vardct mode e 9 is too slow for me,
For 2975x4067 8 bit depth drawing image use over 3 minute encoding time.
|
|
|
Cool Doggo
|
2021-10-28 04:18:36
|
To me e9 is too slow for big images
|
|
2021-10-28 04:18:43
|
But good for smaller ones
|
|
|
|
Jyrki Alakuijala
|
2021-10-28 05:12:14
|
2 years ago e9 used to be e8, and there was an e9 mode that was 7x slower than current e9 🙂
|
|
|
|
Traneptora
|
2021-10-28 11:30:13
|
how do I limit cjxl's memory usage?
|
|
2021-10-28 11:30:18
|
I'm getting OOM killed
|
|
|
|
improver
|
2021-10-28 11:36:57
|
try faster speeds (not really sure how much itll help but it may a bit)
|
|
|
|
w
|
2021-10-28 11:39:25
|
just turn on swap
|
|
|
monad
|
2021-10-28 11:50:22
|
To say it another way, the tool offers no direct setting. ^^
|
|
|
|
Traneptora
|
2021-10-28 11:56:09
|
turning on swap did not work
|
|
2021-10-28 11:56:39
|
I just started thrashing immediately
|
|
2021-10-28 11:56:42
|
and then OOM happened
|
|
2021-10-29 12:40:11
|
Huh, okay, so apparently, (using -e 4 cause thats' my ram limit)
|
|
2021-10-29 12:40:35
|
taking this massive PNG `12718x10089` and doing `-d 0` encoding takes it from 22M to 14M, wh ich is 0.897 bpp
|
|
2021-10-29 12:40:42
|
I get similar results with `-d 0.40`
|
|
2021-10-29 12:40:52
|
and anything positive but less than `0.40` makes the file *bigger*
|
|
2021-10-29 12:40:58
|
why would that be the case?
|
|
2021-10-29 12:41:22
|
the file itself is a fairly png-compressable image, with large truly monocolor black regions
|
|
2021-10-29 12:41:28
|
I wonder if that matters
|
|
|
BlueSwordM
|
|
Traneptora
why would that be the case?
|
|
2021-10-29 01:20:00
|
At one point, varDCT becomes less efficient than lossless modular coding.
|
|
2021-10-29 01:20:31
|
In normal photographic images, that point is around `-d 0.1-0.2` while with simpler images, that point can be reached earlier.
|
|
|
|
monad
|
2021-10-29 01:20:56
|
peculiarities of the content make different encoding strategies more or less efficient
|
|
2021-10-29 01:22:30
|
same reason for some content picking PNG over JPEG is more efficient
|
|
|
|
lithium
|
|
lithium
PR790 also affect d0.5 quality?
> https://github.com/libjxl/libjxl/pull/790
|
|
2021-10-29 09:32:54
|
I think have some quality improve on PR790,🙂
on d0.5 e8 epf3, those red gradient color tiny artifacts spread situation have slight change,
but I still don't know, which reason produce those tiny artifacts.
|
|
|
|
Traneptora
|
2021-10-29 10:52:49
|
ah, I see, cause it uses a different strategy
|
|
2021-10-29 10:53:07
|
and doesn't adapt the strategy based on efficiency, it just does it based on whether d=0 or not
|
|
|
|
w
|
2021-10-29 10:53:44
|
what about lossy modular
|
|
|
|
Jyrki Alakuijala
|
2021-10-29 11:59:41
|
I consider there might be chances of delta palettization taking some role there -- delta palettization is a way to do lossy modular
|
|
2021-10-29 12:00:07
|
currently it operates only at a single fixed quality which is closer to d1 than d0.1
|
|
|
|
fab
|
2021-10-29 12:35:44
|
so delta palett has not adaptive quantization
|
|
|
|
lithium
|
2021-10-29 04:44:15
|
What's reasonable epf vaule on d 0.5 and 1.0 e8 for high contrast sharp edges(line)?
for now I using epf 3 for those distance, and can slight reduce tiny artifacts,
I guess if we have higher strength epf(4~5), probably can reduce more tiny artifacts?
Also I notice some curve high contrast sharp edges(line) will produce much tiny artifacts.
|
|
|
|
Jyrki Alakuijala
|
2021-10-29 05:44:31
|
delta palette is vector quantization -- larger steps will have more error than small steps, the amount of quantization error varies from pixel to pixel
|
|
2021-10-29 05:45:03
|
but there is an upper bound on the error coming from the non-delta cubes
|
|
|
|
_wb_
|
2021-10-29 06:20:59
|
So I made a github org: https://github.com/jxl-community
|
|
2021-10-29 06:21:41
|
If anyone has jxl-related stuff they want to put there, just ask me and I'll add a repo for you and make you admin of it
|
|
2021-10-29 06:23:44
|
Also the jpegxl.info website is now hosted there, to make it clear that it's a community thing about jxl in general, not a page about libjxl specifically
|
|
2021-10-29 06:25:52
|
If you want to add anything there, even a personal subpage on which you just do whatever you want (as long as it's somewhat related to jxl), just make a pull request, or if you want to make more than a one-time change, I can also give you write access
|
|
|
|
Fox Wizard
|
2021-10-29 06:28:53
|
~~Gotta buy jxl.info for 300 bucks <a:DogeEvil:821038587176419381>~~
|
|
|
|
lithium
|
2021-10-29 06:53:46
|
I guess for good drawing quality,
probably need combine delta palette and --separate(vardct)?
but look like both feature still on very experiment status...
I guess higher strength epf probably like av1 inner filter reduce too much detail.
|
|
|
|
_wb_
|
2021-10-30 06:56:08
|
EPF is trying to be careful (it's an edge preserving filter, not a "smooth everything into oblivion" filter), but yes, it might remove subtle details
|
|
|
|
Jyrki Alakuijala
|
2021-10-30 09:27:53
|
even when epf is turned on, I turn it off in some 8x8 squares
|
|
2021-10-30 09:32:04
|
the heuristics are epf is either on/off at a 8x8 square level
|
|
2021-10-30 09:32:19
|
any heuristic is always a bit off due to being a speed/quality/complexity compromise
|
|
|
|
lithium
|
2021-10-31 08:18:02
|
Not criticize jxl quality, this just my opinion,
For current jxl have really great quality for photo content, cjxl d0.5 ~ 1.0 e 7~8 can reach visually lossless,
For non-photo(drawing, manga, comic) content, also have good quality,
but for specific area, high contrast sharp edges(line) featrue have tiny artifacts,
this situation also happen on cjxl d0.5 ~ 1.0 e 7~8,
I think those tiny artifacts is very annoying and let non-photo can't reach visually lossless on cjxl d0.5 ~ 1.0 e 7~8,
in my test, av1 aomenc can clear up those tiny artifacts,
but for some specific case, av1 also remove some subtle detail.
Probably cjxl can implement more filter or strategy to reduce tiny artifacts for non-photo content?
this feature can help those content reach visually lossless,
like Jon create a experiment option --separate for non-photo content,
this option really help reach visually lossless on cjxl d0.5 ~ 1.0 e 7~8.
About --separate and lossy-palette, I have a little test for those option,
and I guess --separate work better on this case?
> https://discord.com/channels/794206087879852103/805176455658733570/902992934771785728
I collect some comment about high contrast sharp edges(line).
> https://discord.com/channels/794206087879852103/805176455658733570/904045263117770833
|
|
|
|
Traneptora
|
2021-10-31 12:45:41
|
aight, writing a full parser appears to be a lot of work when I really only need to know the size of the headers
|
|
2021-10-31 12:45:52
|
so I think what I'm going to do for now is not do that
|
|
2021-10-31 12:45:59
|
after work today I'll see if I can make that happen
|
|
2021-10-31 12:46:06
|
~~after trick or treating ends that is~~
|
|
2021-10-31 01:04:57
|
anyway, does `djxl foo.jxl foo.jpg` work if `foo.jxl` is not a losslessly jpeg transcodec JXL file?
|
|
2021-10-31 01:05:05
|
does it decode to pixels and then encode as JPEG?
|
|
2021-10-31 01:05:35
|
cause IMO it should simply refuse, possibly with a specific nonzero exit status for scripts
|
|
|
|
_wb_
|
2021-10-31 01:19:20
|
Yes, it does decode to pixels and then encode as jpeg, if the jxl is not a recompressed jpeg
|
|
2021-10-31 01:20:01
|
Probably would make sense to at least have an option to make it refuse instead.
|
|
|
|
Traneptora
|
2021-11-02 11:23:50
|
Hm, I have a JXL file and I think it has an animation header
|
|
2021-11-02 11:23:53
|
```
ff 0a ba 21 e8 df 81 95 0c 28 a9 9c 12 00 aa d8
```
|
|
2021-11-02 11:23:58
|
this is the first few bytes of the header file
|
|
2021-11-02 11:24:13
|
but I'm confused, because it's not an animation
|
|
2021-11-02 11:24:17
|
FF0A is the signature
|
|
2021-11-02 11:24:26
|
the first six bits are the size header, contained in BA
|
|
2021-11-02 11:24:53
|
`BA -> 01 01 11 01` in little endian
|
|
2021-11-02 11:25:50
|
wait, 6bit size header is wrong
|
|
2021-11-02 11:26:54
|
figured it out, had a bug parsing the size header
|
|
|
|
Jyrki Alakuijala
|
|
lithium
Not criticize jxl quality, this just my opinion,
For current jxl have really great quality for photo content, cjxl d0.5 ~ 1.0 e 7~8 can reach visually lossless,
For non-photo(drawing, manga, comic) content, also have good quality,
but for specific area, high contrast sharp edges(line) featrue have tiny artifacts,
this situation also happen on cjxl d0.5 ~ 1.0 e 7~8,
I think those tiny artifacts is very annoying and let non-photo can't reach visually lossless on cjxl d0.5 ~ 1.0 e 7~8,
in my test, av1 aomenc can clear up those tiny artifacts,
but for some specific case, av1 also remove some subtle detail.
Probably cjxl can implement more filter or strategy to reduce tiny artifacts for non-photo content?
this feature can help those content reach visually lossless,
like Jon create a experiment option --separate for non-photo content,
this option really help reach visually lossless on cjxl d0.5 ~ 1.0 e 7~8.
About --separate and lossy-palette, I have a little test for those option,
and I guess --separate work better on this case?
> https://discord.com/channels/794206087879852103/805176455658733570/902992934771785728
I collect some comment about high contrast sharp edges(line).
> https://discord.com/channels/794206087879852103/805176455658733570/904045263117770833
|
|
2021-11-02 12:03:11
|
I have been busylooping on non jpeg xl things. Fully agree with what you say. The artefacts appearing in some small part of the image is not a format limitation, but poor selection of loop filter strength (8x8 grid) and adaptive quantization value in that integral transform (often 8x8 choise for low distances)
|
|
2021-11-02 12:03:52
|
i.e., it would be possible to have 0.5 distance in those few blocks where the artefacts become visible at d 1.0, just the current heuristics don't do it
|
|
2021-11-02 12:04:06
|
also, I have improvements in the pipeline, just haven't been able to prioritize them
|
|
|
|
Traneptora
|
2021-11-02 12:08:30
|
so this is specifically an encoder issue
|
|
2021-11-02 12:08:32
|
not a format issue
|
|
2021-11-02 12:08:41
|
luckily the encoder can be improved
|
|
|
|
lithium
|
|
Jyrki Alakuijala
also, I have improvements in the pipeline, just haven't been able to prioritize them
|
|
2021-11-02 01:00:35
|
I understand, Thank you for your work. 🙂
|
|
|
Scope
|
2021-11-03 08:25:58
|
Hmm, `-E 3 -I 1` also affects JPEG lossless re-compression density
|
|
|
|
_wb_
|
2021-11-03 08:29:57
|
It might influence DC recompression, not sure how things are wired up there
|
|
|
|
Traneptora
|
2021-11-04 11:02:54
|
are the names for effort deprecated?
|
|
2021-11-04 11:03:00
|
like squirrel, wombat, kitten, etc.?
|
|
2021-11-04 11:03:10
|
like are we preferring 1-9 now?
|
|
|
|
veluca
|
2021-11-04 11:18:43
|
I don't think names are deprecated... might be wrong though
|
|
|
|
Traneptora
|
2021-11-04 11:19:07
|
while I'm waiting for the decoder to be merged into lavc, I'm going to start working on the encoder
|
|
2021-11-04 11:19:18
|
and I was wondering if I should expose those as AVOptions
|
|
2021-11-04 11:19:36
|
FFmpeg can map names to numbers for options pretty easily
|
|
2021-11-04 11:19:45
|
but "should I" is a big question
|
|
|
|
fab
|
|
Traneptora
FFmpeg can map names to numbers for options pretty easily
|
|
2021-11-04 11:31:09
|
no
|
|
2021-11-04 11:31:43
|
just call s 6
|
|
2021-11-04 11:33:10
|
i think is useful in that case
|
|
2021-11-04 11:33:25
|
but not show evey file you encode the name of speed
|
|
2021-11-04 11:33:54
|
it looks like copy paste
|
|
|
|
_wb_
|
2021-11-04 11:41:41
|
The numbers make sense for effort, the names make sense for speed
|
|
2021-11-04 11:41:50
|
(it's the same thing of course)
|
|
|
|
Traneptora
|
2021-11-04 11:54:27
|
well, the big question is
|
|
2021-11-04 11:54:29
|
"should I?"
|
|
2021-11-04 11:54:39
|
like I can very easily map names to numbers but it's going to be for the effort setting
|
|
2021-11-04 11:54:46
|
like it'll be called `effort`
|
|
2021-11-04 11:55:04
|
I can also call it `speed` btw, and have it do the same thing
|
|
2021-11-04 11:55:08
|
like alias them to each other
|
|
|
|
fab
|
2021-11-04 11:58:04
|
Just Speed 7 (1080x1080p) cjxl
|
|
2021-11-04 11:58:17
|
Like dont edit nothing
|
|
2021-11-04 11:58:39
|
Make it as tacky as possible
|
|
|
|
_wb_
|
2021-11-04 12:07:48
|
Effort makes the most sense (higher number is more effort)
|
|
2021-11-04 12:08:20
|
The speed names are cute though, and people might use it with cjxl.
|
|
|
|
Scope
|
2021-11-04 12:18:45
|
I think for external tools, just `Effort` and `Q` will be enough, additional aliases will only add not really needed and convenient complexity
|
|
|
|
Traneptora
|
2021-11-04 01:46:49
|
I'm using distance, not Q
|
|
2021-11-04 01:46:51
|
since it's definitely preferred
|
|
|
|
Scope
|
2021-11-04 02:01:12
|
I also use `-d` in cjxl, but for most people and according to all other encoders - `Q` will be more usual and understandable, especially for converters with a gigantic number of formats like fffmpeg or ImageMagick, moreover it also correlates with distance and it has no effect on quality, just like with Effort/Speed
|
|
|
|
w
|
2021-11-04 02:02:35
|
right because jpeg quality in ffmpeg is -q/-qscale
|
|
2021-11-04 02:03:43
|
and so is webp
|
|
|
|
Traneptora
|
2021-11-04 02:06:34
|
does `q` map from 0 to 100 in mjpeg's encoder or is `0` lossless
|
|
2021-11-04 02:06:59
|
cause I'd like to use butteraugli distance if possible since it's something that the jpeg xl guys are trying to promote
|
|
|
|
Scope
|
2021-11-04 02:11:27
|
Internally it is still a distance, just which refers to certain q values and also the modular mode has only Q
|
|
|
|
Traneptora
|
|
_wb_
the -q version of the encoder config was added because people expect that to be there
|
|
2021-11-04 02:16:23
|
<@!111445179587624960> see this message
|
|
|
|
fab
|
2021-11-04 02:16:30
|
0 isnt lossless
|
|
|
|
Traneptora
|
2021-11-04 02:16:32
|
and the one right after it
|
|
2021-11-04 02:16:39
|
`-d 0` is lossless, that's literally what it does
|
|
|
|
fab
|
2021-11-04 02:16:40
|
Q 100 Is lossless
|
|
2021-11-04 02:16:49
|
No Just dont do
|
|
|
|
Traneptora
|
2021-11-04 02:16:50
|
unless you mean in ffmpeg's mjpeg encoder
|
|
2021-11-04 02:17:02
|
in which case it's not
|
|
2021-11-04 02:17:12
|
since legacy jpeg can't do lossless at all
|
|
|
|
w
|
2021-11-04 02:21:58
|
I think it's all about what people expect should work
|
|
|
|
Traneptora
|
2021-11-04 02:22:23
|
I'm hoping people just read the docs
|
|
|
|
w
|
2021-11-04 02:22:37
|
people would expect -q100 to be lossless, maybe an extra -lossless which does q100
|
|
|
|
Traneptora
|
2021-11-04 02:22:38
|
I personally would never expect `-q` to just *work* for a new codec
|
|
|
|
w
|
2021-11-04 02:23:01
|
Like how -crf maps to cq-level
|
|
2021-11-04 02:23:07
|
and crf0 is lossless
|
|
|
|
Traneptora
|
2021-11-04 02:23:13
|
crf doesn't map to CQP does it?
|
|
2021-11-04 02:23:17
|
it should map to libx264's crf
|
|
|
|
w
|
2021-11-04 02:23:24
|
I mean for newer codecs
|
|
2021-11-04 02:23:28
|
which don't use crf
|
|
|
|
Traneptora
|
2021-11-04 02:23:29
|
oh, huh
|
|
2021-11-04 02:23:32
|
I didn't know that
|
|
2021-11-04 02:23:51
|
I could always make it so `-distance lossless` has a preset
|
|
|
|
w
|
2021-11-04 02:24:28
|
because then people also expect something like -jxl-params=blah=1:blah2=2
|
|
|
|
Traneptora
|
2021-11-04 02:24:31
|
I could have `-q` and `-distance` both work, but `-q` would be used as a fallback if `-distance` is not set and only if `-distance` is not set
|
|
2021-11-04 02:24:56
|
with `-lossless` as an alias for `-distance 0`
|
|
2021-11-04 02:25:59
|
```c
static const AVOption libjxl_encode_options[] = {
{ "effort", "Encoding Effort", OFFSET(effort), AV_OPT_TYPE_INT, { .i64 = 7 }, 1, 9, VE, },
{ "distance", "Maximum Butteraugli Distance (quality setting)", OFFSET(effort), AV_OPT_TYPE_FLOAT, { .dbl = -1.0 }, 0.0, 15.0, VE, "distance" },
{ "lossless", NULL, 0, AV_OPT_TYPE_CONST, { .dbl = 0.0 }, 0.0, 15.0, VE, "distance" }
{ "default", NULL, 0, AV_OPT_TYPE_CONST, { .dbl = 1.0 }, 0.0, 15.0, VE, "distance" }
{ NULL },
};
```
|
|
2021-11-04 02:26:00
|
this could work
|
|
2021-11-04 02:26:02
|
oh god the spacing
|
|
2021-11-04 02:26:17
|
here's a screenshot
|
|
2021-11-04 02:27:05
|
I'm not planning on mapping *all* of the options yet
|
|
2021-11-04 02:27:10
|
I just want to get this thing off the ground first
|
|
2021-11-04 02:27:50
|
also it would be `-jxlopts` not `-jxl-params`
|
|
2021-11-04 02:27:59
|
since `-x264opts` is preferred over `-x264-params`
|
|
2021-11-04 02:28:13
|
iirc the second one is an old Libav thing and x264opts originated with FFmpeg
|
|
2021-11-04 02:28:20
|
only merged for compat
|
|
|
|
w
|
2021-11-04 02:30:25
|
I think -speed can map to effort
|
|
2021-11-04 02:30:45
|
maybe also -preset, since it's a popular option
|
|
2021-11-04 02:32:36
|
I'm just looking at what ffmpeg has for other codecs because as important as it is for the clarity of libjxl specifically, in my opinion, it's equally as important to be somewhat consistent within the ffmpeg program itself
|
|
|
|
Scope
|
2021-11-04 02:33:39
|
Most people want to use familiar and understandable options, especially in converters with many formats where uniformity is an advantage, if each format has its own quality concepts which are not immediately understandable it is rather bad, much more convenient to have some more familiar to all aliases, many will understand that -q 90 or -q 95 is one of the settings for good quality, but -d 2 or -z 5 or -y 0.00012 requires a deeper understanding
|
|
|
|
Traneptora
|
2021-11-04 02:34:37
|
I'll map `-q` to `-distance` using the same algorithm that libjxl uses
|
|
2021-11-04 02:34:48
|
but if and only if `-distance` is not set
|
|
2021-11-04 02:35:05
|
I can also map `-crf` the same way
|
|
2021-11-04 02:35:28
|
I'm not *entirely* sure how to add an alias to an AVOption though
|
|
|
|
w
|
2021-11-04 02:35:47
|
crf is funny. I think most people who use it understand it's a scale from 0-63 most of the time
|
|
|
|
Traneptora
|
2021-11-04 02:35:55
|
is it normally?
|
|
|
|
w
|
2021-11-04 02:36:14
|
maybe you can just map it 1-1 to distance
|
|
|
|
Traneptora
|
2021-11-04 02:36:28
|
I could but people probably feel like -crf=12 is like, idk, good
|
|
2021-11-04 02:36:30
|
and distance=12 sucks
|
|
|
|
w
|
2021-11-04 02:36:32
|
ah right
|
|
2021-11-04 02:36:59
|
I think 17-23 is what ffmpeg has as visually lossless
|
|
|
|
Traneptora
|
2021-11-04 02:37:17
|
if I multiply it by `15.0/63.0` then `-distance 1` translates to `-crf 4.2`
|
|
2021-11-04 02:37:27
|
which is probably not quite what we're going for
|
|
2021-11-04 02:37:36
|
I could use something piecewise
|
|
|
|
w
|
2021-11-04 02:38:37
|
haha just make it always d1 except for crf 0 being lossless
|
|
|
|
Traneptora
|
2021-11-04 02:38:45
|
lmao
|
|
2021-11-04 02:39:37
|
like maybe `crf=0:18 -> distance=0:1`, `crf=18:27 -> distance=1:4` and then `crf=27:63 -> distance=4:15`
|
|
2021-11-04 02:39:39
|
what about that?
|
|
2021-11-04 02:40:05
|
which would put `-crf 18` at `-d 1`
|
|
|
|
w
|
2021-11-04 02:41:12
|
I know nothing about jxl lossy aside from d1 = good. But last thing I'll leave is -m/-modular might be important
|
|
|
|
Traneptora
|
2021-11-04 02:41:58
|
do you consider `crf18` to be good?
|
|
|
|
Scope
|
2021-11-04 02:42:10
|
For image encoders I think the most necessary and understandable are -Q and Effort/Speed, but Speed requires remembering different animal names and is longer for scripts and stuff, Effort with numbers is more logical and convenient, like higher number - the more dense compression and slower speed
And -crf is more for video formats, I don't think it's needed for images
|
|
|
|
Traneptora
|
2021-11-04 02:42:13
|
at least comparable enough to `d1` for laypeople
|
|
|
|
w
|
2021-11-04 02:42:18
|
I consider it to be a bit overkill good
|
|
2021-11-04 02:42:29
|
something I'd use for myself kind of good
|
|
|
|
Traneptora
|
2021-11-04 02:42:57
|
but that's similar for d1 tbh, isn't it?
|
|
|
|
w
|
2021-11-04 02:43:15
|
<:pepehmmm:879431793797914625>
|
|
|
|
Traneptora
|
2021-11-04 02:43:16
|
or do you think I should map a higher number down to d1
|
|
2021-11-04 02:43:25
|
the question is which CRF number do you think I should map to d1
|
|
2021-11-04 02:43:28
|
I'm thinking 18 works
|
|
|
|
w
|
2021-11-04 02:43:38
|
maybe someone else has another opinion
|
|
2021-11-04 02:43:52
|
but yeah crf is definitely not needed
|
|
2021-11-04 02:44:09
|
I think it's expected that crf won't work for image formats within ffmpeg
|
|
|
|
Cool Doggo
|
2021-11-04 02:44:10
|
it is hard to compare video quality to image quality
|
|
|
|
w
|
2021-11-04 02:45:36
|
what about avif
|
|
|
|
Scope
|
2021-11-04 02:45:39
|
I don't think that -crf also requires an alias, especially for even newer and different values, it would create much more confusion than any convenience
|
|
|
|
fab
|
|
w
I think -speed can map to effort
|
|
2021-11-04 02:46:02
|
Libavif is called speed
|
|
2021-11-04 02:46:15
|
Is a bad idea to call effort
|
|
2021-11-04 02:46:38
|
There is an open e plus 4 more sounds
|
|
2021-11-04 02:46:51
|
So even difficult to Remember
|
|
|
|
w
|
|
|
fab
|
2021-11-04 02:47:18
|
I like just -s 7
|
|
|
|
Traneptora
|
2021-11-04 02:47:46
|
alright, I'll just use `-distance` and `-q` as the only quality settings
|
|
|
|
Scope
|
2021-11-04 02:47:56
|
Speed with numbers is a bad naming, because it is not always clear that a higher number is more speed or more compression
|
|
|
|
fab
|
2021-11-04 02:48:13
|
Anyway people will use a GUI
|
|
|
|
w
|
2021-11-04 02:49:55
|
Is avif higher speed = faster/lower effort? Like in aom/etc?
|
|
|
|
Cool Doggo
|
2021-11-04 02:50:29
|
yes I think
|
|
|
|
w
|
2021-11-04 02:51:29
|
in that case I'd suggest taking the liberty of mapping speed to inverse of effort or whatever
|
|
|
|
Scope
|
2021-11-04 02:51:36
|
Aomenc has cpu-used, which is also a very confusing option for many people as far as I have seen
|
|
|
|
Cool Doggo
|
2021-11-04 02:52:24
|
too many names for stuff
|
|
|
|
w
|
2021-11-04 02:53:11
|
Yeah this is troublesome
|
|
|
|
Scope
|
2021-11-04 02:53:53
|
I think `-q`, `-Effort`, maybe `-m` for modular mode and `-jxlopts` is enough
|
|
|
|
Traneptora
|
|
w
in that case I'd suggest taking the liberty of mapping speed to inverse of effort or whatever
|
|
2021-11-04 02:54:00
|
think that's what I'm going to do
|
|
|
|
Scope
|
2021-11-04 02:57:14
|
And maybe something like `-lossless`, since many encoders have this, because `-q 100` or `-crf 0` is not always true lossless and people are used to adding `-lossless`
|
|
|
|
Traneptora
|
2021-11-04 02:57:21
|
yea, sure
|
|
2021-11-04 03:03:18
|
How does libjxl map `-q` to `-d` anyway?
|
|
2021-11-04 03:03:32
|
it can't be fully piecewise linear since `-q` has no lower bound
|
|
2021-11-04 03:03:45
|
I know `-q 100` maps to `-d 0`
|
|
|
|
fab
|
2021-11-04 03:05:12
|
Q 100 maps to lossless modular
|
|
2021-11-04 03:05:33
|
At least in current version
|
|
|
|
Scope
|
2021-11-04 03:06:41
|
I think it is better to look in the source or GIMP plugin
<https://github.com/libjxl/libjxl/pull/525>
|
|
2021-11-04 03:08:22
|
Also about modular mode changes
<https://github.com/libjxl/libjxl/pull/808>
|
|
|
|
fab
|
2021-11-04 03:09:13
|
Jxl allows three decimals
|
|
2021-11-04 03:09:22
|
Like 94,157
|
|
2021-11-04 03:09:41
|
But you can choose to oversimplify
|
|
|
|
Traneptora
|
2021-11-04 03:09:56
|
found it https://github.com/libjxl/libjxl/blob/main/plugins/gimp/file-jxl-save.cc#L541
|
|
|
|
fab
|
2021-11-04 03:10:13
|
But dont do please unless asked by other ffmpeg developers
|
|
2021-11-04 03:10:26
|
Dont oversimplify
|
|
|
|
Traneptora
|
2021-11-04 03:11:43
|
I think I'll do this
|
|
|
|
fab
|
|
Scope
For image encoders I think the most necessary and understandable are -Q and Effort/Speed, but Speed requires remembering different animal names and is longer for scripts and stuff, Effort with numbers is more logical and convenient, like higher number - the more dense compression and slower speed
And -crf is more for video formats, I don't think it's needed for images
|
|
2021-11-04 03:11:49
|
Spedd
|
|
2021-11-04 03:12:22
|
Or if you need to add the -
|
|
2021-11-04 03:12:27
|
Better effort
|
|
2021-11-04 03:12:43
|
But wb want effort
|
|
|
w
I think -speed can map to effort
|
|
2021-11-04 03:14:22
|
Yes this are the most common used terms in ffmpeg
|
|
|
|
_wb_
|
2021-11-04 03:15:37
|
It is not intuitive that speed 3 is faster than speed 9
|
|
|
|
Traneptora
|
2021-11-04 03:15:59
|
by "map to" we mean, with conversion
|
|
2021-11-04 03:16:02
|
like how quality -> distance
|
|
2021-11-04 03:16:23
|
I'm adding a separate speed option that is just `effort = 9 - speed`
|
|
|
|
_wb_
|
|
|
Traneptora
|
2021-11-04 03:16:33
|
only the speed option gets the nicknames
|
|
|
|
_wb_
|
2021-11-04 03:16:36
|
That is also confusing
|
|
|
|
Traneptora
|
2021-11-04 03:16:37
|
effort gets the numerals
|
|
2021-11-04 03:16:45
|
you think I should just not have speed at all?
|
|
2021-11-04 03:17:01
|
without the nicknames?
|
|
|
|
_wb_
|
2021-11-04 03:17:10
|
Well if cjxl -s 9 is slow but ffmpeg speed 9 is fast, that is confusing
|
|
|
|
Traneptora
|
2021-11-04 03:17:12
|
I suppose that makes sense, since PNG already has -9 in zlib
|
|
2021-11-04 03:17:15
|
ah
|
|
|
|
_wb_
|
2021-11-04 03:17:56
|
Yes, we were inspired originally by zlib/brotli/webp speed settings, where higher number means slower
|
|
2021-11-04 03:18:42
|
We cannot just remove speed from cjxl because some people are already using it
|
|
2021-11-04 03:19:17
|
But when adding it in ffmpeg, I would either have effort only, or just copy cjxl behavior
|
|
2021-11-04 03:20:13
|
Things get very confusing if the same thing means different things depending on where it is specified (ffmpeg vs imagemagick vs cjxl etc)
|
|
|
|
fab
|
|
Traneptora
by "map to" we mean, with conversion
|
|
2021-11-04 03:20:23
|
He means that you write -s and encoder like AVIF display effort
|
|
2021-11-04 03:22:03
|
But ffmpeg don't want this
|
|
2021-11-04 03:22:17
|
Thats the problem
|
|
|
|
Scope
|
2021-11-04 03:22:31
|
Yep, it is better to port the same options and behavior from cjxl with aliases that will be most convenient and understandable for most people
|
|
|
|
Traneptora
|
2021-11-04 03:30:07
|
```c
if (quality >= 90.0) {
distance = (100.0 - quality) * 0.10;
} else if (quality >= 30.0) {
distance = 0.1 + (100.0 - quality) * 0.09;
} else {
distance = 6.24 + pow(2.5, (30.0 - quality) * 0.145615258) * 0.16;
}
```
|
|
2021-11-04 03:30:14
|
I think this is the function I will use to map quality to distance
|
|
2021-11-04 03:30:21
|
unlike the function in the GIMP plugin, it's continuous
|
|
2021-11-04 03:30:46
|
the GIMP plugin has a discontinuity at `quality=30` because the author probably forgot that `pow(2.5, 0) == 1`
|
|
2021-11-04 03:30:47
|
and not zero
|
|
2021-11-04 03:32:13
|
this maps `100 -> 0`, maps `90 -> 1`, maps `30 -> 6.4` and maps `0` to `14.999999746`
|
|
2021-11-04 03:32:20
|
which is good enough
|
|
|
|
fab
|
2021-11-04 03:51:40
|
q90 is d 1
|
|
|
|
Traneptora
|
2021-11-04 04:54:27
|
er, yes, typo
|
|
|
Scope
And maybe something like `-lossless`, since many encoders have this, because `-q 100` or `-crf 0` is not always true lossless and people are used to adding `-lossless`
|
|
2021-11-04 05:34:10
|
not sure I like `-lossless` mostly because `-distance lossless` will be an option
|
|
|
|
Scope
|
2021-11-04 05:38:54
|
A separate `-lossless` option is not that necessary, although I do not think that `-distance lossless` is a good idea either, as well as other options with different names and values compared to cjxl
|
|
|
novomesk
|
|
_wb_
If you want to add anything there, even a personal subpage on which you just do whatever you want (as long as it's somewhat related to jxl), just make a pull request, or if you want to make more than a one-time change, I can also give you write access
|
|
2021-11-04 05:43:08
|
Is there some problem with my PR? https://github.com/jxl-community/jxl-community.github.io/pull/1
|
|
|
|
_wb_
|
2021-11-04 05:44:57
|
No, just that I am not looking there - I should set up the discord github bot to also watch that, I guess :)
|
|
|
|
Traneptora
|
|
Scope
A separate `-lossless` option is not that necessary, although I do not think that `-distance lossless` is a good idea either, as well as other options with different names and values compared to cjxl
|
|
2021-11-04 05:56:01
|
it's just an alias
|
|
2021-11-04 05:56:16
|
`-distance` is there instead of `-d` because that already means something in FFmpeg
|
|
2021-11-04 05:56:29
|
`-distance 0` is still going to be lossless
|
|
2021-11-04 05:56:47
|
the reason for the alias is that someone who reads the documentation will find it immediately obvious what `d=0` does
|
|
|
|
Scope
|
2021-11-04 06:07:16
|
The best variant is when options and behavior are fully matched with cjxl and change only what cannot be added or would be strange for ffmpeg, in the end personal preferences of all people are very different, even creating lots of additional aliases is not always good and so necessary, this is too much complication and also creates additional confusion if for example I read others' scripts and everyone uses a different option
And also `-distance lossless` sounds weird to me, like `-quality lossless` it could still be, but still, there is no such thing in cjxl and mixing numbers and naming is not a good idea
|
|
|
|
_wb_
|
2021-11-04 06:10:04
|
I agree that syncing option syntax between different tools (cjxl, imagemagick, vips, ffmpeg) is a good thing. We can change cjxl syntax too if that helps convergence (but that should be in a bw-compatible way, i.e. only really add stuff, not remove or change behavior)
|
|
|
|
Scope
|
2021-11-04 06:17:14
|
Yep, even in cjxl it is not so rare that people get confused with `-s` and `-e` options, I do not think that many aliases and variations of one option are more convenient, it is not a programming language or a book where beauty of writing matters, for options it is better to have strictness and clarity
|
|
|
|
Traneptora
|
2021-11-04 07:14:07
|
the question is, if we're not added extra aliases
|
|
2021-11-04 07:14:14
|
so no` -lossless` and no `-distance lossless`
|
|
2021-11-04 07:14:35
|
then people will just have to know that `-distance 0` is lossless.
|
|
2021-11-04 07:14:46
|
That is, unless I add documentation to the encoder, which I totally could do.
|
|
|
|
Scope
|
2021-11-04 07:24:20
|
I think this is ok, a separate -lossless is not so necessary
|
|
|
|
Traneptora
|
2021-11-04 07:25:14
|
```
Encoder libjxl [libjxl JPEG XL encoder]:
General capabilities: threads
Threading capabilities: other
Supported pixel formats: rgb24 rgba rgb48le rgba64le gray ya8 gray16le ya16le grayf32le
libjxl AVOptions:
-effort <int> E..V....... Encoding Effort (from 1 to 9) (default 7)
-distance <float> E..V....... Maximum Butteraugli Distance (quality setting) (from 0 to 15) (default -1)
-modular <int> E..V....... Modular Encoding (from -1 to 1) (default -1)
-jxlopts <string> E..V....... Extra libjxl Options
```
|
|
2021-11-04 07:25:17
|
this is what I have so far
|
|
2021-11-04 07:25:35
|
`-modular -1` means "let the encoder decide"
|
|
2021-11-04 07:25:54
|
`-modular 0` means "Use VarDCT"
and `-modular 1` means "Use Modular"
|
|
2021-11-04 07:26:06
|
if you have a better way to do it, please suggest it
|
|
|
|
monad
|
2021-11-04 07:27:40
|
maybe `-mode` instead?
|
|
|
|
Scope
|
2021-11-04 07:27:40
|
Except adding `-q` and this is basically enough for most cases
|
|
|
|
Traneptora
|
2021-11-04 07:27:52
|
`-q` is a global option
|
|
2021-11-04 07:27:58
|
it's also why `-threads` is missing
|
|
|
|
monad
|
2021-11-04 07:29:59
|
Does `-q` do lossy modular?
|
|
|
|
Traneptora
|
2021-11-04 07:30:28
|
uh, what?
|
|
|
|
monad
|
2021-11-04 07:31:05
|
Specifying modular mode plus q<100
|
|
|
|
Scope
|
2021-11-04 07:31:08
|
Then everything is fine, and about the modular mode, it is better to match cjxl, so `-modular` is also okay
``` -m, --modular
Use the modular mode (lossy / lossless)```
|
|
|
|
monad
|
2021-11-04 07:34:25
|
Well, it doesn't match cjxl. It's the same name but different behavior.
|
|
|
|
Scope
|
2021-11-04 07:34:46
|
``` -Q luma_q[,chroma_q], --mquality=luma_q[,chroma_q]
[modular encoding] lossy 'quality' (100=lossless, lower is more lossy)```
And then there are some changes
<https://github.com/libjxl/libjxl/pull/808>
|
|
2021-11-04 07:37:41
|
As far as I remember, ffmpeg has a slightly different design for individual options, except for those that can be set in `-jxlopts`
|
|
|
|
_wb_
|
2021-11-04 07:38:13
|
Best to match the library api, because in the end that is also what cjxl will do once it migrates to the api
|
|
|
|
Traneptora
|
2021-11-04 07:44:09
|
wait, in lossy modular do you accept distance?
|
|
2021-11-04 07:44:18
|
or do you convert it back into Q
|
|
|
|
monad
|
2021-11-04 07:48:40
|
in cjxl, distance is ignored when modular is specified and output is lossless
|
|
|
|
Scope
|
2021-11-04 07:48:52
|
Only -Q if nothing has changed in the meantime, modular mode does not use butteraugli, that's why I also said that using -Q is a more universal solution
|
|
|
|
Traneptora
|
2021-11-04 07:49:11
|
what if output is lossy?
|
|
2021-11-04 07:49:21
|
unless that's what you meant
|
|
|
|
monad
|
2021-11-04 07:50:02
|
if you just specify `-m` `-d 1` distance is ignored and output is lossless instead
|
|
2021-11-04 07:50:26
|
for lossy modular, you need `-q` or `-Q`
|
|
|
|
Scope
|
2021-11-04 07:53:31
|
Btw, will ffmpeg be able to use Jpeg recompression or is it also like ImageMagick only able to transfer pixel data?
|
|
|
|
_wb_
|
2021-11-04 07:54:45
|
-m -q < 100 gets translated to -Q something (which does quantized Squeeze) and -c 0 (XYB), -m -q 100 gets translated to -c 1 (no color transform) with no Squeeze
|
|
|
|
Traneptora
|
|
Scope
Btw, will ffmpeg be able to use Jpeg recompression or is it also like ImageMagick only able to transfer pixel data?
|
|
2021-11-04 07:56:01
|
atm only pixel data. jpeg recompression could be implemented as a bitstream filter but it's much lower priority than just getting this off the ground
|
|
2021-11-04 08:00:34
|
```c
if (avctx->global_quality > 0) {
ctx->mquality = avctx->global_quality / 118.0;
ctx->distance = quality_to_distance(ctx->mquality);
} else {
ctx->mquality = distance_to_mquality(ctx->distance);
}
```
|
|
2021-11-04 08:00:35
|
this could work
|
|
2021-11-04 08:00:47
|
I populate both the mquality and the distance fields in the local context
|
|
2021-11-04 08:00:59
|
and then choose which one to use based on whether or not it's modular mode
|
|
2021-11-04 08:01:16
|
does libjxl always use vardct for lossy, by default?
|
|
2021-11-04 08:01:36
|
and does lossless vardct exist?
|
|
|
|
Scope
|
|
|
Traneptora
|
2021-11-04 08:03:14
|
I see, so I only need `-modular` to force modular for lossy output
|
|
|
|
_wb_
|
2021-11-04 08:08:30
|
Vardct is always lossy
|
|
2021-11-04 08:10:01
|
It's probably almost always best for lossy too, but maybe for some specific cases lossy modular can be better (or lossy delta palette for that matter, which is another kind of lossy modular besides the default quantized squeeze kind)
|
|
2021-11-04 08:12:28
|
Atm cjxl switches to lossy modular by default (without adding -m) at very low -q settings (e.g. negative values)
|
|
|
|
Traneptora
|
2021-11-04 08:17:34
|
does it ever switch to modular at very high values for distance?
|
|
|
|
_wb_
|
2021-11-04 08:20:07
|
I dunno but if it doesn't, it probably would make sense to change it so -q and -d behave the same way, just with a different scale
|
|
|
|
Traneptora
|
2021-11-04 08:20:42
|
this is currently what I have for parsing:
|
|
2021-11-04 08:20:46
|
```c
if (!ctx->modular && ctx->distance > 0.0 && avctx->flags & AV_CODEC_FLAG_QSCALE){
ctx->distance = quality_to_distance((float)avctx->global_quality / FF_QP2LAMBDA);
}
if (!ctx->modular && ctx->distance > 14.0){
ctx->modular = 1;
ctx->mquality = distance_to_mquality(ctx->distance);
}
if (ctx->distance <= 0.0){
ctx->modular = 1;
ctx->mquality = 100.0;
}
if (ctx->modular && ctx->mquality <= 0.0){
if (avctx->flags & AV_CODEC_FLAG_QSCALE) {
ctx->mquality = (float)avctx->global_quality / FF_QP2LAMBDA;
} else {
ctx->mquality = distance_to_mquality(ctx->distance);
}
}
```
|
|
2021-11-04 08:20:51
|
I think this covers all the options I want.
|
|
|
|
_wb_
|
2021-11-04 08:21:10
|
Not sure if switching to modular is still a useful thing though, it might be better to do vardct with more upsampling at the higher distances
|
|
|
|
Traneptora
|
2021-11-04 08:21:23
|
does *libjxl* do the same thing?
|
|
2021-11-04 08:21:26
|
or is this a cjxl thing?
|
|
|
|
_wb_
|
2021-11-04 08:21:36
|
It's atm a cjxl thing
|
|
|
|
Traneptora
|
2021-11-04 08:21:39
|
ah, okay
|
|
2021-11-04 08:22:01
|
and these are the AVOptions I have at the moment
|
|