|
fab
|
2021-09-21 12:01:46
|
https://www.computerweekly.com/blog/CW-Developer-Network/Developers-get-image-conscious-for-royalty-free-image-codec-JPEG-XL?&cf=1
|
|
|
w
|
2021-09-21 12:01:57
|
i dont regularly touch it but only had to since i dont have a laptop to bring to school
|
|
|
|
fab
|
|
|
w
|
2021-09-21 12:02:07
|
waiting for next surface
|
|
|
|
fab
|
2021-09-21 12:02:31
|
Sorry if i waste server bandwidth
|
|
2021-09-21 12:02:56
|
Do you know what it means
|
|
2021-09-21 12:05:43
|
Is it a rhetorical figure?
|
|
2021-09-21 12:06:49
|
The noun Is It dr. Jon sneyers dev of Flif?
|
|
2021-09-21 12:09:04
|
|
|
2021-09-21 12:09:32
|
Lol nathaniel of av1 server not knowing jon sneyers
|
|
2021-09-21 12:10:35
|
than PNG (with the best optimization) in terms of overall efficiency even when encoding at the fastest speeds, moreover, it is more efficient than other existing and new image formats https://i.imgur.com/zVsWSf9.png
|
|
2021-09-21 12:11:31
|
Archived post you can't add a comment
|
|
2021-09-21 12:12:08
|
I searched on Google jpeg xl bitstream freeze
|
|
2021-09-21 12:13:21
|
|
|
2021-09-21 12:13:50
|
Amhwha thats what advanced user of libjxl search
|
|
|
Traneptora
|
2021-09-21 12:17:04
|
it's a 12 year old drive
|
|
2021-09-21 12:17:10
|
2.5inch laptop drive
|
|
2021-09-21 12:17:15
|
wwwwww's thing sounds about right
|
|
|
Fraetor
|
2021-09-22 10:20:58
|
Was there anything the core devs wanted to included in JPEG XL that they couldn't for non-technical reasons, such as patent restrictions?
|
|
|
_wb_
|
2021-09-22 10:59:13
|
I wouldn't have minded to have more complete spot color descriptions (though it's a bit of a niche thing, only relevant for the printing industry), but the commonly used methods are Pantone and co, which are all non-royalty-free, so that's a no-go.
|
|
2021-09-22 11:02:40
|
If HEVC and VVC weren't the patent minefield they are, I might have looked at them to see if they have something worth adding to jxl. I don't know if there would have been anything, since I didn't even look.
|
|
|
spider-mario
|
2021-09-22 01:43:34
|
ICtCp looks rather neat
|
|
2021-09-22 01:43:36
|
but then so is XYB
|
|
|
Scope
|
2021-09-23 10:42:19
|
<https://github.com/libjxl/libjxl/pull/627>
Also, would a complete rewrite be needed to make the lossy palette work in XYB?
And is it possible to get any additional improvements or flexibility from XYB or something like choosing a different quality or is it always less effective overall?
|
|
|
|
_wb_
|
2021-09-23 10:53:08
|
Not a complete rewrite, just a custom delta table instead of the default one would be needed.
|
|
|
|
Scope
|
2021-09-23 11:29:55
|
Hmm, perhaps a separate custom table for grayscale or b/w images might also be effective? 🤔
|
|
|
|
_wb_
|
2021-09-23 11:31:11
|
I don't think delta palette is particularly useful for single-channel images
|
|
|
|
Scope
|
2021-09-23 11:36:35
|
I mean that the images may not be exactly grayscale or b&w, but look like that and some lossy palette conversion to a more limited range can improve the result without a noticeable visual difference, although for b&w maybe some other techniques have to be used
And since the lossy palette is not very effective for such images, perhaps it would be nice to add an automatic switch to regular lossless with a reduced color depth 🤔
|
|
|
|
_wb_
|
2021-09-23 01:11:08
|
funky lossy preprocessing could be done for specific cases, though it's always a bit risky to do that kind of thing, because you're effectively applying different encode techniques depending on heuristics, meaning you can have an image that looks one way when compressed, then you change a few pixels and it falls on the other side of the heuristic and looks a potentially quite different way when compressed
|
|
|
|
Scope
|
2021-09-23 01:29:22
|
Yes, on the other hand there may be banding in grayscale images, which is not so easy to solve or at least just switch to true lossless mode, which is safer
Because there are many grayscale and b/w images in general, both from past times such as faxes or non-color scans and now in the form of manga, comics, art, UI or medical images (but for them lossy methods can be dangerous)
|
|
|
|
fab
|
2021-09-23 01:49:50
|
can jxl do modular lossy and vardct simultaneously
|
|
2021-09-23 01:50:00
|
or delta + vardct
|
|
|
|
_wb_
|
2021-09-23 02:00:09
|
Yes - at least if you use two frames
|
|
|
monad
|
|
_wb_
if you really want best lossless and time is no concern, `-d 0 -e 9 -E 3 -I 1` and then choosing best of `-g {0,1,2,3}` and also try `-d 0 -e 9 -I 0 -P 0 -g 3 --patches 0` in case it's one of those where png is hard to beat
|
|
2021-09-23 10:29:24
|
So, I actually tried this on a set of 273 PNGs of various types of content (much of the photographic content likely former JPGs).
```wins encoder
9 cjxl -d 0 -e 9 -E 3 -I 1 -g 0
23 cjxl -d 0 -e 9 -E 3 -I 1 -g 1
187 cjxl -d 0 -e 9 -E 3 -I 1 -g 2
35 cjxl -d 0 -e 9 -I 0 -P 0 -g 3 --patches 0
18 cwebp -z 9 -metadata all
1 pingo -s9 -nostrip```
|
|
|
BlueSwordM
|
2021-09-24 12:11:51
|
`-z 9` is always lossless.
|
|
|
|
w
|
|
Cool Doggo
|
2021-09-24 12:18:11
|
webp loses that much when lossy?
|
|
|
|
Scope
|
2021-09-24 12:19:13
|
Depends on content
|
|
|
|
BlueSwordM
|
|
Cool Doggo
webp loses that much when lossy?
|
|
2021-09-24 12:45:14
|
JXL lossless is stronger in WebP in photographic images.
|
|
|
|
monad
|
|
BlueSwordM
JXL lossless is stronger in WebP in photographic images.
|
|
2021-09-24 05:10:14
|
In this test, the WebP wins were predominantly on text content and other repetitive content (e.g. tile-based game screenshots). `-I 0 -P 0 --patches 0` won on nearly all pixel art. Other content included photos, 2D & 3D digital art, graphics, and mixed content.
|
|
|
|
fab
|
2021-09-25 09:15:51
|
you can't convert a jxl in jxl so there's no sense in ffmpeg integration
|
|
2021-09-25 09:16:26
|
if you want to make a GUI that's limitating
|
|
2021-09-25 09:16:36
|
you want things to work as more as possible
|
|
|
diskorduser
|
2021-09-25 10:05:49
|
Convert jxl to jxl? What?
|
|
|
|
fab
|
|
diskorduser
Convert jxl to jxl? What?
|
|
2021-09-25 02:02:25
|
Ffmpeg wont implement jxl without strict 2
|
|
|
|
diskorduser
|
2021-09-25 02:03:07
|
Ok. I don't know what strict 2 means.
|
|
|
|
BlueSwordM
|
|
diskorduser
Ok. I don't know what strict 2 means.
|
|
2021-09-25 02:06:33
|
`-strict 2` is an option to be less strict when it comes to any external pipes or something.
|
|
|
|
fab
|
2021-09-25 02:08:44
|
Experimental
|
|
2021-09-25 02:09:04
|
That cant be enabled
|
|
|
|
Cool Doggo
|
|
fab
you can't convert a jxl in jxl so there's no sense in ffmpeg integration
|
|
2021-09-25 04:49:12
|
I think ffmpeg would decode to pixel data before encoding, like I believe imagemagick does
|
|
2021-09-25 04:50:45
|
it would be the same as if you decoded jxl to png then encoded it again with cjxl
|
|
|
|
fab
|
2021-09-25 04:58:05
|
It would be slow
|
|
|
|
_wb_
|
2021-09-25 07:25:24
|
At least ffmpeg has a concept of remuxing without transcoding
|
|
2021-09-25 07:25:39
|
Imagemagick only has pixel buffers
|
|
|
|
Deleted User
|
2021-09-25 07:32:39
|
Reencoding a whole video is typically a big deal compared to an image encode, so remuxing should be natural. Though this is not often enough the case.
|
|
|
|
_wb_
|
2021-09-25 07:39:35
|
Yes, also video has other tracks like audio and subtitles which makes it more useful to have remuxing as a basic concept
|
|
2021-09-25 07:40:29
|
In images, there's basically exiftool/exiv2 to 'remux' metadata
|
|
2021-09-25 07:41:21
|
And there's jpegtran to do operations to a jpeg without full decode/encode
|
|
2021-09-25 07:41:56
|
But afaik about anything else is always decode to pixels, do stuff, encode from pixels
|
|
2021-09-25 07:44:30
|
With jxl there are quite a few relevant 'remuxing' things that could/should be implemented, like adding an alpha channel to an existing jpeg, overlaying something on an existing jpeg, doing reversible crops (just adjusting frame header crop/offset without changing any pixel data, so you can do a crop that you can still undo later)
|
|
2021-09-25 07:44:42
|
etc etc
|
|
|
|
fab
|
2021-09-25 08:09:40
|
Adding audio to a jxl animation
|
|
2021-09-25 08:24:18
|
How to do
|
|
2021-09-25 08:25:06
|
Is there a tool or you need to add something in the container
|
|
2021-09-25 08:25:35
|
for %i in (C:\Users\Utente\Documents\332q\*.png) do cjxl -d 4.768 -s 5 -I 0.868 --gaborish=0 --intensity_target=244 --use_new_heuristics %i %i.jxl
|
|
2021-09-25 08:25:42
|
i use those settings
|
|
2021-09-25 08:26:11
|
also i need a serious GUI that does that to me (Mp4 to jxl)
|
|
2021-09-25 08:26:19
|
with the encoder i desire
|
|
2021-09-25 08:26:56
|
that's located here
|
|
2021-09-25 08:26:57
|
https://artifacts.lucaversari.it/libjxl/libjxl/2021-09-24T19%3A45%3A40Z_d245c7e2b1380bf99af849aeb55ca1806881ba0c/
|
|
2021-09-25 08:27:10
|
if i haven't this i won't like jpeg xl
|
|
2021-09-26 05:45:00
|
What Will be presented at image ready 1.2
|
|
|
|
_wb_
|
2021-09-26 05:48:58
|
I dunno yet, you tell me!
|
|
|
|
fab
|
2021-09-26 06:03:14
|
Better quality
|
|
|
|
monad
|
|
monad
So, I actually tried this on a set of 273 PNGs of various types of content (much of the photographic content likely former JPGs).
```wins encoder
9 cjxl -d 0 -e 9 -E 3 -I 1 -g 0
23 cjxl -d 0 -e 9 -E 3 -I 1 -g 1
187 cjxl -d 0 -e 9 -E 3 -I 1 -g 2
35 cjxl -d 0 -e 9 -I 0 -P 0 -g 3 --patches 0
18 cwebp -z 9 -metadata all
1 pingo -s9 -nostrip```
|
|
2021-09-26 09:36:27
|
Ugh, I just found a typo in my script which meant the encodes labeled '-g 3' were actually using `-g 2` (a side effect of the size comparison meant they were all bucketed under that one label). I was curious why `-g 2` had no apparent wins, but I missed this. I guess I'll rerun later, it was only like 8 hours of encode time ^^.
|
|
|
|
_wb_
|
2021-09-28 01:46:14
|
i'm on a code cleanup spree
|
|
2021-09-28 01:47:10
|
https://app.codecov.io/gh/libjxl/libjxl
|
|
|
|
Scope
|
2021-09-28 02:14:18
|
Percentage, is that code that is actually used?
|
|
|
|
_wb_
|
2021-09-28 02:52:05
|
that's code covered by the unit tests
|
|
2021-09-28 02:52:53
|
not including the jxl_test roundtrip tests, some things are only tested there but it takes too long to compute coverage including that for every commit
|
|
2021-09-28 02:53:31
|
then there's also code tested by fuzzing, which is mostly useful for checking that it doesn't explode on bad bitstreams
|
|
2021-09-28 02:54:41
|
so the aim is not to get it to 100%, because it's overkill to write unit tests for e.g. all kinds of corrupt bitstreams
|
|
2021-09-28 02:55:18
|
but it does help to spot dead code or things that aren't well-tested
|
|
|
|
Traneptora
|
2021-09-29 05:54:37
|
Is there a spec for the container format?
|
|
2021-09-29 05:54:43
|
not the bitstream itself, I can use the library for that
|
|
2021-09-29 05:54:54
|
but I'd like to at least parse the headers myself for the purpose of the libavformat wrapper I'm writing
|
|
|
|
veluca
|
2021-09-29 08:10:35
|
it's just ISOBMFF
|
|
2021-09-29 08:11:19
|
https://github.com/libjxl/jxl-rs this does fundamentally that (specifically this file: https://github.com/libjxl/jxl-rs/blob/main/src/bmff.rs) + parsing of some of the JXL "inner" headers
|
|
|
eddie.zato
|
2021-09-30 07:23:11
|
From the `exiv2` repo:
> Attention is drawn to the possibility that bmff support may be the subject of patent rights. Exiv2 shall not be held responsible for identifying any or all such patent rights. Exiv2 shall not be held responsible for the legal consequences of the use of this code.
https://github.com/Exiv2/exiv2#2-19
|
|
2021-09-30 07:23:27
|
And from here:
> There has been a lot of discussion in Team Exiv2 concerning the legality of reading this file. I don’t believe it’s illegal to read metadata from a container. I believe it’s illegal to decode proprietary encoded data stored in the image. However the metadata is not protected in anyway.
https://exiv2.org/book/index.html#BMFF
|
|
2021-09-30 07:27:24
|
Can bmff cause problems for jxl adoption? I'm not an expert in this.
|
|
|
The_Decryptor
|
2021-09-30 07:35:31
|
I doubt it, but I'm not a lawyer
|
|
|
|
veluca
|
2021-09-30 07:36:04
|
IIRC ISOBMFF is old enough that this ought not be an issue
|
|
|
|
The_Decryptor
|
2021-09-30 07:36:10
|
It's over 20 years old, and used for other formats that have seen adoption in apps like browsers (Mozilla have their own parser for it in Firefox)
|
|
|
|
veluca
|
2021-09-30 07:36:23
|
the *standard* is from 2000 IIRC, but it should be even older than that
|
|
2021-09-30 07:36:53
|
some of the things people put in BMFF *might* be patented, but AFAIU none of that is necessary to correctly read a JXL file
|
|
2021-09-30 07:37:42
|
probably some of the details of how HEIF puts image information in ISOBMFF have some sort of patent
|
|
2021-09-30 07:37:54
|
but AVIF uses the HEIF container too, and people feel OK with that, so...
|
|
2021-09-30 07:38:16
|
(HEIF is also ISOBMFF based)
|
|
|
|
eddie.zato
|
2021-09-30 07:40:10
|
Ok, I get it. 😄
|
|
|
|
Scope
|
2021-09-30 10:22:07
|
This might improve compression for emojis sets and such?
https://github.com/libjxl/libjxl/pull/651
|
|
|
|
veluca
|
2021-09-30 10:28:58
|
possibly
|
|
2021-09-30 10:29:05
|
but only small ones
|
|
|
|
_wb_
|
2021-09-30 11:32:30
|
most of them would be smaller than 256x256, so yes, should help a little bit
|
|
2021-09-30 11:33:05
|
HEIF does actually have Nokia patents on it that are not expired yet since HEIF is relatively recent
|
|
2021-09-30 11:34:44
|
ISOBMFF itself is old enough (and also never had patent claims on it afaik), and the subset of it that we use in jxl is very limited (basically just the box syntax itself, the `ftyp` box, and the `Exif` box — all other boxes are jxl-specific)
|
|
|
|
Scope
|
2021-09-30 11:36:34
|
Yep, it helped noticeably, now jxl is better than webp2 on the emoji set <:Hypers:808826266060193874>
|
|
2021-09-30 11:45:59
|
Before
|
|
2021-09-30 11:46:06
|
After
|
|
|
|
veluca
|
2021-09-30 11:53:40
|
cooooool 😄
|
|
|
|
Scope
|
2021-09-30 12:11:07
|
And an additional 5% better with `--premultiply`, but sometimes the images are 50-90% larger, it would also be nice in the future to make an option to automatically choose a more efficient mode
|
|
2021-09-30 12:12:16
|
|
|
2021-09-30 12:15:03
|
Like, for some reason, this (bad for --premultiply):
|
|
|
|
fab
|
|
Scope
|
|
2021-09-30 01:22:05
|
the image is the same
|
|
|
|
Scope
|
2021-09-30 02:37:59
|
<@!794205442175402004> So it's time to update the results before ImageReady presentation 🤔
<https://docs.google.com/presentation/d/1LlmUR0Uoh4dgT3DjanLjhlXrk_5W2nJBDqDAMbhe8v8/edit#slide=id.gae1d3c10a0_0_28>
|
|
2021-09-30 02:38:23
|
<https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/>
|
|
|
|
_wb_
|
2021-09-30 02:41:22
|
good point, updating...
|
|
|
|
w
|
2021-09-30 02:50:23
|
but 300kb difference over 1mb/s will only take 17 transfers to be better so imo speed shouldn't be considered
|
|
2021-09-30 02:51:05
|
encode speed* it's something that only happens once
|
|
2021-09-30 02:57:31
|
Also for that spreadsheet it's all slowest settings
|
|
2021-09-30 02:58:08
|
I think even the PNG it gets as slow
|
|
2021-09-30 02:58:14
|
with pingo + ect
|
|
|
|
Scope
|
2021-09-30 02:59:25
|
I didn't measure encoding/decoding speed because it's very difficult to make proper comparisons, very much influenced by CPU type and SIMD support, number of cores, types of images, some large images in lossless are encoded much faster than other small images and for more reliable result it is desirable to have a large set of different images.
In addition, for lossless is not so easy to improve efficiency without increasing the complexity of the encoding especially for images, rather than general data, besides the encoders are still in an active development stage
|
|
2021-09-30 03:00:23
|
But yes, that would be useful, although for now it's more important for lossy encoding
|
|
|
|
w
|
2021-09-30 03:00:38
|
It at least contains regular -s 9 which is what I use, and 5% smaller than the next lowest is pretty big a difference
|
|
|
|
Scope
|
2021-09-30 03:02:57
|
I have one of the old charts, although the set and size have changed a lot since then
|
|
2021-09-30 03:03:57
|
But, it is very large and versatile and the total encoding with different options and formats can take months
|
|
2021-09-30 03:05:47
|
Also it is not only the slowest speeds (at least for JXL), but I have hidden some results for other formats in order not to overcomplicate the summary
|
|
2021-09-30 03:15:41
|
And as it was already discussed, decoding speed is much more important, as well as just the ability to compress more, for example PNG even with the best optimizers, even with a theoretical search of all possible options, which would take years, it is impossible to significantly improve the result
Or there are symmetric codecs where encoding speed is higher and better than JXL and other practical formats, but they have the same decoding speed as encoding and if waiting 10 minutes to encode one image is not so rare, then waiting 10 minutes for decoding to view one image excludes this format from suitable for practical use
|
|
2021-09-30 03:21:47
|
-
Also about speed comparisons (but, this is an old version of Jpeg XL, it is almost purely photographic set and many encoders have been tuned only for this set, they perform very poorly on other data)
<https://globalcompetition.compression.ru/#leaderboards>
|
|
2021-09-30 03:26:14
|
Or (but somewhat outdated and without JXL)
http://qlic.altervista.org/LPCB.html
|
|
2021-09-30 03:41:04
|
And also some of my old comparisons
|
|
|
|
w
|
2021-09-30 03:42:31
|
you should make one for real time taken and total time
|
|
|
|
Scope
|
2021-09-30 03:54:09
|
Yes, but anyway, this is also outdated data and also it will not give much more information because it will need exactly the same hardware configuration and about the same type of images, because as I said there are very complex images that can compress tens of minutes or hours at maximum settings, while others even with higher resolution and size are encoded in seconds at the same settings (because I guess some things are skipped and not tried)
And roughly a comparison with WebP as the better known format also gives a fair idea of the speeds
|
|
|
|
fab
|
2021-09-30 07:44:59
|
we are 22:00 here 21:44
|
|
2021-09-30 07:45:26
|
are 21 hours and 15 minutes
|
|
2021-09-30 07:45:32
|
needed for the imageready event
|
|
|
|
_wb_
|
2021-10-01 12:16:05
|
from the av1 discord, some musings on the biggest possible jxl image:
|
|
2021-10-01 12:16:09
|
the DC image of a 1 Exapixel image is a 16 Petapixel image
the DC image of the DC image would be 256 Gigapixel
the DC of the DC of the DC would be 4 Gigapixel
and the DC of that (the max amount of recursive DC images the bitstream alllows) would be 64 Megapixel
so the 4-th level DC frame would be 64 megapixel, and its own DC (if it's VarDCT encoded) would be 1 megapixel
the main image would consist of 2^44 AC groups though, so that's a TOC with that many entries
permuting that guy for center-first reordering will not be fun, best to stick to default group order
in default group order, you could figure out the relevant offsets for the AC of a region of interest without needing to load the whole TOC in memory (though you would need to scan through it all)
theoretically it should be possible to decode a random screenfull of 1:1 image with bounded memory
in practice it's probably a better idea to tile up your exapixel image in a billion independent gigapixel chunks
assuming 1bpp, the compressed image would be 128 petabytes, which is likely not something you want to have as a single file
|
|
|
|
Traneptora
|
2021-10-01 05:47:04
|
```http
GET /img/avatar/512.png HTTP/3
Host: thebombzen.moe
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:92.0) Gecko/20100101 Firefox/92.0
Accept: image/jxl,image/webp,*/*
Accept-Language: en-US
Accept-Encoding: gzip, deflate, br
DNT: 1
...
```
Looks like firefox is now requesting `image/jxl`
|
|
2021-10-01 05:47:17
|
in the `Accept:` header
|
|
2021-10-01 05:53:07
|
not sure why, since firefox 92 does not support it
|
|
2021-10-01 05:53:18
|
maybe cause I enabled the "image.jxl.enabled" flag
|
|
2021-10-01 05:53:26
|
but it doesn't actually support JXL images yet, since it's not nightly
|
|
|
|
_wb_
|
2021-10-01 06:00:41
|
It's a bit strange that the flag is there even when it's a build without jxl support
|
|
2021-10-01 06:01:43
|
Anyway, I hope they can add it to non-nightly too, at least for desktop builds I see no reason not to add it...
|
|
2021-10-01 06:02:48
|
(for mobile, binary size might be a bit more sensitive, though even there, it's less than 200kb in android chrome so not a huge problem imo)
|
|
|
|
fab
|
2021-10-01 08:19:34
|
jxl new heuristics
|
|
2021-10-01 08:19:36
|
finally
|
|
2021-10-01 08:19:46
|
i'm seeing an increase of file size
|
|
|
novomesk
|
2021-10-01 08:19:57
|
I remember I did some experiments to convert APNG to JXL using cjxl. It works fine for small APNG files. However, cjxl seems to decompress all frames from APNG to memory first and after that it starts JXL encoding. It can easily fill all my RAM+swap with decoded APNG frames. The transcoding via swap was very slow.
If possible, I suggest keeping just one frame in memory and send it progressively to JXL encoder after each decompressed APNG frame (to prevent high memory usage).
I understand that encoding big/long animations is not top important for JXL, the suggestion is just my €0.02
|
|
|
|
fab
|
2021-10-01 08:19:58
|
maybe corrupt or different compiling of jamaika build
|
|
2021-10-01 08:20:04
|
they are 26000 kb each
|
|
2021-10-01 08:20:09
|
even the vvc builds
|
|
2021-10-01 08:20:22
|
so that could be fault of jamaika
|
|
2021-10-01 08:20:36
|
i will test anyway
|
|
2021-10-01 08:21:04
|
i'm getting extremely low bpp
|
|
2021-10-01 08:21:17
|
0.107 bpp 0.089 bpp range with s 7 d 1
|
|
2021-10-01 08:21:33
|
so those are new heuristics
|
|
2021-10-01 08:22:05
|
0.570 bpp
|
|
2021-10-01 08:22:14
|
0.469 bpp the biggest files
|
|
2021-10-01 08:23:21
|
is extremely slow 0,39 mpx s
|
|
2021-10-01 08:23:34
|
the one before was 0,89 mpx s
|
|
2021-10-01 08:24:21
|
ram consumption it gone to 70 mb to 300 mb awkardly
|
|
2021-10-01 08:24:36
|
30 mb 278 mb
|
|
2021-10-01 08:24:40
|
it's random
|
|
2021-10-01 08:42:12
|
using q 91.7 with s 7 has no sense
|
|
2021-10-01 08:42:21
|
higher quality than 90 you need s 8
|
|
2021-10-01 08:42:41
|
with new heuristics that could change
|
|
2021-10-01 08:42:56
|
but of course the artifacts are weaker
|
|
2021-10-01 08:49:49
|
310 photos is super slow
|
|
2021-10-01 08:49:59
|
i'm converting over a hour
|
|
2021-10-01 08:55:13
|
it finished after 35 minutes of 2400x1080 jpg 8 bit
|
|
2021-10-01 08:55:22
|
to jxl s 7 d 1
|
|
2021-10-01 08:55:44
|
quality is not sure what lee user likes
|
|
2021-10-01 08:56:26
|
grey look like there is an added noise, red and brown look similar and a bit dithered like a better camera unlocked
|
|
2021-10-01 08:56:42
|
i didn't see the image now i will try
|
|
2021-10-01 08:57:36
|
|
|
2021-10-01 08:57:41
|
lee won't like
|
|
2021-10-01 08:58:07
|
is a jpg source
|
|
2021-10-01 08:59:56
|
jxl looks better than original
|
|
2021-10-01 09:00:02
|
the clock is sharper
|
|
2021-10-01 09:00:13
|
is more neat
|
|
2021-10-01 09:00:47
|
moving to benchmarks
|
|
2021-10-01 09:44:04
|
<@!794205442175402004> how insert opus audio in jxl 0.7.0 86eed13
|
|
2021-10-01 09:44:11
|
is worth to try animation
|
|
2021-10-01 09:44:26
|
i will try should i start with speed 7?
|
|
2021-10-01 09:45:00
|
could you make a tool compatible with 0.7.0 86eed13
|
|
2021-10-01 09:50:25
|
can APNG have audio
|
|
2021-10-01 10:02:31
|
the encoder don't encode animation
|
|
2021-10-01 10:04:38
|
is a .png animation supported <@!794205442175402004>
|
|
2021-10-01 10:06:16
|
|
|
|
|
_wb_
|
2021-10-01 11:25:18
|
apng is half supported as input - it's not fully implemented
|
|
2021-10-01 11:25:38
|
(so if it fails, it means the apng uses a dispose mode that wasn't implemented yet in the loader)
|
|
|
|
fab
|
2021-10-01 11:33:05
|
Can apng have audio
|
|
2021-10-01 11:33:10
|
In opus format
|
|
2021-10-01 11:33:27
|
Or heaac v1
|
|
2021-10-01 11:34:04
|
Is supported by the apng specification?
|
|
2021-10-01 11:37:19
|
Jxl animation encoding is ram intensive
|
|
2021-10-01 11:38:47
|
My computer is suffering
|
|
2021-10-01 11:38:59
|
I think im gonna damage It
|
|
2021-10-01 11:39:31
|
S 1 d 1 with 720 900 30 frame per seconds
|
|
2021-10-01 11:39:43
|
Ah 30 frame thats the fault
|
|
2021-10-01 11:40:44
|
Task manager dont say more the CPU usage
|
|
|
Jyrki Alakuijala
|
2021-10-01 11:44:24
|
If you repeat this graph with normal sized data, it will be less impressive
|
|
|
|
fab
|
2021-10-01 11:48:48
|
Is doing the same operations
|
|
2021-10-01 11:52:34
|
It's starting to use cpu After 19 minutes
|
|
2021-10-01 11:52:51
|
It did 16 mb
|
|
2021-10-01 11:52:57
|
Is not efficient
|
|
2021-10-01 11:53:27
|
19 minutes for 426 KB to exactly 16 mb
|
|
2021-10-01 11:53:40
|
0.441 bpp per frame
|
|
2021-10-01 11:54:30
|
1:8 compression normal compression
|
|
2021-10-01 11:54:47
|
There s no intra sure
|
|
2021-10-01 11:56:42
|
Is not working at all
|
|
2021-10-01 11:57:35
|
1.2. GB in qimgv and still isnt playing
|
|
2021-10-01 11:58:00
|
After 34 seconds of wait
|
|
2021-10-01 12:00:03
|
48 CPU usage
|
|
2021-10-01 12:00:43
|
Considering is 30 fps and High bitrate Is good
|
|
2021-10-01 12:04:55
|
|
|
2021-10-01 12:16:07
|
Now is using no cpu
|
|
|
|
_wb_
|
2021-10-01 02:25:48
|
Slightly improved jpeg recompression: https://github.com/libjxl/libjxl/pull/679
|
|
2021-10-01 02:26:24
|
makes -e 9 both better than before and 10x faster
|
|
|
Fox Wizard
|
2021-10-01 02:29:45
|
<a:ajxl:806626687130533959><a:FurretSpeed:832307660035850310>
|
|
|
|
Scope
|
2021-10-01 03:16:24
|
With `--strip` is it possible to do the same optimizations as MozJpegTran?
Because currently, when pre-optimizing with it, before JXL re-compression, the result was slightly better
|
|
|
|
fab
|
2021-10-01 03:51:34
|
IM CHARGING THE PHONE TO WATCH IMAGEREADY
|
|
|
|
Traneptora
|
2021-10-01 03:57:17
|
ISOBMFF being the Mp4 container?
|
|
|
|
veluca
|
2021-10-01 04:42:38
|
yup
|
|
2021-10-01 04:42:41
|
I think
|
|
|
|
_wb_
|
2021-10-01 04:59:28
|
yes, to the point that the registration authority for isobmff boxes is called mp4ra: http://mp4ra.org/#/
|
|
2021-10-01 05:04:43
|
ImageReady is about to start
|
|
|
|
Traneptora
|
|
_wb_
makes -e 9 both better than before and 10x faster
|
|
2021-10-01 06:31:06
|
I have some losslessjpeg JXLs. Can I recompress them or do I have to djxl them back to jpeg, and then cjxl them?
|
|
2021-10-01 06:31:18
|
To take advantage of this better ratio
|
|
|
|
_wb_
|
2021-10-01 06:31:48
|
easiest will be to djxl and cjxl again
|
|
2021-10-01 06:32:32
|
the difference will not be huge though, but would be interesting to see results on a different corpus
|
|
|
|
fab
|
2021-10-01 08:29:21
|
honestly at -d 2.079 -s 8 jxl is already enough
|
|
2021-10-01 08:30:06
|
for encoding instagram files
|
|
|
|
Traneptora
|
2021-10-02 01:34:21
|
I'm thinking of taking my photographic library taken with my phone
|
|
2021-10-02 01:34:22
|
and doing that
|
|
2021-10-02 01:34:40
|
would you consider that to be a similar dataset to the one you have?
|
|
|
stephengold
|
2021-10-02 03:50:26
|
is anyone working on a Java API for decoding JXL images?
|
|
|
|
w
|
2021-10-02 03:54:20
|
there's already a JNI wrapper in libjxl
|
|
|
|
stephengold
|
2021-10-02 03:54:48
|
cool thanks
|
|
|
|
w
|
2021-10-02 03:55:48
|
i was gonna question why it's in libjxl, but it's under tools so 🤷
|
|
|
|
stephengold
|
2021-10-02 04:00:34
|
libjxl is also the name of the repo, which includes the tools so ...
|
|
|
|
Traneptora
|
2021-10-02 04:20:37
|
I check MP4RA and there's four different jpeg XL registered types
|
|
2021-10-02 04:20:40
|
how do I handle that?
|
|
2021-10-02 04:20:49
|
Signature I figure is the first atom
|
|
2021-10-02 04:20:57
|
but what about the codestream/frame index/partial codestream?
|
|
2021-10-02 04:21:00
|
how does those atoms work?
|
|
|
|
_wb_
|
|
Traneptora
would you consider that to be a similar dataset to the one you have?
|
|
2021-10-02 06:12:44
|
No. The set I used are older images, not taken with a phone camera.
|
|
|
Traneptora
how does those atoms work?
|
|
2021-10-02 06:17:16
|
That's what part 2 of the spec describes, but basically jxlc contains the jxl image data (you can take the contents of that box and it's a valid jxl file too)
|
|
2021-10-02 06:18:32
|
jxlp contains part of a codestream; if you concatenate all the jxlp boxes you get the full codestream
|
|
2021-10-02 06:19:37
|
jxli contains an index of offsets to keyframes (not implemented yet to write that or to take advantage of it)
|
|
|
|
Traneptora
|
|
_wb_
That's what part 2 of the spec describes, but basically jxlc contains the jxl image data (you can take the contents of that box and it's a valid jxl file too)
|
|
2021-10-02 07:10:51
|
oh, so if I'm having libjxl handle the codestreams, then jxlc's contents are literally just, that
|
|
2021-10-02 07:11:04
|
a jxl codestream?
|
|
2021-10-02 07:11:55
|
so if I'm writing a demuxer, I can ignore jxli if it is present, concatenate all the jxlp's contents together into a codestream, or just take jxlc and that's a codestream
|
|
2021-10-02 07:12:09
|
you mentioned a JXL codestream is a valid JXL file
|
|
2021-10-02 07:12:28
|
so JXL is basically a codestream format that is optionally contained inside ISOBMFF as above?
|
|
|
|
_wb_
|
2021-10-02 07:12:32
|
yes
|
|
2021-10-02 07:12:51
|
the jxlp's also have a counter as the first 4 bytes of the payload iirc
|
|
|
|
Traneptora
|
2021-10-02 07:12:52
|
do JXLP have to be consecutive? for example, in PNG, partial IDATS are required to be consecutive
|
|
|
|
_wb_
|
2021-10-02 07:13:12
|
let me check the spec
|
|
|
|
Traneptora
|
2021-10-02 07:13:14
|
counter? like, byte offset counter?
|
|
|
|
_wb_
|
2021-10-02 07:13:33
|
no a counter that starts with 0 and every next jxlp increments it
|
|
|
|
Traneptora
|
2021-10-02 07:13:35
|
yea, having the spec would be easier but I'm aware that you cant just drop it cause ISO/IEC rules
|
|
2021-10-02 07:13:37
|
oh, like
|
|
2021-10-02 07:13:40
|
number of them
|
|
|
|
_wb_
|
2021-10-02 07:13:54
|
with a special value for "this is the last one", iirc
|
|
|
|
Traneptora
|
2021-10-02 07:13:57
|
do they have to be in order? like, can you go 0 -> 2 -> 1 and make the decoder re-order them
|
|
2021-10-02 07:14:05
|
or is that forbidden
|
|
2021-10-02 07:14:27
|
I don't know why you would want to do that, but idk?
|
|
2021-10-02 07:14:49
|
how would I recognize a JXL codestream as a JXL file?
|
|
2021-10-02 07:15:04
|
is there a header magic?
|
|
2021-10-02 07:15:54
|
or something? like PNG's `137 80 78 71 13 10 26 10`
|
|
2021-10-02 07:15:57
|
which begin every PNG file
|
|
|
|
_wb_
|
2021-10-02 07:16:14
|
|
|
2021-10-02 07:16:30
|
a jxl codestream starts with 0xFF0A
|
|
2021-10-02 07:16:49
|
only two bytes of magic but it _should_ be unique/enough
|
|
|
|
Traneptora
|
2021-10-02 07:17:03
|
just wondering if there's a risk of false positives with that
|
|
2021-10-02 07:17:23
|
0A is a newline, isn't it? that's printable
|
|
|
|
_wb_
|
2021-10-02 07:17:34
|
someone tried to find false positives in a big archive of stuff and didn't find any
|
|
|
|
Traneptora
|
2021-10-02 07:17:43
|
ah, okay
|
|
|
|
_wb_
|
2021-10-02 07:18:02
|
all JPEG markers are of the form FFxx
|
|
|
|
Traneptora
|
|
|
_wb_
|
2021-10-02 07:18:19
|
they keep track of which ones have already been used and for what
|
|
2021-10-02 07:18:33
|
FF0A was assigned to be "start of jpeg xl codestream"
|
|
|
Traneptora
do they have to be in order? like, can you go 0 -> 2 -> 1 and make the decoder re-order them
|
|
2021-10-02 07:20:02
|
looks like they do have to be in order
|
|
|
|
Traneptora
|
2021-10-02 07:20:30
|
yea, it looks like it says they have to be in order but they are not required to be consecutive
|
|
2021-10-02 07:20:45
|
you can insert metadata in the middle
|
|
|
|
_wb_
|
2021-10-02 07:21:05
|
I think we only added the counter in case some dumb isobmff handler would reorder boxes
|
|
|
|
Traneptora
|
2021-10-02 07:21:26
|
as a way to "fix" it
|
|
2021-10-02 07:21:30
|
but that's not supposed to happen at all I take it
|
|
|
|
_wb_
|
2021-10-02 07:21:33
|
the jxlp counter has to be incremented by 1, but there can be non-jxlp boxes in between
|
|
2021-10-02 07:21:47
|
nope, and if it happens, it's not a valid jxl file at that point
|
|
2021-10-02 07:21:56
|
just an invalid one that can still be fixed
|
|
|
|
Traneptora
|
2021-10-02 07:22:11
|
yea, it looks like it has to be incremented by 1, except the last one must also have the sign bit set
|
|
2021-10-02 07:22:14
|
in addition to the others
|
|
2021-10-02 07:22:21
|
`+ (1 << 31)`
|
|
|
|
_wb_
|
2021-10-02 07:22:36
|
yes, basically first bit of counter is a boolean `is_last`
|
|
|
|
Traneptora
|
2021-10-02 07:22:46
|
makes sense
|
|
2021-10-02 07:22:56
|
4-byte size is probably unnecessary
|
|
2021-10-02 07:23:04
|
but that probably doesn't add much in the grand scheme of things
|
|
|
|
_wb_
|
2021-10-02 07:23:28
|
yeah it's overkill but in the isobmff way of doing things, it's often like that
|
|
2021-10-02 07:23:43
|
it's not quite aiming for minimal container overhead
|
|
|
|
Traneptora
|
2021-10-02 07:23:52
|
and once I have a jpeg codestream I've identified, can just send it to libjxl which can handle it
|
|
|
|
_wb_
|
2021-10-02 07:24:02
|
well you can also send the container to libjxl
|
|
|
|
Traneptora
|
2021-10-02 07:24:12
|
I'm writing a parser in libavformat
|
|
|
|
_wb_
|
2021-10-02 07:24:14
|
libjxl can find the jxlc or deal with jxlp fine
|
|
|
|
Traneptora
|
2021-10-02 07:24:27
|
which has its own ISOBMFF demuxer
|
|
|
|
_wb_
|
2021-10-02 07:24:33
|
ah i see
|
|
|
|
Traneptora
|
2021-10-02 07:24:39
|
it'll be easier to handle that internally since it has its own autodetection
|
|
2021-10-02 07:24:47
|
and its own autodetection will go "oh, this is ISOBMFF"
|
|
2021-10-02 07:24:56
|
and that way I check for the signature atom
|
|
2021-10-02 07:25:11
|
and if it exists then I go "nice" and send the codestream to jxl
|
|
2021-10-02 07:25:17
|
sending the whole file over after that is more complicated
|
|
2021-10-02 07:25:46
|
speaking of which, the signature atom has no payload, it just exists as the first atom to mark the file as a JXL file, right?
|
|
2021-10-02 07:27:05
|
also is there ever necessary information in JXLI
|
|
2021-10-02 07:27:10
|
like can it be ignored if you don't want to bother?
|
|
|
|
_wb_
|
2021-10-02 07:49:20
|
jxli is optional and never necessary
|
|
2021-10-02 07:49:33
|
It's just there to facilitate seeking
|
|
2021-10-02 07:50:21
|
And yes, the signature atom is just an empty marker that acts like magic
|
|
|
|
fab
|
2021-10-02 08:45:19
|
The photo look like q 77.2 and slightly underexposed
|
|
2021-10-02 08:46:56
|
In some parts like q 71.3 but Better exposed
|
|
2021-10-02 08:48:47
|
Max 81.93 It spends 8% on detail and 2 on colour
|
|
|
|
Traneptora
|
2021-10-02 01:51:24
|
that's the goal
|
|
|
|
Scope
|
|
Traneptora
yea, having the spec would be easier but I'm aware that you cant just drop it cause ISO/IEC rules
|
|
2021-10-02 01:56:11
|
https://discord.com/channels/794206087879852103/804324493420920833/885951456954441748
|
|
|
|
Traneptora
|
|
Scope
https://discord.com/channels/794206087879852103/804324493420920833/885951456954441748
|
|
2021-10-02 01:56:50
|
I would like to avoid that if possible
|
|
2021-10-02 01:56:59
|
which is why I'm asking these questions
|
|
|
|
Scope
|
2021-10-02 02:03:52
|
I don't think it means something bad, the format is free and open, as well as its sources, it's not even a kind of reverse engineering of some proprietary formats, a lot of which were added to ffmpeg in this way
|
|
|
|
_wb_
|
2021-10-02 02:19:59
|
I hope we can soon have drafts of the 2nd edition publicly available, which will be way more useful than the drafts of the 1st edition
|
|
|
|
veluca
|
2021-10-02 06:47:12
|
in the meantime, if you can digest rust, jxl-rs can be more readable for header purposes...
|
|
|
|
Traneptora
|
2021-10-02 11:47:34
|
What would you recommend to be a reasonable chunks size?
|
|
2021-10-02 11:48:06
|
if I'm muxing a *large* JXL (think, more than 10k by 10k in pixels)
|
|
2021-10-02 11:48:16
|
how many bytes per jxlp chunk do you think is reasonable
|
|
2021-10-02 11:48:18
|
(or intended)
|
|
2021-10-02 11:48:36
|
since while it's tempting to just write *one* jxlc chunk and never write jxlp chunks
|
|
2021-10-02 11:48:45
|
might be a good idea sometimes
|
|
2021-10-03 12:06:37
|
Also are there any file extensions other than `.jxl` that I need to care about
|
|
2021-10-03 12:56:47
|
What does `XL` stand for in Jpeg XL
|
|
2021-10-03 12:56:55
|
I've heard "Extended Foo" but idr what the Foo is
|
|
|
|
Cool Doggo
|
2021-10-03 02:09:21
|
i think ive seen "Extended Life"
|
|
|
|
monad
|
|
Traneptora
What does `XL` stand for in Jpeg XL
|
|
2021-10-03 05:31:26
|
> The etymology of the name "XL" is as follows: JPEG has called all its new standards since j2k something that starts with an X: XR, XT, XS (S for speed, since it is very fast and ultra-low-latency), and now XL. The L is supposed to mean Long term, since the goal is to make something that can replace the legacy JPEG and last as long as it did.
<https://news.ycombinator.com/item?id=22270148>
|
|
|
|
_wb_
|
|
Traneptora
since while it's tempting to just write *one* jxlc chunk and never write jxlp chunks
|
|
2021-10-03 06:10:53
|
Yes, just jxlc is fine, there is no advantage to splitting it in jxlp from the libjxl point of view
|
|
2021-10-03 06:15:34
|
One use case for jxlp might be cases where you want to have largeish metadata available in a progressive/streaming scenario: then you could do enough jxlp to get a preview, then the metadata, then a jxlp with the rest of the image.
|
|
|
|
Traneptora
|
2021-10-03 06:49:42
|
doing more than one jxlp seems like it's mostly useful to cap the buffer size
|
|
2021-10-03 06:50:31
|
like, okay I've got 4096 kiB, I'll just dump it and keep going
|
|
2021-10-03 06:50:59
|
that way the libjxl client doesn't have to read the entire thing into memory and then write the payload size to the jxlc header
|
|
|
|
_wb_
|
2021-10-03 07:06:47
|
the current encoder isn't made yet to do things in a chunked way, so it'll always have the whole thing in memory
|
|
|
|
veluca
|
|
Traneptora
that way the libjxl client doesn't have to read the entire thing into memory and then write the payload size to the jxlc header
|
|
2021-10-03 07:36:45
|
Thing is that the last box can have a size of 0 which means "till EOF"
|
|
2021-10-03 07:37:02
|
So you don't even have to use jxlp for that
|
|
|
|
Scope
|
|
Scope
With `--strip` is it possible to do the same optimizations as MozJpegTran?
Because currently, when pre-optimizing with it, before JXL re-compression, the result was slightly better
|
|
2021-10-03 12:06:12
|
Hmm, yep, with the new changes at `-s 9` the MozJpegTran optimization is no longer giving any improvements or even a little worse sometimes 🤔
|
|
|
|
Fraetor
|
2021-10-03 02:17:04
|
https://github.com/libjxl/libjxl/blob/main/doc/format_overview.md#jpeg-bitstream-reconstruction-data
So if you were to remove the JPEG Bitstream Reconstruction Data you would still be able to reconstruct a pixel perfect JPEG? Just without any other metadata?
|
|
|
|
Traneptora
|
|
veluca
Thing is that the last box can have a size of 0 which means "till EOF"
|
|
2021-10-03 02:22:50
|
oh so if the jxlc size is 0 it's literally just "the rest of the file"
|
|
|
Fraetor
https://github.com/libjxl/libjxl/blob/main/doc/format_overview.md#jpeg-bitstream-reconstruction-data
So if you were to remove the JPEG Bitstream Reconstruction Data you would still be able to reconstruct a pixel perfect JPEG? Just without any other metadata?
|
|
2021-10-03 02:25:16
|
I can't speak about that block, but cjxl --strip creates jxl files that are imagedata exact, not filedata exact
|
|
2021-10-03 02:25:35
|
without --strip it's file data
|
|
|
|
_wb_
|
2021-10-03 02:32:25
|
The jxl image data doesn't care about the way the jpeg data is encoded: anything you do losslessly with it using jpegtran (changing the progressive scan script or the huffman tables) doesn't matter for what the jxl codestream will encode, which is just quantized dct coeffs encoded with jxl entropy coding
|
|
2021-10-03 02:35:01
|
So if you use --strip, it doesn't matter if you used jpegtran optimization first or not.
The jpeg bitstream itself does matter for the jbrd box, which does get encoded by default. It contains details about whatever is in the jpeg bitstream that is not dct coeffs, like padding between segments, whatever APP markers, etc etc.
|
|
2021-10-03 02:36:28
|
Jpegtran will change those details, for better or worse (typically better because it will tend to remove or simplify stuff)
|
|
|
lithium
|
2021-10-03 02:41:53
|
cjxl --strip option also process Adobe APP 14 segment?
|
|
|
|
_wb_
|
2021-10-03 02:46:58
|
--strip removes anything that is not needed to show the image
|
|
2021-10-03 02:47:21
|
Including the data to reconstruct the jpeg file
|
|
2021-10-03 02:48:22
|
But of course it will first read the jpeg file and check also that marker to know whether the image is rgb, ycbcr or cmyk/ycck
|
|
|
|
lithium
|
2021-10-03 02:50:27
|
I understand, thank you 🙂 ,
I have some png gAMA question on <#794206087879852106> could you teach me about this?
|
|
2021-10-03 05:19:42
|
I hate windows cmd code page issue 😢
I think windows terminal or powershell core or 8.3 is solution.
(not recommend old version powershell)
> https://github.com/libjxl/libjxl/issues/683
|
|
|
|
_wb_
|
2021-10-03 06:26:29
|
<@!532010383041363969> PTAL at https://github.com/libjxl/libjxl/pull/684
|
|
2021-10-03 06:27:48
|
I think it's a ~0.5% improvement, maybe a bit more at d4
|
|
2021-10-03 06:28:56
|
but I'm too tired to look at images, and since this affects quality, it would be good if people could check for regressions
|
|
|
|
lithium
|
|
_wb_
but I'm too tired to look at images, and since this affects quality, it would be good if people could check for regressions
|
|
2021-10-03 06:32:56
|
affects quality, it's mean new quality improve PR? <:BlobYay:806132268186861619>
|
|
|
|
_wb_
|
2021-10-03 06:34:09
|
Nah, I don't think it will improve quality
|
|
2021-10-03 06:34:25
|
It improves density by almost 1%
|
|
2021-10-03 06:34:39
|
I hope without affecting quality
|
|
2021-10-03 06:35:08
|
Ugh I have been silly
|
|
2021-10-03 06:35:21
|
I have been looking at d2 images to try to spot problems
|
|
2021-10-03 06:35:47
|
But d<2.5 has only lossless changes in this PR
|
|
2021-10-03 06:36:28
|
So <@532010383041363969> no need to look at anything under ~~d2.5~~ d2 (also not above d4.5)
|
|
|
|
lithium
|
2021-10-03 06:36:30
|
You work too hard, need some rest 🙂
|
|
|
|
Scope
|
2021-10-03 07:39:06
|
But, improving density actually also improves quality for the same size
|
|
|
|
_wb_
|
|
_wb_
Ugh I have been silly
|
|
2021-10-03 07:40:50
|
Actually I am double silly, it's only d<2 that has only lossless changes, d2 does actually change slightly
|
|
2021-10-04 07:58:21
|
If it really causes no quality regression, it's a nice improvement of 0.5 to 1%
|
|
|
|
fab
|
2021-10-04 08:32:12
|
|
|
2021-10-04 08:32:18
|
for %i in (D:\immagini\DACONVERTIRE\FB\*.jpg) do cjxl -j -s 7 -d 4.586 %i %i.jxl
for %i in (D:\immagini\DACONVERTIRE\INSTAC\*.jpg) do cjxl -j -q 78.198 -s 8 --use_new_heuristics %i %i.jxl
for %i in (D:\immagini\DACONVERTIRE\SCREENSHOT\Screenshots\*.png) do cjxl -s 7 -d 1 %i %i.jxl
|
|
|
|
diskorduser
|
2021-10-05 01:20:45
|
<#840831132009365514>
|
|
|
|
_wb_
|
2021-10-05 11:44:24
|
https://github.com/libjxl/libjxl/pull/689
|
|
2021-10-05 11:45:09
|
anyone who feels like doing some manual image peeping: please compare your favorite images before and after that pull request
|
|
2021-10-05 11:46:50
|
I _think_ this is a huge improvement in density (4-5% smaller) without too much negative impact on quality (in fact even some positive impact), but I might be too optimistic here
|
|
2021-10-05 11:47:38
|
(when evaluating a tweak you did yourself, it's hard to remain unbiased)
|
|
|
|
lithium
|
2021-10-05 12:34:03
|
Thank you for your work, I like quality improve 🙂
|
|
|
|
Scope
|
2021-10-05 01:17:32
|
Is this commit standalone and doesn't need any previous?
|
|
|
|
_wb_
|
2021-10-05 01:25:02
|
in the PR it also includes commits from two previous PRs that aren't merged yet, you don't really need those, only the change in enc_group.cc from the new commit there is what causes the big changes
|
|
2021-10-05 01:25:22
|
Lee, is there any test image you want me to try it on?
|
|
|
|
Scope
|
2021-10-05 01:32:13
|
This images set may also be okay for comparison with different quality (mostly for ringing and edge problems)
https://discord.com/channels/794206087879852103/803645746661425173/857401138805473281
|
|
|
|
_wb_
|
2021-10-05 01:34:44
|
ah cool i'll try it on those
|
|
|
|
Scope
|
2021-10-05 01:35:40
|
And
https://discord.com/channels/794206087879852103/803645746661425173/857408503861477408
|
|
|
|
Traneptora
|
2021-10-05 03:45:09
|
What does "zero threads" mean?
|
|
2021-10-05 03:45:17
|
does it mean zero worker threads? cause clearly it's doing something
|
|
2021-10-05 03:45:45
|
and if it means worker threads, does that mean "4 threads" is actually 4 threads, or is it 4 worker threads and one control thread?
|
|
|
|
_wb_
|
2021-10-05 03:49:01
|
Zero threads means no fork, worker running on main thread
|
|
|
|
lithium
|
|
_wb_
Lee, is there any test image you want me to try it on?
|
|
2021-10-05 03:50:32
|
Sorry so late to reply,
I build ac_quant_deadzone_tweaks this branch, and take some test on my gitlab issue anime image
(blue,red plane have gradient and sharp edge).
I think this PR improve quality and reduce some tiny artifacts, Thank you for your work :),
but I still can find some tiny artifacts on red plane and have gradient and sharp edge(on -d 0.5 -e 9),
tiny artifacts spread on red ribbon plane and gradient area also have tiny artifacts,
this tiny artifacts not easy to see, but if I flicker test two image, I can find this.
> -d 0.5 -e 9
> djxl --bits_per_sample=16
|
|
|
|
Traneptora
|
2021-10-05 03:50:44
|
So, does 4 threads mean 4 worker threads and one main/control thread?
|
|
2021-10-05 03:50:47
|
i.e. 5?
|
|
2021-10-05 03:50:50
|
or does it mean actually four
|
|
2021-10-05 03:51:30
|
is `num_threads=1` then going to really be two threads?
|
|
2021-10-05 03:52:30
|
also, if `num_threads` is nonzero, does the main thread do anything other than I/O, or does it also do computationally intensive things?
|
|
|
|
_wb_
|
2021-10-05 03:53:02
|
<@179701849576833024> do you know?
|
|
|
|
veluca
|
2021-10-05 03:54:01
|
if num_threads is nonzero the main thread should not be doing much - and either way the total number of threads that are actually doing something is at most `num_threads`
|
|
2021-10-05 03:54:16
|
(when the workers are active, the main thread is just waiting for them)
|
|
|
|
Traneptora
|
2021-10-05 03:54:36
|
so what is the main difference between `num_threads=0` and `num_threads=1`
|
|
2021-10-05 03:54:55
|
if `num_threads=1` does that mean the main thread handles I/O but offloads the cpu to the worker thread?
|
|
|
|
_wb_
|
2021-10-05 03:55:04
|
So it's either num_threads workers and main thread dispatching jobs, or no workers and main thread doing stuff that hasn't been parallelized yet
|
|
|
|
Traneptora
|
2021-10-05 03:55:26
|
I see, so the `num_threads` is the number of worker threads, which if it's nonzero, do all the work
|
|
2021-10-05 03:55:45
|
which makes me wonder if there's a practical advantage to doing `1` over `0`, which to me feels like it would be I/O
|
|
2021-10-05 03:55:52
|
if that's how it works, at least
|
|
|
|
_wb_
|
2021-10-05 03:56:43
|
I think 1 is slightly worse than 0
|
|
|
|
veluca
|
2021-10-05 03:56:47
|
1 is almost never useful IIRC
|
|
|
|
Traneptora
|
2021-10-05 04:00:44
|
I think what I'll do for the wrapper is "number of worker threads"
|
|
2021-10-05 04:00:55
|
and make it so `threads=0` becomes "auto" (which is standard in FFmpeg)
|
|
2021-10-05 04:01:33
|
`threads=1` will be translated into libjxl `num_threads=0`
|
|
2021-10-05 04:01:41
|
and `threads > 1` will be passed as-is
|
|
|
|
lithium
|
2021-10-05 04:10:39
|
I think gaborish file size inflate issue also happen on new version libjxl.
> https://discord.com/channels/794206087879852103/794206087879852106/889546375208005653
|
|
|
|
Traneptora
|
2021-10-05 04:16:46
|
what's the max thread count, btw?
|
|
2021-10-05 04:19:53
|
It looks like it's `num_threads=32708` on my system
|
|
2021-10-05 04:20:00
|
not sure if that's universal tho
|
|
|
|
_wb_
|
2021-10-05 04:27:31
|
More threads than (virtual) cpu cores does not really make sense
|
|
|
|
Traneptora
|
2021-10-05 04:27:42
|
I'm not asking about *sense* I'm asking about absolute max
|
|
2021-10-05 04:27:45
|
is there a library limitation
|
|
2021-10-05 04:28:02
|
that is, `djxl --num_threads=32708` on my system works, but 32709 gives me aborted(core dumped)
|
|
|
|
_wb_
|
2021-10-05 04:28:13
|
No idea
|
|
|
|
Traneptora
|
2021-10-05 04:28:18
|
so you don't have one
|
|
2021-10-05 04:28:32
|
I'd like to put a cap in the wrapper as a sanity check
|
|
2021-10-05 04:28:47
|
do you think `(1 << 14)` is a reasonably futureproof thread count max (i.e. 16,384)
|
|
|
|
_wb_
|
2021-10-05 04:29:05
|
It will certainly be a while before anyone will hit that cap
|
|
|
|
Traneptora
|
2021-10-05 04:29:10
|
```
$ djxl --num_threads=32708 test.jxl test.png
JPEG XL decoder v0.7.0 c31dca7 [AVX2]
Read 412559 compressed bytes.
Decoded to pixels.
512 x 512, 0.35 MP/s [0.35, 0.35], 1 reps, 32708 threads.
Allocations: 196545 (max bytes in use: 8.977437E+10)
$ djxl --num_threads=32709 test.jxl test.png
JPEG XL decoder v0.7.0 c31dca7 [AVX2]
Read 412559 compressed bytes.
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
Aborted (core dumped)
```
|
|
2021-10-05 04:29:38
|
I'll cap it at 16,384 just for sanity
|
|
|
|
_wb_
|
2021-10-05 04:29:44
|
Likely the OS has caps on number of threads too
|
|
|
|
Traneptora
|
2021-10-05 04:30:11
|
```c
#define FF_LIBJXL_THREADS_MAX (1 << 14)
static size_t libjxl_get_threadcount(int threads)
{
if (threads > FF_LIBJXL_THREADS_MAX)
return FF_LIBJXL_THREADS_MAX;
if (threads <= 0)
return av_cpu_count();
if (threads == 1)
return 0;
return threads;
}
```
|
|
2021-10-05 04:30:17
|
I'll do this
|
|
2021-10-05 04:30:20
|
seems fine
|
|
2021-10-05 04:30:55
|
ideally it should never be higher than the max, or negative, or ffmpeg will reject it earlier
|
|
2021-10-05 04:30:57
|
but good to be safe
|
|
2021-10-05 05:21:08
|
What's this? how fix?
|
|
2021-10-05 05:21:13
|
```
CMake Warning:
Manually-specified variables were not used by the project:
JPEGXL_ENABLE_GIMP_SAVING
```
|
|
2021-10-05 05:21:16
|
is it worth fixing?
|
|
2021-10-05 05:51:35
|
also do threads ever change the output bitstream?
|
|
|
|
_wb_
|
|
|
veluca
|
|
Traneptora
What's this? how fix?
|
|
2021-10-05 06:42:16
|
ignore it, perhaps delete your build dir
|
|
2021-10-05 06:42:32
|
the max on the # threads is whatever libstdc++ decides is the max
|
|
|
|
Traneptora
|
2021-10-05 06:45:57
|
so there is no max basically
|
|
2021-10-05 06:46:19
|
I'll just set 16384 as a pseudo-cap for sanity reasons
|
|
2021-10-05 06:46:34
|
the user requesting that many getting dumb stuff is 'their fault' territory
|
|
|
|
Scope
|
2021-10-05 10:09:51
|
<@!794205442175402004> <https://github.com/libjxl/libjxl/pull/689#issuecomment-934914665>
> What would be the easiest/best way to get d>2 to increase bpp so it stays closer to what it is now?
I think the easy way is to encode with `-d x` with a normal encoder and then encode with the same size from the resulting images (`--target_size=`) with these changes (I do the same)
|
|
|
|
_wb_
|
2021-10-06 05:07:48
|
Yes, but I think jyrki meant: change the pull request so e.g. -d 4 before and after that pull request keeps having roughly the same bpp
|
|
|
|
veluca
|
2021-10-06 08:09:53
|
Go to enc_adaptive_quantization.cc
|
|
2021-10-06 08:10:06
|
There's a kQuantAc constant I believe
|
|
2021-10-06 08:10:11
|
Tweak that
|
|
2021-10-06 08:10:18
|
(also one for DC)
|
|
|
|
_wb_
|
2021-10-06 11:32:31
|
did that in https://github.com/libjxl/libjxl/pull/691 (also further tweaked the deadzone thresholds themselves)
|
|
|
|
Scope
|
2021-10-06 01:02:09
|
Overall a little better than #689 (although, the artifacts also shift to other areas and it's not as unambiguous)
https://discord.com/channels/794206087879852103/895000336325017641/895293511119802460
But, this is only #691, it's not entirely clear to me whether the other changes bring improvements on top of this
|
|
|
|
fab
|
2021-10-07 08:00:59
|
37,2% 42,11% of appeal it needs to improve
|
|
2021-10-07 08:01:44
|
currently even after 2 days it can be at level of 15012020 libaom
|
|
2021-10-07 08:03:04
|
the new heuristics don't improve the results for journals for chromaticity for photographic quality for explicit content body face for dvd scans for animations
|
|
2021-10-07 08:03:13
|
for selfies
|
|
2021-10-07 08:07:50
|
also 0.6.0 is good enough it deserves an automatic distance chooser and settings chooser
|
|
2021-10-07 08:08:11
|
currently is marked as bugged build so nobody would bother
|
|
2021-10-07 08:08:27
|
even in squoosh it got skipped they don't update codecs in 2 month
|
|
2021-10-07 08:10:44
|
though new pulls request they are making use way less blurring
|
|
|
|
lithium
|
2021-10-08 08:26:53
|
I think still have some comic,manga file size bloat issue on cjxl,
this issue happen on grayscale or black and white image, less color,
and jpg also have this issue.
> png-4299x6071, 4 bits Indexed grayscale, 8 color,
> - 932,710
> jxl d1.0-e8 4,169,996
> jxl d1.0-e7 4,312,103
> avif q15-s4 614,828
> avif q7-s4 644,127
>
> avifenc --min 0 --max 63 --minalpha 0 --maxalpha 0 -d 10 -s 4 -j 12 -a end-usage=q -a cq-level=7
> -a color:enable-dnl-denoising=0 -a color:denoise-noise-level=5 -a color:sharpness=2 -a color:qm-min=0
> https://discord.com/channels/794206087879852103/794206087879852106/889546375208005653
|
|
2021-10-08 11:42:46
|
# Denoise test
> denoise-pingo-s9-version-4299x6071, 4 bits, Indexed grayscale, 5 color
> - 414,581
> jxl d1.0-e8 3,693,225
> jxl d1.0-e7 3,807,163
> jxl d1.0-e7-gaborish=0 2,214,185
> jxl d1.0-e8-gaborish=0 2,135,824
> avif q15-s4 304,185
> avif q7-s4 309,857
>
> avifenc --min 0 --max 63 --minalpha 0 --maxalpha 0 -d 10 -s 4 -j 12 -a end-usage=q -a cq-level=7
> -a color:enable-dnl-denoising=0 -a color:denoise-noise-level=5 -a color:sharpness=2 -a color:qm-min=0
|
|
|
|
diskorduser
|
|
lithium
I think still have some comic,manga file size bloat issue on cjxl,
this issue happen on grayscale or black and white image, less color,
and jpg also have this issue.
> png-4299x6071, 4 bits Indexed grayscale, 8 color,
> - 932,710
> jxl d1.0-e8 4,169,996
> jxl d1.0-e7 4,312,103
> avif q15-s4 614,828
> avif q7-s4 644,127
>
> avifenc --min 0 --max 63 --minalpha 0 --maxalpha 0 -d 10 -s 4 -j 12 -a end-usage=q -a cq-level=7
> -a color:enable-dnl-denoising=0 -a color:denoise-noise-level=5 -a color:sharpness=2 -a color:qm-min=0
> https://discord.com/channels/794206087879852103/794206087879852106/889546375208005653
|
|
2021-10-08 03:29:14
|
Is the source lossless without any artifacts?
|
|
|
|
lithium
|
|
diskorduser
Is the source lossless without any artifacts?
|
|
2021-10-08 06:33:24
|
I guess already denoise source can define to near-lossless, and I don't see any can visible artifacts with my eyes(maybe still have some very tiny noise),
I think if didn't compare quality, it's not necessary input lossless source,
I mean already denoise source also is acceptable source.
|
|
|
lithium
# Denoise test
> denoise-pingo-s9-version-4299x6071, 4 bits, Indexed grayscale, 5 color
> - 414,581
> jxl d1.0-e8 3,693,225
> jxl d1.0-e7 3,807,163
> jxl d1.0-e7-gaborish=0 2,214,185
> jxl d1.0-e8-gaborish=0 2,135,824
> avif q15-s4 304,185
> avif q7-s4 309,857
>
> avifenc --min 0 --max 63 --minalpha 0 --maxalpha 0 -d 10 -s 4 -j 12 -a end-usage=q -a cq-level=7
> -a color:enable-dnl-denoising=0 -a color:denoise-noise-level=5 -a color:sharpness=2 -a color:qm-min=0
|
|
2021-10-09 11:15:48
|
# Denoise test, another page.
> JPEG XL encoder v0.7.0 233db00 [AVX2,SSE4,Scalar]
> denoise-pingo-s9-version-4299x6071, 4 bits, Indexed grayscale, 5 color
> - 962,047
> jxl d1.0-e8 4,316,108
> jxl d1.0-e8-gaborish=0 2,440,659
> avif q15-s4 314,645
> avif q7-s4 329,353
>
> // another page case lossless, already denoise source
> jxl -m-q100-e8-g3 962,047 => 573,441
>
> // previous case lossless, already denoise source
> jxl -m-q100-e8-g3 414,581 => 466,847
I don't know, why manga content will happen this...😢
|
|
|
|
fab
|
2021-10-09 01:37:13
|
also i'm considering
|
|
2021-10-09 01:37:14
|
for %i in (C:\Users\Utente\Documents\a\*.jpg) do cjxl -j -s 7 -d 0.11 --use_new_heuristics %i %i.jxl
-q 71.288 -s 3 av1enc
|
|
2021-10-09 01:37:56
|
can av1enc give valid avif file that can be read on mozilla firefox and android 12 <@!111445179587624960>
|
|
2021-10-09 01:38:02
|
not avienc, av1enc
|
|
2021-10-09 01:38:08
|
the tool included in wp2
|
|
2021-10-09 01:38:16
|
basically this
|
|
2021-10-09 01:38:17
|
https://forum.doom9.org/showthread.php?t=174300&page=23
|
|
2021-10-09 01:43:41
|
i think from q 82.12 and up jxl wins
|
|
2021-10-09 01:44:03
|
17,88
|
|
|
|
lithium
|
|
lithium
# Denoise test, another page.
> JPEG XL encoder v0.7.0 233db00 [AVX2,SSE4,Scalar]
> denoise-pingo-s9-version-4299x6071, 4 bits, Indexed grayscale, 5 color
> - 962,047
> jxl d1.0-e8 4,316,108
> jxl d1.0-e8-gaborish=0 2,440,659
> avif q15-s4 314,645
> avif q7-s4 329,353
>
> // another page case lossless, already denoise source
> jxl -m-q100-e8-g3 962,047 => 573,441
>
> // previous case lossless, already denoise source
> jxl -m-q100-e8-g3 414,581 => 466,847
I don't know, why manga content will happen this...😢
|
|
2021-10-09 05:08:30
|
Update avif q15-s4 and avif q7-s4 for another page lossy test.
This test not prove avif quality is better, I only compare file size,
I think jxl probably have some issue for manga or grayscale,
because this issue also happen on jpg.
|
|
|
|
diskorduser
|
2021-10-09 05:12:22
|
Does avif performs well on Manga with more complex brush strokes?
|
|
|
|
lithium
|
2021-10-09 05:13:59
|
I don't know, but I don't want file size bloat too much...
|
|
|
|
_wb_
|
2021-10-09 05:15:07
|
Afaik avif performs well on ligne claire style - it is good at avoiding ringing at those edges
|
|
2021-10-09 05:16:08
|
In different drawing/painting styles, things are likely a bit different
|
|
|
|
lithium
|
2021-10-09 05:32:17
|
I trust jxl high fidelity quality and butteraugli,
so I prefer use vardct -d 0.5~1.0 -e 8, not avif q for still image,
but for I still get some issue on vardct, and those issue can fix or reduce on avif,
I really want use lossy jxl...😢
(file size bloat, red color gradient tiny artifacts issue, contrast sharp edges or line ringing issue)
|
|
|
|
diskorduser
|
2021-10-09 05:34:40
|
avif will always perform better on simple / line art drawing. I think it's better to stick with avif for those images.
|
|
|
|
_wb_
|
2021-10-09 05:38:48
|
Well or do lossless and have no doubt about artifacts
|
|
|
|
lithium
|
|
diskorduser
avif will always perform better on simple / line art drawing. I think it's better to stick with avif for those images.
|
|
2021-10-09 05:40:23
|
But libjxl vardct have butteraugli and best dev group,
I think jxl lossy should always better than avif lossy?
|
|
|
|
diskorduser
|
2021-10-09 05:42:08
|
Avif can eliminate certain artifacts and noise. So it can compress better.
|
|
2021-10-09 05:43:17
|
It's like avif can draw sharp lines even on low bpp and still look good on those images.
|
|
|
|
lithium
|
2021-10-09 05:44:08
|
But I already denoise source, and avif already disable denoise + enabled grain synthesis.
|
|
|
WoofinaS
|
2021-10-09 05:46:07
|
~~`denoise-noise-level` is the strength for grainsynthesis unless you mean something else~~
Nvm i missread
Av1 has other features to clean up the image like dual filter and cdef
|
|
|
|
_wb_
|
2021-10-09 05:47:45
|
Also I suspect palette blocks help in avif
|
|
2021-10-09 05:48:46
|
We can improve encoder heuristics to use more non-dct coding tools when that is better
|
|
2021-10-09 05:50:25
|
Avif encoders are slow and the bitstream possibilities regarding palette encoding are limited, so they can try it.
|
|
|
|
lithium
|
2021-10-10 07:22:05
|
I test JPEG XL encoder v0.6.0 229f574, nonphoto_separation branch --separate option,
In my test, separate option can correct process those image, still have some issue and need improve,
but I think this option is right path for process non-photo image(drawing),
Scope is right, this option is very important, I think I need wait this option back to main branch.😢
> denoise-png 414,581 => jxl 741,530
> denoise-png 962,047 => jxl 826,388
> denoise-png 4,050,574 => jxl 5,851,033
> cjxl -d 1.0 -e 8 --separate=1 --num_threads=12
|
|
2021-10-11 09:09:32
|
Update gaborish=0 result, nonphoto_separation branch.
> denoise-png 414,581 => jxl 756,760
> denoise-png 962,047 => jxl 851,624
> denoise-png 4,050,574 => jxl 3,226,185
> cjxl -d 1.0 -e 8 --separate=1 --gaborish=0 --num_threads=12
|
|
|
retr0id
|
2021-10-11 08:03:46
|
where can I find the winners for the jxl-art competition(s)?
|
|
|
|
_wb_
|
2021-10-11 08:42:34
|
<#824000991891554375>, check archived threads
|
|
2021-10-12 08:11:51
|
Next week is JPEG meeting week again
|
|