JPEG XL

Info

rules 57
github 35276
reddit 647

JPEG XL

tools 4225
website 1655
adoption 20712
image-compression-forum 0

General chat

welcome 3810
introduce-yourself 291
color 1414
photography 3435
other-codecs 23765
on-topic 24923
off-topic 22701

Voice Channels

General 2147

Archived

bot-spam 4380

off-topic

elfeïn
Traneptora I was wondering what the frontend was
2023-05-31 11:16:29
those are called boobs
Traneptora
2023-05-31 11:16:36
<:Weird:1025390746234277919>
gb82
2023-06-01 03:54:11
https://github.com/crablang/crab/issues/53
2023-06-01 03:54:54
🦀<:Hypers:808826266060193874>
elfeïn
2023-06-01 03:56:25
wat
diskorduser
Traneptora what AI software is this <:KEKW:643601031040729099>
2023-06-01 06:56:02
Fotor dot com. It's not free
2023-06-01 09:07:13
https://www.youtube.com/watch?v=JwQ_u5xV1XI
spider-mario
Traneptora it's really just a meme
2023-06-01 11:48:40
I suspect it’s probably an instance of the https://rationalwiki.org/wiki/Toupee_fallacy
diskorduser
diskorduser
2023-06-01 02:09:30
jonnyawsom3
2023-06-01 02:16:06
"It's time for anime"
diskorduser
2023-06-01 02:23:47
Another "steam deck" image
2023-06-01 06:05:36
https://twitter.com/WinRAR_RARLAB/status/1663493174642323457?s=19
Traneptora
diskorduser https://twitter.com/WinRAR_RARLAB/status/1663493174642323457?s=19
2023-06-02 03:07:02
I'm OOTL, what's this?
diskorduser
2023-06-02 03:07:52
windows 11 supports rar file extraction. inbuilt feature
Traneptora
2023-06-02 03:08:03
isn't there some kind of unrar licensing issue?
2023-06-02 03:08:29
either way, the only people who use WinRAR are people who have never heard of 7-zip
afed
2023-06-02 03:25:13
the thing is, not only decompression (and there are open source alternatives for unrar which are allowed by the license), but also compression (which is basically not allowed without special licensing, because rar is a proprietary and closed format, only decompression is open and documented, maybe Microsoft paid for it) ```“We have added native support for additional archive formats, including tar, 7-zip, rar, gz and many others using the libarchive open-source project,” says Windows chief Panos Panay We also plan to provide support for creating those files in those formats in 2024 ```
Traneptora
2023-06-02 03:27:14
oh you can `.tar.gz` something now, interesting
username
2023-06-02 03:30:01
I used 7-zip for years but one day I got bored and decided to try winrar and compare it to 7-zip and I came to find out that I actually like winrar way better then 7-zip
2023-06-02 03:30:17
so now I just use winrar and only pull out 7-zip for edge cases
spider-mario
2023-06-02 03:30:49
may I ask what kind of aspects you prefer from winrar?
afed the thing is, not only decompression (and there are open source alternatives for unrar which are allowed by the license), but also compression (which is basically not allowed without special licensing, because rar is a proprietary and closed format, only decompression is open and documented, maybe Microsoft paid for it) ```“We have added native support for additional archive formats, including tar, 7-zip, rar, gz and many others using the libarchive open-source project,” says Windows chief Panos Panay We also plan to provide support for creating those files in those formats in 2024 ```
2023-06-02 03:32:13
obligatory https://twitter.com/zacbowden/status/1643594814120656897
MSLP
2023-06-02 03:32:54
I love translations without context
Traneptora
2023-06-02 03:47:47
it reminds me of when the cleveland plain dealer said that someone was Back In African-American
2023-06-02 03:47:57
after a system find/replace from Black -> African-American
username
spider-mario may I ask what kind of aspects you prefer from winrar?
2023-06-02 03:48:53
I generally think as a GUI program it's better designed and doesn't suffer from a lot of the problems of 7-zip such as: - Doesn't understand UAC at all leading to the program failing either silently or with a unknown error if you try to do something that requires admin such as changing certain settings or trying to do something with a protected folder. - Doesn't use the native Windows date time format when displaying time - Doesn't allow you to change what font it uses and just uses MS Shell Dlg. - Doesn't prompt you to setup file associations during install which I have seen lead to quite a few problems for end users over time (for example I had a friend that for the longest time didn't even know 7-zip had a GUI and they where using it as their main program to extract things). - Settings menu is lacking a lot of nice options. - Doesn't understand/make us of mouse 4 and mouse 5 buttons. - Just in general has a bunch of smaller issues that add up The person who made 7-zip really understands their stuff when it comes to compression algorithms but I think they are lacking a bit when it comes to program functionality and design.
2023-06-02 03:49:20
this is why I just use 7-zip as a backup for edge cases nowadays
2023-06-02 03:51:58
I also don't really recommend friends and others to use 7-zip anymore but if they insist on using it then I usually give them a small fork I made of it
2023-06-02 03:55:40
sometimes ill ask people why they state that 7-zip is better then winrar and the only reason I have really ever heard is either "because it's open source and free" or "it just is"
2023-06-02 03:56:05
being open source and free doesn't automatically make something better but it is nice
Traneptora
username I generally think as a GUI program it's better designed and doesn't suffer from a lot of the problems of 7-zip such as: - Doesn't understand UAC at all leading to the program failing either silently or with a unknown error if you try to do something that requires admin such as changing certain settings or trying to do something with a protected folder. - Doesn't use the native Windows date time format when displaying time - Doesn't allow you to change what font it uses and just uses MS Shell Dlg. - Doesn't prompt you to setup file associations during install which I have seen lead to quite a few problems for end users over time (for example I had a friend that for the longest time didn't even know 7-zip had a GUI and they where using it as their main program to extract things). - Settings menu is lacking a lot of nice options. - Doesn't understand/make us of mouse 4 and mouse 5 buttons. - Just in general has a bunch of smaller issues that add up The person who made 7-zip really understands their stuff when it comes to compression algorithms but I think they are lacking a bit when it comes to program functionality and design.
2023-06-02 03:59:58
> Doesn't prompt you to setup file associations during install which I have seen lead to quite a few problems for end users over time (for example I had a friend that for the longest time didn't even know 7-zip had a GUI and they where using it as their main program to extract things).
2023-06-02 04:00:00
yes it does
2023-06-02 04:00:38
and nagging you to buy it is a massive downside of software, fr
2023-06-02 04:01:04
it's why I stopped using sublime text
2023-06-02 04:01:07
nagware is annoying
username
Traneptora yes it does
2023-06-02 04:01:55
not for me
2023-06-02 04:02:09
it just installed with 0 options
2023-06-02 04:02:20
well 1 option
Traneptora
2023-06-02 04:02:33
are you reinstalling or installing for the first time
2023-06-02 04:02:35
that may affect it
username
2023-06-02 04:03:24
I use a fork of it which resides in a different folder
2023-06-02 04:03:31
ill try testing in a VM maybe
Traneptora
2023-06-02 04:03:43
it at least used to do that, I don't know if it still does
2023-06-02 04:03:51
unrelated but interesting point about 7zFM is that the GUI is apparently relying on privimitive windows APIs, which is why it's backwards compatible all the way down to windows 2000
2023-06-02 04:03:57
but as a side-effect of this it works in recovery mode
2023-06-02 04:04:13
whereas windows explorer is not
jonnyawsom3
2023-06-02 04:04:29
7zip associates .7z files by default but will leave most other filetypes to their own programs
2023-06-02 04:05:20
At least recalling back a year ago when I had to reinstall after a dead drive
username
7zip associates .7z files by default but will leave most other filetypes to their own programs
2023-06-02 04:06:17
something else I wanna point out is from what I have seen if you do not setup associations from the GUI in the settings menu then the files you associate it with will not have they custom icons and will just have the 7-zip logo.
2023-06-02 04:06:50
I have seen a lot of computers where 7-zip associations where not done from 7-zip's GUI leading to the custom file type icons not being used
2023-06-02 04:07:02
most computers with 7-zip I see are like that
jonnyawsom3
2023-06-02 04:07:28
In my case the 7zip GUI makes the files show as the logo
MSLP
2023-06-02 04:09:34
WinRar context menu also has a nice property of displaying only "compress" subset of selections for uncompressed files and "uncompress" for compressed, while 7-zip displays everything which can be confusing for beginner users
jonnyawsom3
2023-06-02 04:11:51
Yeah, 7zip is more intermediate-expert and winrar is from amateur. Got some nice features but takes time to learn them
diskorduser
2023-06-02 04:21:05
I install 7zip daily on many computers. It doesn't even register .7z files during installation.
Traneptora
2023-06-02 04:32:20
it used to do that then, iirc
2023-06-02 04:32:25
unless I'm just misremembering
MSLP
2023-06-02 04:43:17
there are msi and exe installers, maybe they behave differently
Traneptora
2023-06-02 04:43:46
perhaps
w
2023-06-02 05:24:24
i used winrar 15 years ago
2023-06-02 05:24:36
7zip may not have the best defaults but after setting up the context menu I can't go back
diskorduser
MSLP there are msi and exe installers, maybe they behave differently
2023-06-03 02:44:27
Both exe and msi have the same behaviour
2023-06-03 02:44:42
https://twitter.com/AISafetyMemes/status/1664745714809724928?s=19
DZgas Ж
2023-06-03 09:21:58
winrar can all 🥱 do not update to 11
diskorduser
2023-06-03 11:55:12
Linux can all. Do not install windows
DZgas Ж
diskorduser Linux can all. Do not install windows
2023-06-03 03:25:28
Linux can not reFS ⚰️
diskorduser
2023-06-03 03:26:20
Windows cannot btrfs, zfs ⚰️
DZgas Ж
2023-06-03 03:29:12
⚰️ ok ⚰️
Demez
diskorduser Windows cannot btrfs, zfs ⚰️
2023-06-03 08:00:21
https://github.com/maharmstone/btrfs no native support, but there is this
Traneptora
diskorduser Windows cannot btrfs, zfs ⚰️
2023-06-03 11:34:08
tbf, zfs is incompatible with the kernel licensing-wise
2023-06-03 11:34:15
so it can't be shipped by default with linux distros
2023-06-03 11:34:32
also ext4 supremacy
2023-06-03 11:34:34
it just works ™️
diskorduser
2023-06-04 12:28:27
https://www.paragon-software.com/business/refs-linux/#
DZgas Ж
diskorduser https://www.paragon-software.com/business/refs-linux/#
2023-06-04 12:59:31
Not free Just Business
diskorduser
2023-06-04 12:59:52
I know
Husam
2023-06-04 02:49:38
https://cdn.discordapp.com/attachments/831016078187429958/1114746443790561300/image.png
Traneptora
2023-06-04 04:55:45
that's the kind of thing escher did a lot
spider-mario
2023-06-04 11:23:47
not to be confused with Theodor Escherich, discoverer of E. coli (although he was not the one who named it after himself – he originally named it _Bacterium coli commune_; it was later renamed to _Bacillus coli_ in 1895 and then to _Escherichia coli_ after Escherich’s death)
Foxtrot
2023-06-04 12:11:16
btrfs, zfs, refs? nope, true king is coming 😄 https://bcachefs.org/
diskorduser
Foxtrot btrfs, zfs, refs? nope, true king is coming 😄 https://bcachefs.org/
2023-06-04 02:20:29
why would someone use it when btrfs exists? Is it better than btrfs?
2023-06-04 02:24:51
I am asking it because I don't know anything bcachefs
Foxtrot
2023-06-04 02:25:04
Well, it should be. I dont think anyone would make new FS if they didnt think its better then the previous ones.
2023-06-04 02:26:05
but i cant tell you exactly how, I am just excited to see what becomes of it
2023-06-04 02:26:38
btw, I personaly use ZFS on my TrueNAS... its shame that stupid licencing problems keep it from kernel
diskorduser
2023-06-04 02:26:44
from https://news.ycombinator.com/item?id=22005653
Foxtrot
2023-06-04 02:30:58
it's crazy that it's faster to develop completely new FS for linux kernel than to change licence of already developed one to be used in kernel (ZFS) 😄
Traneptora
2023-06-04 02:30:58
"advanced new filesystem for Linux" to me is a downside I like ext4 cause it's on so many systems most of the bugs have been ironed out
Foxtrot
2023-06-04 02:32:21
ext4 is very good for what it does, but it's just different from CoW filesystems
Traneptora
2023-06-04 02:32:41
end users generally have no need for CoW snapshots
2023-06-04 02:32:51
they aren't backups
Foxtrot
2023-06-04 02:32:53
i guess you can simulate CoW like behavior like red hat does https://stratis-storage.github.io/
Traneptora end users generally have no need for CoW snapshots
2023-06-04 02:33:07
i am end user and I very much like snapshots on my TrueNAS
diskorduser
Traneptora end users generally have no need for CoW snapshots
2023-06-04 02:33:22
why? it is needed for rollbacks in case of updates failure.
Traneptora
2023-06-04 02:33:40
it's not a backup though, if you have corruption, it corrupts the snapshot
2023-06-04 02:33:52
because of the cow nature
Foxtrot
2023-06-04 02:34:09
what corruption? write corruption? thats why it check integrity
2023-06-04 02:34:18
i even have ECC ram
diskorduser
2023-06-04 02:34:19
Some updates cause bugs. not file system corruption
Traneptora
Foxtrot what corruption? write corruption? thats why it check integrity
2023-06-04 02:34:28
saying "my filesystem will never have corruption" is a delusion
diskorduser Some updates cause bugs. not file system corruption
2023-06-04 02:35:01
oic what you mean, like you try to install something and it fails and you want to roll back to pre-install
Foxtrot
2023-06-04 02:35:08
but what do you mean by that? you mean that some bug occures and whole filesystem implodes?
Traneptora
2023-06-04 02:35:16
whole filesystem? no
2023-06-04 02:35:20
but parts of it
diskorduser
2023-06-04 02:35:58
on opensuse tw, it creates snapshots during package installation and removal. it is an useful feature
2023-06-04 02:36:13
like restore point in windows
Foxtrot
2023-06-04 02:36:40
i get what you mean... I use ZFS as target for my PC backups so if ransomware got on my PC and encrypted even backups I can just restore snapshot because they are read only
veluca
2023-06-04 02:36:49
also it's easy to backup snapshots, fwiw
Foxtrot
2023-06-04 02:38:06
you are right, that I should ideally have second NAS and fully backup the first NAS to the second one... but I just do the best I can with with only one
2023-06-04 02:39:21
I like that I can fuck up whatever and just use snapshot and revert it.... I like that snapshots dont use much space, they only store differences
veluca
veluca also it's easy to backup snapshots, fwiw
2023-06-04 02:41:48
in particular, snapshots give you a temporally consistent view of the fs at a certain point in time - you *don't* want to backup different parts of your fs at a different time, it's a great way to break stuff
diskorduser
2023-06-04 02:44:38
It looks like kernel like linux-tkg comes with bcachefs
TheBigBadBoy - 𝙸𝚛
2023-06-04 03:33:10
Is this server interested of having 2 more boosts ? I got 2 free and won't use them otherwise
Fox Wizard
2023-06-04 03:42:27
Too bad 2 more boosts won't do anything <:Cheems:884736660707901470>
Traneptora
2023-06-04 03:42:33
why not ask in <#848189884614705192> ?
diskorduser
2023-06-04 05:14:48
https://www.gizmochina.com/2023/06/04/huawei-challenges-eye-protection-benefits-pwm-dimming-1440hz-honor-responds/
spider-mario
2023-06-04 08:26:26
> Honor believes that the impact of PWM dimming frequency on eye protection is influenced not only by the frequency itself but also by factors such as fluctuation depth and duty cycle. that becomes a very believable hypothesis as soon as you learn of this difference between the flickering of incandescent vs. LED lighting, and then realise (for example by looking at the measurements on notebookcheck) that displays vary in those aspects too
2023-06-04 08:26:43
when they review a device, notebookcheck shows graphs for various brightness levels https://www.notebookcheck.net/Apple-iPhone-14-Pro-review-The-Apple-smartphone-is-ready-for-the-island.659550.0.html#:~:text=Measured%20at%20a%20fixed%20zoom%20level%20at%20various%20brightness%20settings
2023-06-04 08:28:41
(I wish they’d make these a bit easier to read)
2023-06-04 08:32:39
my 7th gen X1 Carbon had deep, slow PWM that caused a visible rainbow effect when moving my eyes, similar to single-chip DLP projectors
2023-06-04 08:33:15
from what I understand, it was specific to the 4K model (the 1080p version was spared) of the 7th gen (which introduced the 4K option, and they fixed that issue for the following generation)
2023-06-04 08:33:30
bad luck for me
2023-06-04 08:34:35
ah, no, 8th gen still had it: https://www.notebookcheck.net/Lenovo-ThinkPad-X1-Carbon-2020-Business-Laptop-Review-4K-display-costs-battery-runtime.482570.0.html
2023-06-04 08:34:52
> However, there is a massive drawback: At a brightness level of 99% or lower, we can detect PWM at a very low frequency of 200 Hz, which can definitely result in problems.
2023-06-04 08:37:15
it looked like this in slow motion
2023-06-04 08:37:43
it looks as though the red subpixels didn’t flicker?
diskorduser
2023-06-05 03:03:56
My eyes gets irritated and tired whenever I use my oled phone in dark place unless I turn on flicker prevention setting.
spider-mario
2023-06-05 08:28:41
I don’t think my eyes minded the X1 Carbon’s flickering, but my brain did, in that it was very distracting
diskorduser
2023-06-05 12:42:58
Is the gesture page navigation is not supposed work on floorp browser?
spider-mario
2023-06-05 03:43:20
> Miyamoto enjoys rearranging furniture in his house, even late at night.[3] He also stated that he has a hobby of guessing the dimensions of objects, then checking to see if he was correct, and reportedly carries a measuring tape with him everywhere.[143]
2023-06-05 03:43:22
interesting hobby
Fraetor
diskorduser https://twitter.com/WinRAR_RARLAB/status/1663493174642323457?s=19
2023-06-05 09:42:40
They'll probably be fine, as the foss version of rar doesn't support encryption, which you often see in the wild.
diskorduser
2023-06-06 01:37:43
http://boards.4channel.org/g/thread/93896441/apple-to-support-jpegxl-in-gecko
2023-06-06 01:58:53
https://gs.statcounter.com/browser-market-share
2023-06-06 01:59:49
I didn't expect opera with this much percentage.
DZgas Ж
2023-06-06 12:28:33
I have a question for compression specialists, for some reason I can't find an answer to this question either in the RLE study material and even the chatgpt neural network can't answer it. The question sounds like this: I have a binary file that contains information, and all Byte variants are used to transmit information. The question is how Intermediate service data can be written, that the sequence has started, or what is the length of the RLE sequence, if all the byte value variations are already use in the most compressible file and cannot be used as pointers
2023-06-06 12:28:47
<:Thonk:805904896879493180>
Traneptora
2023-06-06 12:40:38
what
sklwmp
2023-06-06 12:43:40
(2)
2023-06-06 12:43:40
what
DZgas Ж
2023-06-06 12:43:53
```00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F 40 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D 5E 5F 60 61 62 63 64 65 66 67 68 69 6A 6B 6C 6D 6E 6F 70 71 72 73 74 75 76 77 78 79 7A 7B 7C 7D 7E 7F 80 81 82 83 84 85 86 87 88 89 8A 8B 8C 8D 8E 8F 90 91 92 93 94 95 96 97 98 99 9A 9B 9C 9D 9E 9F A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 AA AB AC AD AE AF B0 B1 B2 B3 B4 B5 B6 B7 B8 B9 BA BB BC BD BE BF C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 CA CB CC CD CE CF D0 D1 D2 D3 D4 D5 D6 D7 D8 D9 DA DB DC DD DE DF E0 E1 E2 E3 E4 E5 E6 E7 E8 E9 EA EB EC ED EE EF F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 FA FB FC FD FE FF``` the byte values
sklwmp
2023-06-06 12:49:51
https://www.reddit.com/r/StableDiffusion/comments/141hg9x/controlnet_for_qr_code/
2023-06-06 12:49:56
https://twitter.com/victormustar/status/1665824339218898948
veluca
2023-06-06 12:52:05
yeah pretty cool stuff
DZgas Ж
DZgas Ж I have a question for compression specialists, for some reason I can't find an answer to this question either in the RLE study material and even the chatgpt neural network can't answer it. The question sounds like this: I have a binary file that contains information, and all Byte variants are used to transmit information. The question is how Intermediate service data can be written, that the sequence has started, or what is the length of the RLE sequence, if all the byte value variations are already use in the most compressible file and cannot be used as pointers
2023-06-06 12:53:17
all formats always have pointers -- that containers, that codecs, service information that is determined in advance. Even unicode has this in order to use more than 1 byte, but these are all pre-known values. I can't find the answer to the question - what do compression algorithms, archives, do in order to solve this problem. There is literally not a word about it in any educational material. And all the examples are done on numbers or Letters. At the same moment, the neural network says "just take one byte and write that it is the beginning of the RLE sequence" -- how can I do this if all the bytes are already used in the Compressed file? <@853026420792360980><@557099078337560596>
Traneptora
2023-06-06 12:54:11
????
2023-06-06 12:54:16
compression formats add headers and things like that
DZgas Ж
Traneptora compression formats add headers and things like that
2023-06-06 12:54:58
Examples?
Traneptora
2023-06-06 12:56:26
https://www.rfc-editor.org/rfc/rfc1952.txt
DZgas Ж
Traneptora https://www.rfc-editor.org/rfc/rfc1952.txt
2023-06-06 12:58:28
it looks surprisingly small... wow 12 pages. need to read
veluca
2023-06-06 12:59:18
that's just of the container, the compression spec is https://www.ietf.org/rfc/rfc1951.txt (still not very long)
DZgas Ж
veluca that's just of the container, the compression spec is https://www.ietf.org/rfc/rfc1951.txt (still not very long)
2023-06-06 01:00:47
not much either. In fact, I didn't even think about going to read the real documentation of archives, because I'm used to the fact that there are more than 200 pages
2023-06-06 01:52:10
hm.
derberg🛘
sklwmp https://twitter.com/victormustar/status/1665824339218898948
2023-06-06 01:52:13
Amazing, that's something I wanted to be able to generate ten years ago (lol)
DZgas Ж
Traneptora https://www.rfc-editor.org/rfc/rfc1952.txt
2023-06-06 01:53:21
why does it exist? <:Thonk:805904896879493180> one of the most useless things
2023-06-06 01:54:16
it's been almost 30 years, but it's literally not used for anything anywhere, I can't create a file with a different kind of compression or without compression at all
Traneptora
DZgas Ж why does it exist? <:Thonk:805904896879493180> one of the most useless things
2023-06-06 01:54:49
because in 1996 they wanted to futureproof it
2023-06-06 01:54:58
it's three bits, who cares
DZgas Ж
2023-06-06 01:55:55
but at the same time, the archive will break if I try to change the number 8 to any other, it's just amazing that 7zip does not skip checking this number at all lol
Traneptora it's three bits, who cares
2023-06-06 01:56:39
lost functional,
Traneptora
2023-06-06 01:56:56
PNG does this as well, it has a compression method flag
2023-06-06 01:56:59
in IHDR
2023-06-06 01:57:09
but the only permitted value is the one indicating zlib
DZgas Ж
2023-06-06 01:57:40
do png 2 now <:KekDog:805390049033191445>
Traneptora
2023-06-06 01:58:01
~~lossless JXL~~
DZgas Ж
Traneptora PNG does this as well, it has a compression method flag
2023-06-06 01:58:25
but no, it can be changed, there is a compression level of 0 that makes PNG analogous to BMP
Traneptora
2023-06-06 01:58:59
that's zlib compression level isnt' the same thing as compression method
DZgas Ж
Traneptora that's zlib compression level isnt' the same thing as compression method
2023-06-06 01:59:17
?
Traneptora
2023-06-06 01:59:50
compression method in the PNG IHDR says what algorithm is used for compression
2023-06-06 02:00:04
zlib compression level 0 still outputs zlib-compatible compressed data
2023-06-06 02:00:23
it just doesn't actually do anything interesting to "compress" it
DZgas Ж
Traneptora zlib compression level 0 still outputs zlib-compatible compressed data
2023-06-06 02:00:44
without compress
Traneptora
2023-06-06 02:00:52
ye
2023-06-06 02:01:01
it just sticks a header in front of it and that's about it
DZgas Ж
Traneptora it just sticks a header in front of it and that's about it
2023-06-06 02:01:58
where is GZIP compression 0 ?
Traneptora
2023-06-06 02:02:09
gzip is a container for DEFLATE it's not actually a compression algorithm
DZgas Ж
2023-06-06 02:02:49
zlib - DEFLATE - GZIP
Traneptora gzip is a container for DEFLATE it's not actually a compression algorithm
2023-06-06 02:03:27
Where is the answer?
Traneptora
2023-06-06 02:03:41
you'd have to look at DEFLATE compression 0
2023-06-06 02:03:48
which is detailed in the DEFLATE RFC
DZgas Ж
2023-06-06 02:05:13
<:Thonk:805904896879493180>
2023-06-06 02:06:29
Well, in this case, we can say that the developers are complete idlers, they could not write a couple of lines of code that would encode all the data in blocks marked Not compressed, and make the compression mode 0
Traneptora
2023-06-06 02:15:10
you can do that in ZIP
2023-06-06 02:15:12
not in gzip tho
DZgas Ж
2023-06-06 02:15:18
for some reason, the uncompressed compression mode was cut out of 7zip-zstd when the 7z archive is marked as "copy". This is why 7zip-zstd just doesn't have it, but it does in the original 7zip. Also, all levels 1 of 7zip-zstd are marked as "Store"-- What does that even mean?
Traneptora
2023-06-06 02:15:19
as the whole point of gzip is to compress a single file
2023-06-06 02:15:35
store means "don't compress"
2023-06-06 02:15:41
zip and 7z both support different compression methods
2023-06-06 02:15:48
gzip does not, as gzip's purpose is to compress a single file
DZgas Ж
Traneptora you can do that in ZIP
2023-06-06 02:16:03
no, if I need to compress without compression, I use the best archive that contains a whole normal file system in its head -- 7z
Traneptora
2023-06-06 02:16:26
if you need to compress a *single file* without compression you just use the original file......
DZgas Ж
2023-06-06 02:16:43
single file ok just use tar
Traneptora
2023-06-06 02:16:47
or just
2023-06-06 02:16:49
the original file
2023-06-06 02:16:55
7z and zip are *archive* formats, which is why they support multiple methods
DZgas Ж
2023-06-06 02:17:30
sometimes it is useful that no one knows what the file really is, for example, when sending .webp in telegram
MSLP
2023-06-06 02:17:35
<@226977230121598977> on a side note, this tool may be interesting to you: https://github.com/wader/fq It has some analysis for gzip and png, I didn't use it, so don't know how good is it, but maybe it can be handy for you
DZgas Ж
2023-06-06 02:20:21
in fact, you are surprised when you find out that there are only 2 archives that can compress and contain more than 1 file, these are only zip and 7z
2023-06-06 02:20:41
literally all
jonnyawsom3
DZgas Ж for some reason, the uncompressed compression mode was cut out of 7zip-zstd when the 7z archive is marked as "copy". This is why 7zip-zstd just doesn't have it, but it does in the original 7zip. Also, all levels 1 of 7zip-zstd are marked as "Store"-- What does that even mean?
2023-06-06 02:21:03
"Store" means no compression
MSLP
DZgas Ж I have a question for compression specialists, for some reason I can't find an answer to this question either in the RLE study material and even the chatgpt neural network can't answer it. The question sounds like this: I have a binary file that contains information, and all Byte variants are used to transmit information. The question is how Intermediate service data can be written, that the sequence has started, or what is the length of the RLE sequence, if all the byte value variations are already use in the most compressible file and cannot be used as pointers
2023-06-06 02:21:10
I'd use some "escape sequence" - eg. define "ff xx" as a RLE times xx, where xx = 00 .. fe, and encode single "ff" byte as "ff ff"
Traneptora
2023-06-06 02:21:32
that's one way to do it, yea
DZgas Ж
MSLP <@226977230121598977> on a side note, this tool may be interesting to you: https://github.com/wader/fq It has some analysis for gzip and png, I didn't use it, so don't know how good is it, but maybe it can be handy for you
2023-06-06 02:22:21
most recently, I was parsing mp3 through the documentation, in fact, I don't think I need this tool for anything
MSLP <@226977230121598977> on a side note, this tool may be interesting to you: https://github.com/wader/fq It has some analysis for gzip and png, I didn't use it, so don't know how good is it, but maybe it can be handy for you
2023-06-06 02:22:49
looks so geek
"Store" means no compression
2023-06-06 02:24:28
Aah, well, then it's clear, the author 7zip-zstd is dumb
2023-06-06 02:27:38
literally
2023-06-06 02:28:13
2023-06-06 02:29:42
the 7zip-zstd project is already 7 years old, and are you serious about it?
MSLP I'd use some "escape sequence" - eg. define "ff xx" as a RLE times xx, where xx = 00 .. fe, and encode single "ff" byte as "ff ff"
2023-06-06 02:33:34
Hmm, but you just came up with the idea. But I would like to know how it really is, now I will finish reading GZIP and start reading DEFLATE documentation
2023-06-06 02:43:33
another real unreal useful flag
2023-06-06 02:55:19
the most important information at the beginning takes only 3 bytes, but then there is a gathering of completely unnecessary garbage, at least 7 bytes of the GZIP format itself are spent on nothing
Traneptora https://www.rfc-editor.org/rfc/rfc1952.txt
2023-06-06 02:57:27
In fact, it was necessary to start with the documentation of the compression algorithm itself, there is literally nothing in the GZIP documentation, in general, except that it sets the type of compression algorithm - nothing in gzip format is done anymore.
2023-06-06 03:08:40
That's it, I found out
2023-06-06 03:09:24
in fact, it's even strange that I didn't realize this a little earlier, because I've already worked with huffman trees
DZgas Ж I have a question for compression specialists, for some reason I can't find an answer to this question either in the RLE study material and even the chatgpt neural network can't answer it. The question sounds like this: I have a binary file that contains information, and all Byte variants are used to transmit information. The question is how Intermediate service data can be written, that the sequence has started, or what is the length of the RLE sequence, if all the byte value variations are already use in the most compressible file and cannot be used as pointers
2023-06-06 03:12:30
The answer to my question will be this: Due to the Huffman tree algorithm, any amount of data with any content variants can be converted into bits of any length that will be unique during reading, moving exactly along the tree. Bits can be of any length, from 1 to any number - and the size of the tree can be any size.
2023-06-06 03:18:03
this also means that the RLE algorithm cannot be used at all without Huffman
2023-06-06 03:21:47
n.txt.gz 165 bytes n.7z zstd 260 bytes n.txt.zst 160 bytes
2023-06-06 03:22:29
💁‍♂️
2023-06-06 03:48:57
my self-written (with the help of a neural network) huffman algorithm compressed into 373 bytes. it is worth saying that lz77 works well
DZgas Ж n.txt.gz 165 bytes n.7z zstd 260 bytes n.txt.zst 160 bytes
2023-06-06 04:12:53
on such data sizes, Brotli is really the best compression algorithm for text, bypassing the other ppmd algorithm by 1-2%
diskorduser
2023-06-07 02:44:52
https://twitter.com/jaffathecake/status/1666006638799605762?s=19
2023-06-07 02:45:21
What does he mean by sizes larger than ideal for the web? Is he spreading misinformation?
username
2023-06-07 02:45:59
it depends on what you consider "ideal for the web"
jonnyawsom3
2023-06-07 02:46:04
Well, that depends if you count 'ideal for the web' as 'lowest size possible'
diskorduser
2023-06-07 02:46:23
Maybe he thinks emojis and smudgy images are ideal for the web
jonnyawsom3
2023-06-07 02:47:54
Ironically, emoji's would probably compress better with JXL
diskorduser
2023-06-07 02:54:29
For lossless jxl yeah what about lossy jxl vs avif?
190n
2023-06-07 03:49:21
who wants lossy emojis
2023-06-07 03:49:23
yuck
2023-06-07 03:55:01
but if the file is small enough avif loses due to its massive header
2023-06-07 03:55:09
idk if they've fixed that
w
2023-06-07 03:58:03
that is not a real problem
diskorduser
190n who wants lossy emojis
2023-06-07 04:04:02
I think the avif team did benchmarks on lossy jxl vs avif to show how avif is better
190n
w that is not a real problem
2023-06-07 04:05:26
for emojis?
w
2023-06-07 04:05:58
just use fonts
diskorduser
2023-06-07 04:06:34
Yeah
username
2023-06-07 04:08:49
COLRv1 should allow emojis in fonts to be possible now (it was possible before but very limited)
diskorduser
2023-06-07 04:17:55
https://twitter.com/Anno0770/status/1666096806718218243?s=19
2023-06-07 04:18:58
https://unsplash.com/photos/i8xzA_chRgs
2023-06-07 04:20:20
They lose their minds whenever they try to justify avif's superiority.
w
2023-06-07 04:21:08
tbh any (insert image format) advocate
2023-06-07 04:21:10
including jxl
spider-mario
2023-06-07 08:31:38
I count at least the following: - red herring + appeal to ridicule + ad hominem + guilt by association (once, one of you proposed this unrelated and obviously (to me) ridiculous hypothesis, so clearly everything your group says is doubtful) - motte and bailey (chrome’s support for jxl was behind a flag -> there was no support to remove, no invested effort, zero potential intent to make it the default) - poisoning the well (“Folks with a taste for spreading misinformation will twist this though 😀”) - shifting the burden of proof (wb presented contradicting data, so suddenly, Jake supporting his original claim is “Brandolini’s law”)
2023-06-07 08:31:44
am I missing anything?
elfeïn
spider-mario I count at least the following: - red herring + appeal to ridicule + ad hominem + guilt by association (once, one of you proposed this unrelated and obviously (to me) ridiculous hypothesis, so clearly everything your group says is doubtful) - motte and bailey (chrome’s support for jxl was behind a flag -> there was no support to remove, no invested effort, zero potential intent to make it the default) - poisoning the well (“Folks with a taste for spreading misinformation will twist this though 😀”) - shifting the burden of proof (wb presented contradicting data, so suddenly, Jake supporting his original claim is “Brandolini’s law”)
2023-06-07 08:59:41
ahaha- I see you are a fellow maxim-enjoy.
VcSaJen
2023-06-07 11:21:27
SVG can seriously slow down the browser, especially animated ones. Bitmaps makes sense.
2023-06-07 11:25:31
I wonder if progressive vector graphics are theoretically possible with tesselation. Icons don't need that much detail.
jonnyawsom3
2023-06-07 11:48:27
Didn't know animated SVG was possible, although with JXL splines you could probably do animated progressive similarly
190n
2023-06-07 11:54:52
is it possible without embedding javascript
yurume
2023-06-08 01:07:50
recently: I'm now working for some NPU startup since last December and doing too many dirty works (*cough* Python C API *cough*)
2023-06-08 01:09:40
I actually enjoy doing dirty works, as long as I have a reasonable confidence that I can do this, but it does take so much energy that my Github presence is greatly reduced (I use a separate GH account for work)
2023-06-08 01:12:48
ah sorry, hardware accelerator for machine learning. there are several synonyms but NPU seems to be the best known terminology.
2023-06-08 01:15:21
while GPGPU excels at parallel computing, ML computation payload (e.g. matrix multiplication) requires close cooperation between memory and computation unit which GPGPU can't fully do, so dedicated HW solution is useful
2023-06-08 01:17:48
once again Google does have its own NPU, namely Tensor Processing Unit, but you can't directly buy TPU for various reasons, hence startups.
2023-06-08 01:19:09
that said as I've mentioned before I do not design hardwares (I can't do), I'm mainly a plumber 😉
2023-06-08 01:19:52
a plumber who knows enough hardware details to (hopefully) deliver solutions, that is
190n
2023-06-08 02:09:38
maybe you should ask on /r/jpegxl
Traneptora
2023-06-08 10:37:50
what's this blackout?
spider-mario
2023-06-08 10:40:01
reddit is introducing API pricing that will effectively kill third-party clients
2023-06-08 10:40:08
various subs are protesting
Traneptora
2023-06-08 10:41:14
ah
lonjil
2023-06-08 11:24:28
Notably, many large subreddits could not function without third party moderation tools that rely on API access
2023-06-08 11:24:59
Also notably, the official app and website have poor accessibility so many blond users use 3rd party apps.
Traneptora
2023-06-08 11:26:00
those dumb blond users
2023-06-08 11:26:07
<:Pausers:1019605474565435423>
lonjil
2023-06-08 11:26:46
Oops
2023-06-08 11:26:48
Blind
Traneptora
2023-06-08 11:26:54
ik :D
spider-mario
2023-06-09 09:40:51
https://pastebin.com/raw/MNVmY1TA
2023-06-09 09:42:54
(ChatGPT)
gb82
diskorduser https://twitter.com/jaffathecake/status/1666006638799605762?s=19
2023-06-09 11:59:17
His MO is shilling for whatever Google does. I'd just ignore him
diskorduser
2023-06-09 11:59:46
MO means?
gb82
2023-06-09 12:00:55
> (I also recall you telling everyone that Google forced WebP on everyone by dropping support for JPEG on YouTube. The source of this was your imagination) As a response to Jon's legitimate benchmarks. When in reality Google *did* do this, kinda - allowing WebPs to be 30% smaller (not sure if there was an efficiency benefit here) & making Chrome WebP only. I think that's prob what he's referring to, but still, why bring up a totally different argument?
diskorduser MO means?
2023-06-09 12:01:14
modus operandi
diskorduser
2023-06-09 12:01:22
Ah.
_wb_
2023-06-09 12:17:56
Jake doesn't even work at Google anymore. I guess he somehow thinks that jxl is only useful at q > 90 and that jxl getting adoption will mean that everyone will start using high qualities, making the web slower. Or something like that.
2023-06-09 12:22:59
Google did kind of push webp, not by making their own sites like YouTube break on other browsers though. They let other sites do that, and kind of encouraged that behavior by making their web perf tools and search rank metrics favor lossy webp over progressive jpeg, and not penalizing the use of webp without fallback in any way... Probably not intentionally or with some kind of evil plan, but the end result was that some websites started to serve webp without fallback, which basically forced firefox and safari to add support because whenever a page works on chrome but doesn't in your browser, that's not considered a bad page, it's considered a bug in your browser.
2023-06-09 12:31:01
So I made an incorrect claim saying that YouTube did that — but that was an honest mistake: I did test YouTube in Chrome with "disable WebP image format" checked, and I saw that all video thumbnails stopped working, so I assumed YouTube didn't work on browser that don't support WebP (but it turns out it does, it just has some weird way of detecting webp support, maybe it only checks the UA string and not Accept headers or something)
2023-06-09 12:33:56
but my main point does still stand imo. I said > Chrome can force things because they can control both the client and server side. E.g. with WebP, they first made it work in Chrome, then forced other browsers to also support it by making YouTube use it without fallback, effectively 'breaking the web' for Firefox and Safari. and while the "making YouTube use it without fallback" part was false, the rest of the sentence is imo still true: > Chrome can force things because they can control both the client and server side. E.g. with WebP, they first made it work in Chrome, then forced other browsers to also support it by ~~making YouTube use it without fallback, ~~ effectively 'breaking the web' for Firefox and Safari.
yoochan
2023-06-09 12:45:03
Victor Hugo said once : don't spend too much saliva justifying yourself on twitter
_wb_
2023-06-09 12:52:34
yoochan
2023-06-09 01:11:02
I did this more than once
spider-mario
2023-06-09 01:11:49
after many online “discussions”, my strategy now involves checking that the other party’s points are actually relevant
2023-06-09 01:12:07
otherwise, there is a risk of getting drawn into a different direction that has nothing to do with the point that you were originally trying to make
2023-06-09 01:12:20
and potentially even feeling that you have to defend a position you don’t even hold
2023-06-09 01:14:48
if a point they make is not relevant (like here, with “once, you said something based on wrong data”), I might briefly address it (“part of my data was wrong and I quickly retracted it”), but then I would try to recenter the discussion (“but back to the point, do you have data suggesting AVIF performs better?”)
2023-06-09 01:15:53
(not trying to say that this is the best way to do it or anything)
spider-mario if a point they make is not relevant (like here, with “once, you said something based on wrong data”), I might briefly address it (“part of my data was wrong and I quickly retracted it”), but then I would try to recenter the discussion (“but back to the point, do you have data suggesting AVIF performs better?”)
2023-06-09 01:19:32
(this is the polite version)
190n
diskorduser https://twitter.com/jaffathecake/status/1666006638799605762?s=19
2023-06-09 07:03:10
he's been awful quiet since jon asked for evidence
bonnibel
2023-06-09 08:44:42
fun fact: for some reason ssimu2 really likes the following mpv upscaling options on my very small dataset: ``` profile=gpu-hq vo=gpu-next scale=lanczos scale-radius=2 scale-window=kaiser scale-wparam=1 ```
2023-06-09 08:51:57
probably just a quirk but was the only combination that scored over 60 on a 2.4x upscale in this test
lonjil
2023-06-09 10:06:21
> Jake doesn't even work at Google anymore. I guess he somehow thinks that jxl is only useful at q > 90 so, the most useful qualities? 😄
2023-06-09 10:40:40
jonnyawsom3
2023-06-09 11:40:57
What
elfeïn
lonjil
2023-06-10 12:27:50
lol toxic
DZgas Ж
2023-06-10 01:14:32
It's always funny
prick
2023-06-10 07:20:18
https://www.tomshardware.com/news/gddr6-vram-prices-plummet
Traneptora
elfeïn lol toxic
2023-06-12 01:15:45
this was back when discord used to doublesend messages
2023-06-12 01:16:04
like all the time
elfeïn
Traneptora this was back when discord used to doublesend messages
2023-06-12 01:17:35
ohhhh
Traneptora
2023-06-12 01:19:48
I'm pretty sure I'm the "deleted user" in that post as I'm the one who recommended mpv to lonjil and I'm also on a different account than I used to be on back in 2017
2023-06-12 01:20:14
and I def wouldn't have mocked them <:kek:857018203640561677>
diskorduser
2023-06-12 07:42:43
https://photographylife.com/is-your-browser-color-managed
2023-06-12 07:42:57
Looks like firefox is sh*t
DZgas Ж
2023-06-12 07:43:05
The most powerful I know 7-zip - bruteforce - REcompressor ( m7zRepacker last version ) Really some kind of lost thing. But keep in mind it is very powerful and slow. I use it exclusively to compress my code, 18 kb txt --> 2.8 kb 7 zip ppmd --> 2.4kb bruteforce recompress (10 seconds it does it) ```m7zRepacker.exe -m1 -d1024 -mem1024 something.7z``` https://encode.su/threads/1201-m7zRepacker?p=79788&viewfull=1#post79788
diskorduser
diskorduser https://photographylife.com/is-your-browser-color-managed
2023-06-12 07:45:08
I can see color difference on firefox, edge and chrome on windows 11. Does it mean these browsers are bad?
w
2023-06-12 07:54:02
they depend on your system
2023-06-12 07:54:11
are you color managed
Traneptora
2023-06-12 10:40:47
I'm using Arch and the two pictures look identical to me
2023-06-12 10:40:52
and Firefox
2023-06-12 10:40:58
did you turn on iccv4 (off by default)
w
2023-06-12 10:43:33
the website's srgb version is untagged so you would need mode 1
2023-06-12 10:43:45
insufficient test anyway imo
2023-06-12 10:45:24
mode 1 to color manage all
2023-06-12 10:47:26
here have some greens
2023-06-12 10:47:36
should be noticeably different on a monitor with wled backlight or newer
2023-06-12 10:48:04
on windows 11 both firefox and chrome don't support windows advanced color api (for when auto color management is on), but at least firefox allows setting a display profile manually
Traneptora
2023-06-12 10:48:23
is prophoto HDR?
w
2023-06-12 10:48:33
it's wide(r) gamut
Traneptora
2023-06-12 10:48:47
I ask cause I have an sRGB SDR monitor and they look the same
2023-06-12 10:49:03
which means I don't have a "wled backlight" or whatever
w
2023-06-12 10:49:34
basically any 2017 ips or newer
Traneptora
2023-06-12 10:49:50
ah, I don't have an IPS
2023-06-12 10:49:59
also I think my monitor is from 2015
2023-06-12 10:50:02
so that's 0 for 2
w
2023-06-12 10:58:20
I use it to test if color management is working
2023-06-12 10:58:47
if you know your monitor is less than srgb then it matters less since you wont be seeing anything oversaturated.
spider-mario
2023-06-12 11:19:02
it’s likely that the ProPhoto image still only encodes colors that are in sRGB, and that they are therefore supposed to look identical regardless of your monitor’s gamut
Traneptora
2023-06-12 11:27:05
my favorite way to test if color management is working is the green bench
2023-06-12 11:27:57
2023-06-12 11:28:19
it looks green if the ICC is stripped
spider-mario
2023-06-12 11:28:21
is it missing the ICC profile? it looks green even in Chrome
Traneptora
2023-06-12 11:28:22
which I think discord is doing
2023-06-12 11:28:37
https://0x0.st/Hcxm.jpg
spider-mario
2023-06-12 11:28:51
ah, this one works
Traneptora
2023-06-12 11:28:55
this one is a link
spider-mario
2023-06-12 11:28:56
is that what you uploaded?
2023-06-12 11:29:06
does it now strip on upload then?
Traneptora
2023-06-12 11:29:08
same file I sent to discord
spider-mario
2023-06-12 11:29:15
disappointing
Traneptora
2023-06-12 11:29:20
so it looks like discord is stripping ICCs on upload
spider-mario
2023-06-12 11:29:28
it didn’t use to, did it?
username
2023-06-12 11:29:42
heres a tip you can prevent discord from messing with your PNGs and JPEGs by adding ".webp" at the end of their name
Traneptora
2023-06-12 11:30:15
this is the green bench from the conformance repo, reconstructed as a JPEG (djxl bench_oriented_brg/input green-bench.jpg)
username
2023-06-12 11:30:30
for PNG sometimes ICC gets stripped while other times it doesn't I think it depends on how the ICC profile is embedded (I think? I might be mistaking another program)
spider-mario
2023-06-12 11:30:39
what do the people who program “let’s strip the ICC profile” think it does?
2023-06-12 11:30:42
or do they just forget about it?
Traneptora
spider-mario what do the people who program “let’s strip the ICC profile” think it does?
2023-06-12 11:31:15
they probably think that the ICC profile is random unnecessary metadata cause they have sRGB monitors and all their images are sRGB, ig
w
2023-06-12 11:32:41
strange.. how come my prophoto green wasnt stripped
yoochan
2023-06-12 03:06:43
on strike if I remember, to protest against the new policy which will put a paywall for third party apps using the API
Sauerstoffdioxid
2023-06-12 03:08:37
the subreddits are striking, yes. but reddit itself is also currently down https://www.redditstatus.com/
_wb_
2023-06-12 03:45:48
lol, I wonder what happened — is reddit "striking back"? did some of their systems break by suddenly having many subreddits switch to private? did they get hacked?
lonjil
2023-06-12 03:46:11
Maybe the IT people at reddit are also striking against the new policy
2023-06-12 03:46:28
Wouldn't be surprised if none of them use the official app 😂
elfeïn
2023-06-12 07:18:39
Man, they even managed to make color complicated.
_wb_
2023-06-12 07:48:10
How could color not be complicated? It's all about the biology of our eyes, the physics of light, and the engineering of reproduction technology from inkjet printers to OLED displays.
yoochan
elfeïn Man, they even managed to make color complicated.
2023-06-13 07:24:49
if you missed it : https://www.youtube.com/watch?v=xAoljeRJ3lU is an entertaining video on the topic
DZgas Ж
2023-06-14 01:02:45
Why utf-8 use this
2023-06-14 01:03:00
Why not this 1xxxxxxx 1xxxxxxx 0xxxxxxx 1xxxxxxx 0xxxxxxx 0xxxxxxx
yurume
2023-06-14 01:05:18
because removing 0xxxxxxx will yield a valid encoding in your scheme
2023-06-14 01:06:17
UTF-8 was designed to be a drop-in replacement for C libraries, thus no nulls except for U+0000, and no valid UTF-8 sequence is a subsequence of other UTF-8 sequence (otherwise naive byte-based C libraries will be confused)
DZgas Ж
yurume because removing 0xxxxxxx will yield a valid encoding in your scheme
2023-06-14 01:07:16
? 0xxxxxxx 0xxxxxxx 1xxxxxxx 0xxxxxxx 1xxxxxxx 1xxxxxxx
yurume
2023-06-14 01:07:33
`0xxxxxxx` is a subsequence of `0xxxxxxx 1xxxxxxx`, isn't it
2023-06-14 01:08:07
so if you search for the former bytewise, the latter may be mistaken for the former
DZgas Ж
2023-06-14 01:08:32
Oh yea 0xxxxxxx 1xxxxxxx 0xxxxxxx 1xxxxxxx 1xxxxxxx 0xxxxxxx
2023-06-14 01:08:37
Now all same
yurume
2023-06-14 01:08:45
now the second is a subsequence of the third 😉
DZgas Ж
DZgas Ж Why utf-8 use this
2023-06-14 01:10:13
<@268284145820631040> same?
yurume
2023-06-14 01:10:23
no, because of the first byte
DZgas Ж
2023-06-14 01:10:43
Why first?
2023-06-14 01:10:58
Read byte to 0xxxxxxx
yurume
2023-06-14 01:11:25
I'm not sure what do you want to say, if you think UTF-8 has an overlapping sequence, give me an example
DZgas Ж
2023-06-14 01:11:56
overlapping ?
yurume
2023-06-14 01:12:16
as in, some sequence can be mistaken for other sequence
2023-06-14 01:12:45
I can make a concrete example for your scheme
DZgas Ж
yurume as in, some sequence can be mistaken for other sequence
2023-06-14 01:12:56
how
yurume I can make a concrete example for your scheme
2023-06-14 01:13:14
do
yurume
2023-06-14 01:13:52
say a character A is encoded as `01010101`, B as `10101010 01010101`, C as `10101010 10101010 01010101`
2023-06-14 01:14:18
now searching `B` from `C` will yield a false positive
DZgas Ж
2023-06-14 01:14:32
Not, only 7 bit in byte
yurume
2023-06-14 01:14:38
what?
DZgas Ж
2023-06-14 01:14:45
Same as unicode
yurume
2023-06-14 01:14:59
``` B is represented as 10101010 01010101 C is represented as 10101010 10101010 01010101 ```
DZgas Ж
2023-06-14 01:15:07
Ansi is 7 bit
yurume
2023-06-14 01:15:18
so a bytewise search routine will think `C` contains `B` at the offset 1
DZgas Ж
2023-06-14 01:15:44
Why are you write first bit as symbols
yurume
2023-06-14 01:15:57
sorry, updating
2023-06-14 01:16:09
is this okay now
DZgas Ж
DZgas Ж Why not this 1xxxxxxx 1xxxxxxx 0xxxxxxx 1xxxxxxx 0xxxxxxx 0xxxxxxx
2023-06-14 01:17:38
The other day I created my tokenizer based on a complete search of all the values of the text and the creation of a dictionary. So, to store 21 bits of information in the form of numbers, I use just such a storage system. Its works
yurume
2023-06-14 01:18:04
well your search routine can be made to recognize and reject such false positives
2023-06-14 01:18:13
that's fine, everyone did that for legacy character encodings
2023-06-14 01:18:37
UTF-8 was designed so that byte-wise routine can work as is, which was a big improvement
DZgas Ж
2023-06-14 01:19:34
But of course if you don't read the last byte then everything will break
2023-06-14 01:19:48
but i use first in my system
yurume
2023-06-14 01:20:46
UTF-8 can detect that though
2023-06-14 01:20:54
(technical term: self-synchronizing code)
DZgas Ж
DZgas Ж The other day I created my tokenizer based on a complete search of all the values of the text and the creation of a dictionary. So, to store 21 bits of information in the form of numbers, I use just such a storage system. Its works
2023-06-14 01:22:04
But here are my current results with the most common text tokenizer. mrsteyk/rupgt-chatml-tokenizer 274, 4674, 306, 309, 2789, 3057, 6957, 294, 296, 13124, 16, 492, 505, 19041, 2712, 309, 1209, 14938, 7150, 1209, 571, 2040, 4678, 16, 7155, 35250, 15559, 3136, 790, 435, 3417, 20755, 12538, 266 My -5935-18486-201-14619-4887-402-451-917-271-16-4960-5513-1346-12997-12482-3420-4302-25335-18191-13724-8737-278-16282-572-641-4187-982-6055-26049-27225-519-7151-20733-11137-1604-5697-271-88-252-397-384
yurume UTF-8 can detect that though
2023-06-14 01:22:28
My same
yurume
2023-06-14 01:22:55
in my example, `C` but the first byte missing will look exactly like `B`
2023-06-14 01:23:14
you can indeed detect the last byte missing, but not in the other direction
DZgas Ж
2023-06-14 01:23:30
Each character always starts with "1" bit, so if something breaks, will one value
yurume
2023-06-14 01:25:03
yeah, your design is indeed better than most legacy multibyte encodings, but worse than UTF-8 I meant
DZgas Ж
2023-06-14 01:25:24
in general... In my memory, unicode looked like this 111xxxxx 11xxxxxx 1xxxxxxx But in fact it turned out to be some kind of ineffective bullshit, so I had to come up with my own
yurume
2023-06-14 01:26:03
nope, UTF-8 is designed so that the first byte alone gives the total information about how subsequent bytes would look like
2023-06-14 01:26:37
`0xxxxxxx` is a single byte. `110xxxxx` is two bytes long (i.e. one byte more to read). `1110xxxx` is three bytes long, etc.
2023-06-14 01:26:48
and all "subsequent" bytes are `10xxxxxx`
DZgas Ж
2023-06-14 01:27:42
If you lose one byte of 10xxxxxx everything will break. I don't see the point
yurume
2023-06-14 01:27:45
you can argue about the exact byte allocation (for example, UTF-8 originally allowed up to 6 bytes, and nowadays 5--6 bytes are gone, leaving some bytes completely unused)
2023-06-14 01:27:52
why?
2023-06-14 01:27:58
let me give some concrete example
2023-06-14 01:28:39
I have this sequence: ``` 11101100 10010101 10001000 11101011 10000101 10010101 ```
2023-06-14 01:29:04
if I understand you correctly, removing the third byte `10001000` should break everything, right? is this what did you mean?
DZgas Ж
yurume I have this sequence: ``` 11101100 10010101 10001000 11101011 10000101 10010101 ```
2023-06-14 01:30:35
in my 1xxx1100 0x010101 0x001000 1xxx1011 0x000101 0x010101 x = now is free bit
yurume
2023-06-14 01:30:59
sorry, my example is in UTF-8, not in your scheme
DZgas Ж
yurume sorry, my example is in UTF-8, not in your scheme
2023-06-14 01:32:04
I made a translation from unicode into my scheme, saving the characters, all the information, and freeing up space
yurume
2023-06-14 01:32:36
yeah you have free bits, but removing `0x001000` as I did will result in a *valid* sequence
2023-06-14 01:33:31
which is by itself, fine
DZgas Ж
yurume if I understand you correctly, removing the third byte `10001000` should break everything, right? is this what did you mean?
2023-06-14 01:34:09
Yes
yurume
2023-06-14 01:34:11
but this implies an overlapping subsequence issue which may be problematic in bytewise routines
DZgas Ж
2023-06-14 01:34:21
Broke one symbol
yurume
2023-06-14 01:34:27
yeah, one symbol only
DZgas Ж
2023-06-14 01:34:34
Same in my
yurume
2023-06-14 01:34:39
that's a definition of "self-synchronization"
2023-06-14 01:34:49
yeah your code is also self-synchronizing
2023-06-14 01:34:57
but not safe for bytewise routines
2023-06-14 01:35:17
I don't doubt your code is better than most multibyte sequences
2023-06-14 01:35:37
and can be useful for cases where you want self-synchronization but don't use bytewise routines ever
2023-06-14 01:36:01
unfortunately bytewise routines are there, and they are thoroughly optimized (because they don't assume anything)
2023-06-14 01:36:22
so if a coding scheme can work with bytewise routines, it will get optimal routines for free
DZgas Ж
yurume but not safe for bytewise routines
2023-06-14 01:36:56
I don't know what you mean. In my program it works like this If bit = 1, then see if the next bit is equal to 1 or 0, if 0, then see if the next bit is equal to 1 or 0, if 0, then create a number from these 3 bytes
yurume
2023-06-14 01:37:12
yeah, "if bit = 1" condition should be a part of your algorithm
2023-06-14 01:37:31
bytewise routines... say, C's `strstr`, don't know about your bits
2023-06-14 01:37:52
they don't concern if a particular byte has an MSB set or not
DZgas Ж
2023-06-14 01:39:08
For beauty, I work with real mathematics, converting a number into bits, then dividing it by 7 bits, and adding this control bit to the end of each byte
2023-06-14 01:39:38
just real 0010101+"1"
2023-06-14 01:40:38
and in my program like this xxxxxxx1 xxxxxxx0 xxxxxxx0 xxxxxxx1 xxxxxxx1 xxx...
2023-06-14 01:40:57
and 7 14 21 bits saved
yurume yeah, "if bit = 1" condition should be a part of your algorithm
2023-06-14 01:43:03
As I understand it, it is important for unicode to have a clear understanding of how many bytes the end of a character is
yurume
2023-06-14 01:43:28
only matter when you implement Unicode-specific algorithms
2023-06-14 01:43:50
a generic string search works same for byte sequences and UTF-8 sequences
elfeïn
2023-06-14 01:49:00
that's true so long as the inputs are correct
lonjil
2023-06-14 01:59:18
UTF-8 needs to reserve 0xxxxxxx for ASCII compatibility.
DZgas Ж
lonjil UTF-8 needs to reserve 0xxxxxxx for ASCII compatibility.
2023-06-14 08:29:07
ok. Not problem.
lonjil UTF-8 needs to reserve 0xxxxxxx for ASCII compatibility.
2023-06-14 08:37:27
0xxxxxxx ansi cmptblty 0xxxxxxx 1xxxxxxx - new (14 bit of data) 0xxxxxxx 1xxxxxxx 1xxxxxxx - new2 (21 bir)
2023-06-14 12:13:23
BASED
2023-06-14 12:14:14
https://github.com/google/brotli/issues/591
2023-06-14 12:15:04
I was really surprised to find no ready-made build from Brotli. So bad
2023-06-14 12:46:11
https://github.com/google/brotli/issues/787
2023-06-14 12:46:19
real where
2023-06-14 12:47:33
My friend and I are trying to generate our own dictionary for BROTLI compression.... But nowhere is there a word about how to do it but the instructions directly say that it can be used ```-D FILE, --dictionary=FILE: use FILE as raw (LZ77) dictionary; same dictionary MUST be used both for compression and decompression ```https://github.com/google/brotli/blob/master/c/tools/brotli.md
2023-06-14 12:49:58
https://github.com/google/brotli/issues/697 So nothing was done
2023-06-14 12:52:26
It's good that the person literally wrote here that for ZSTD it is done like this - Whoop and done
VcSaJen
2023-06-16 05:01:12
I think Discourse forum is needed, or at least open Discussions tab in GitHub.
elfeïn
2023-06-16 06:45:45
😏
2023-06-16 06:45:49
it's this again
2023-06-16 06:46:03
actually I think drama magnitude increases every cycle
2023-06-16 06:47:25
🍿
2023-06-16 06:47:51
i think it's hilarious how people are entitled enough to complain about this strike
Foxtrot
2023-06-16 10:20:10
what about going back to self-hosted php forums like the old days?
yoochan
2023-06-16 11:07:27
Or IRC ? interconnected individual servers is the future
lonjil
2023-06-16 11:08:38
we can set up a channel on pissnet, the most reliable IRC network
yoochan
2023-06-16 11:11:50
pissnet ?! really
lonjil
2023-06-16 11:12:28
that was a joke :p
2023-06-16 11:12:47
Though I do run one of the servers so I can guarantee 90% uptime
VcSaJen
2023-06-16 03:58:49
About Win11 Auto Color Management (ACM): on what mode Windows Explorer works? sRGB mode, or monitor profile mode?
w
2023-06-16 06:15:02
defaults everything to srgb
gb82
2023-06-21 02:03:19
I vote we start looking into Lemmy as a federated Reddit alternative. I started a community on lemmy here: https://lemmy.ml/post/1404566
2023-06-21 02:03:47
let me know if we want a better icon/banner, if someone should have mod, etc
2023-06-21 02:08:00
I like Lemmy. It is federated, there are relatively decent clients available on Android w iOS stuff on its way, and AV1 has already moved to Lemmy
2023-06-21 02:09:29
2023-06-21 03:45:20
Yeah, and I believe it isn't coming back
2023-06-21 03:46:00
Simulping & many others (including myself) are working on an AV1 wiki, & Montec (subreddit mod) wants to move everything there
2023-06-21 03:46:09
The Lemmy will prob just be for news
BlueSwordM
gb82 Yeah, and I believe it isn't coming back
2023-06-21 04:22:53
Yeah, it's not coming back 😔
2023-06-21 04:22:59
Great decision <:Poggers:805392625934663710>
gb82
2023-06-21 04:24:58
real agreed
2023-06-21 04:25:20
I personally hope Lemmy catches on, I like it a lot currently
VcSaJen
2023-06-21 05:10:26
Discord is a private chat, not a public board, so it's not a real replacement.
fab
2023-06-21 11:39:00
So louis is a web font
2023-06-21 11:40:22
2023-06-21 11:40:33
2023-06-21 11:40:43
This scores 77 108
fab This scores 77 108
2023-06-21 11:42:09
No real one 41 50
2023-06-21 11:42:26
It's too thin to be a readable font
2023-06-21 11:46:04
2023-06-21 11:46:11
85 64
2023-06-21 11:46:36
This is a readable programming font
2023-06-21 11:56:43
2023-06-21 11:57:15
This you can read at 80 fps to 160hz
2023-06-21 12:02:14
Update with esme font I do 91 53
2023-06-21 12:02:37
Readability is bad on YouTube Music app
2023-06-21 12:06:12
With Promoteur regular I can do 123fps
2023-06-21 12:06:20
Is good enough
2023-06-21 12:06:30
But on readability is 37 less
2023-06-21 12:06:33
So
2023-06-21 12:06:39
86
2023-06-21 12:07:02
And is low for PC usage
2023-06-21 12:09:37
But I think that Galmur font 206 67
2023-06-21 12:09:55
Max I can understand is 67 fps
fab With Promoteur regular I can do 123fps
2023-06-21 12:13:08
92 133 I think