Backups, exFAT, Rsync, WebAuthn/Passkeys, Element, Btrfs, Zstd

1. Now I've got backups configured so that it'll run completely automatically, only off-site / off-line backups need to be refreshed at times by plugging in external drive. Otherwise everything is fully automated, including file system snapshots and backups to 16 TB Exos backup drive, which contains daily copies for 12 months. Yet the external backup is updated in background, based on the previous snapshot taken automatically on system shutdown.

2. exFAT / btrfs - Funny observation. Most of my notes, which on exFAT took 128 KiBytes/note, are now compressed and inlined on btrfs tree, not even reserving a single (full) storage block.

3. Incredible amount of tuning with some Linux services, either the script doesn't run, runs too late, too early or... Joy of endless tuning, or pain? Who knows. But I won't stop until it works. I want 100% automation with all daily, weekly and monthly tasks. And some of the monthly key tasks I can trigger manually (which require physical things to happen as well), but rest of the script must be fully automated.

4. Rsync - I said: When using rsync always if possible use --preallocate and --whole-file (depending on copy data and media type) and in some cases --inplace. ... And as expected, was immediately disputed. ... My reply to that: That's right. That's why I said in some cases and "copy data" and "media type". As example with CMR disk using ext4, it's pointless to update only the 1% of the file, because you have to read the file and reading and writing is just as fast. So finding that 1% which to update, is extra effort, compared just to copying the whole file. - Also in many cases when you setup the job, you already know, if there are 2 TB files, which are 1% modified, and or if there are files which are never modified but just added and deleted. With SSDs and especially if using snapshots like with btrfs, then the inplace option is beneficial instead of the whole file. I found out that many tasks became much faster with whole file, especially encrypted whiles are 100% modified, it's pointless to read the file and detect that it needs to be fully copied.

5. After having long experience with different FIDO2 / WebAuthn / Passkeys - I would say that I really love G310 eWBM security key, it's so small, nice and easy to use. Something to really like. Only minor drawback is the need for the udev rules, but that's easy to fix.

6. Element X finally got image annotations. Yet they still allow sharing only one image at a time, which is quite an limitation. But maybe it's fix the the issue what I've reported earlier that the client used to get totally messed up when sharing batches of images, especially using bad network (while traveling). In worst cases that even wrecked encryption and everything.

7. Enjoyed running btrfs-convert on multiple data drives, switching from ext4 to btrfs for data integrity and more modern file system with snapshots. For key systems, I did copy data to two destinations, verify the copies, check original data's integrity and then reset the original file system and recreate it. But for some of the data drives which have very simple file system basically just a few directories with bunch of large files, and I already have copies and mirrors of all data it didn't make sense to rebuild everything. So I ended up using btrfs-convert and it worked like a charm. Only thing which slightly surprised me on the first run was the execution time, it seems that the whole drive is verified during the conversion, which means that it takes a long time to read the large drives completely through. Yet it wasn't much slower tan just read testing whole drive with a single read pass. I did expect it to only read metadata and write it in new format, but of course that's not directly possible, because the checksums of the data checksums need to be added, which forces reading all data for checksumming. - Well, after all it worked as expected, I didn't ask UUID to be converted, so I had to define new UUIDs for the mounts in fstab, but that's all.

8. Compression - Time to remind people again WebP is better than PNG, JPG, GIF on everything. AVIF is even better for lossy tightly compressed images, ie. most of web photos. Similarly it's time to let ZIP / DEFLATE go. Zstandard (Zstd) compresses better and is way faster, making it superior on both meaningful measures, ie. efficiency. And uncompressing is even faster than comressing when comparing to DEFLATE.

2025-06-15