X-road, Wrangler, Mars, fswebacam, rps, SOS, Python 3.10

Studied Suomi.fi integrations, there's new docker access module called sidecar which allows accessing the secure network. See: X-Road - From virtual machines to containers blog post - @ NIIS.org.
Data Wrangling (@ Wikipedia) - Oh well, this is painfully true and familiar situation. Some random stash of data in some data lake or even worse just bunch of information copy pasted into Excel from HTML pages or something like that, and then you would need to handle it. Correctly, and automatically. Oh well, that's horrible. Been there, done that, over and over again. Always wondering why, oh why, the data was such pseudo random bloat of bits to begin with.
Had to watch NASA Perseverance (@ Wikipedia) landing. Amazing and great job. First time usage of the sky crane was of course the most horrifying part. Really waiting to see the Mars drone flight test results. Interesting concept allowing much greater surface access than traditional rovers.
Installed software to new server, needed a software to capture single video frames every now and then. I tried several approaches and as usual, found out that the system requirements wary greatly: fswebcam 11.1 megabytes, mplayer 663 megabytes, ffmpeg 695 megabytes, , vlc 1482 megabytes. All of these programs can be used for simple task of reading a single frame from a webcam and finally streamer whopping 2025 ... kilobytes! haha. But why? Improved my random number generator by adding one really bad web-cam with high noise to as one random source. Haha. Finally I ended up using the fswebcam which is pretty awesome. Captured million still frames hashed those and didn't find as single match. All the frames contain nice amount of noise... Even if the camera is in totally dark space with one led light source and CPU fan.
Other great reads: 12 requests per second: A realistic look at Python web frameworks (@ suade.org). I'm not surprised at all. But what causes the actual end result is often huge load of all kind of overhead + lots of bad unoptimized code and doing things which do not matter at all. I did read many stories commenting this post and the end result was pretty much the same. ORMs, doing lots of selects, reading unnecessary data and all kind of overhead caused by the framework and the code on top of it, lack of caching. Not carefully thinking what kind of transactions and locks should be used, etc. Basically now knowing or understanding what actually IS happening and how some simple query could be extremely expensive to run in some cases. But it works very well in the dev environment with a few records. Nothing new here in that sense.
Scaleway Object Storage is nice for backups and stuff, especially the C14 for long-term-cold-storage. But for active data it really sucks. Constant issues with availability and request rate limits. The active data had to be moved else where with better availability. After certain point most of requests will start to fail and retrying constantly just makes the situation worse.
Re-read this article and still liked the facts that they're acknowledging the shortcomings: Response to analysis of ProtonMail’s cryptographic architecture (@ protonmail.com). Of course it all comes down to threat model and what level of protection you'll be needing. Usually it won't even help, if the tech would be perfect. Next problem are the users.
There's nothing new that "hardware" devices run really old software and it's a known security issue. Smile. - Router Security (@ schneier.com / Schneier on Security).
Finally upgraded home server hardware, now it's bit more than 7 times faster than previous hardware was. Migration went smoothly, but it took a lot of work to prepare, configure and test everything in advanced so the switch over would be as smooth as updating DNS records. And it worked.
Python 3.10, nice. Anything interesting? Let's see. Distutils finally going deprecated, ok. Using setuptools (@ pypi.org) instead in future, nice. Structural Pattern Matching (PEP 636) (@ python.org) improved, I just love it. It's one of the features I've missed most in Python. Also the examples using old text adventures as samples is awesome, haha.

2022-04-24