ETL, State Machines, UTF-8, Backup, .SRT

• Once again disinformation from people. One team claimed they'll need faster servers because production goes so slow, when they run their ETL jobs. Yet, they didn't prioritize the tasks in anyway. Issue solved, now 96% of the resources go to the back end job (with lower priority), and the higher priority production jobs work just as before still consuming just a small fraction of the resources. All those claims about needing faster servers, were based on pure lies and incompetence. Fact is that in this case it doesn't matter how fast the ETL jobs complete, problem was just the ETL jobs slowing down the production, because tasks were run on same priority. - Of course it's not always this easy, but for read only committed tasks, it's quite easy. Things could get more complex, if the background tasks would make big writes to the database. Priority inversion can easily happen in this kind of environment, especially if simple spin locks are used.
• Two interesting articles about State Machines. I personally prefer state machines. I've seen so many programs which are clearly broken because there's no proper state machine which would guarantee safe transition from state to another. Yet, if possible, I like to avoid having complex state to begin with. But if you can't avoid it, then Using Finite State Machine is an excellent idea. Oh, the articles I mentioned in the beginning: Never use state machines and Love the State Machine.
• Encountered few environments where the locale wasn't set to UTF-8 version. Sigh. Ok, anyway, ssh won't show smileys on terminal, but sure, it will stop some badly written applications from crashing when printing UTF-8 data to ASCII output. Also latest Python sets Windows terminal default output to UTF-8, which will save even more trouble from many many users and developers.
• Made backup service comparison, Backblaze, Crashplan, Carbonite, SpiderOak, Acronis, Storage IT, Duplicati 2, Mozy, SOS Online Backup, Storage IT and so on. Unfortunately results are proprietary. But I tested out Backblaze, SpiderOak (Enterprise), SOS Online Backup and Storage IT. Now it comes to the service concept and pricing. Initial impression after checking the services is that SOS Online Backup looks really good.
• Once again wondered really bad credentials management. How insecurely credentials used to manage the key systems and data are being handled in many cases.
• Found one way to map directories into drives on Windows, which also works on server and doesn't require running user specific subst, or something similar (mounting as local network drive, or so): [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\DOS Devices] set "T:"="\\DosDevices\\C:\\Temp" - Btw. experience has shown that sometimes Windows updates seem to clear this data. - For some very commonly accessed paths, this approach is great on desktop use. You can directly refer to a drive letter, instead of navigating long and complicated paths. Yes, I've used subst for decades, but this is better for desktop. Even better, it works with Windows Server too. Very useful when migrating servers or something. Old server might have had several drives, no need to deal with the configuration if you merge to single new larger raid array, and then simply map paths from that array to drives. Everything can be in a single partition configuration. Also some programs don't like to be split between multiple drives. In those cases, it's awesome to create volumes and mount those to directories, which is trivial from the standard storage / partition / disk management. As example, some service providers provide N sized volumes, but you can't create larger volumes, nor change existing volume size. In those cases, it's important to efficiently merge those volumes into single usable storage space. As nice as symlinking key stuff under home folder on Linux.
• UpCloud published awesome new pre-configurations with extra disk space and memory. I've been resizing lot of file systems lately, and checking system configurations.
• Had to write a program to fix a few .srt subtitle files, which were in format that VLC didn't like. It seems that utf-32 can be tricky. I don't know why the English subtitles were in this format to begin with. But maybe it's because the movie had Asian backgrounds. And maybe they just prefer utf-32 instead of utf-8 there for good reasons.

2019-07-02