Email, Credentials, Archiving, CBOR, Hacks, Spanner

• Have I mentioned bad configuration of SMTP & SPF before? Today I received complaint from one IT department, that they can't use abuse my domain to send email, because because recipients block connections due to the SPF rules, I've set... Great job guys. I think I should tell them to order consulting to resolve the issue.
• Once again, please do not use any default or shared credentials. I wonder if this will never ever stop. Anyway, common answer usually is, that it doesn't matter if default credentials are used. Ok, well fine then. It's just pointless to point out security issues, when nobody really cares about security at all. - Another strange misconception is that if the system doesn't (yet) contain any critical data, it doesn't need to be secured. They claim it can be secured, after critical data has been loaded. Well, that's obviously very bad attitude. Because, if the system is compromised when it's insecure, it's extremely hard or impossible to secure later. Without very tedious processes. Which nobody is practically interested doing in real world. Surprisingly this time the default credentials account did not have bulk admin rights on the server.
• In one case archives of data from past decades was seriously mislocated, allowing wide (but not public) access to the records. So typical. Documents are secured again. But nobody got any clue, how many (and if any) copies or unauthorized access has happened. In this case I guess that there were not unauthorized access, because we just don't happen to know about it. So, all is well, I guess. It's also probable that most of the data in the archives should have been destroyed after the retention period, which had passed by decades in this case. Anyway, it's good that the documents were recovered, because some of the documents are still valid archival documents and are required to be kept. So actually loss of the documents would probably have been worse than someone making illegal copies. Of course it's hard to say, if any of the material got further mislocated aka missing. How this popped in my mind? Well I were just browsing through old NSA security posters.
• Quickly tested CBOR RFC 7049 data serialization and gPRC RPC framework using Python 3.x libraries (cbor, grpcio). Yet currently I don't have need for those. Yet in broader terms using something like CBOR could be nice. Most of stuff is going over JSON currently, even if it got it's drawbacks it's still nicer to use than XML.
• Ugly hacks, more quality code. In one case files were named like tablea_timestamp. That was great, until at the nearly end of the project there was sudden requirement to add tablea_extra_timestamp file. Great. It was required to be really quickly, And all the filters were based on the start of the filename and fully configurable. So this presented a kind of problem. Options: A) Import in order, where longer names are processed before shorter names. Problem? Transactionality fails can happen. Each table processing step individually rescanned source file directory so file types could get mixed, if new files have been added between steps. Of course it's possible to refactor this stuff to use single directory scan and sort it. But it's not quick to implement. B) Add some kind of logic for excluding longer parallel matches. Uh, yeah, doable, but not quick. C) Just change configuration so that instead of tablea the matching filter for some files is tablea_2 excellent. Now it works until year 3000 is reached. This is yet again perfect example, how kind of bad code ends up produced due to ugly hacks and quick change requirements. Do this now. Quick'n'dirty. -> Done. For fun the filter could be tablea_201 and then everybody would be wondering why the stuff blows up when 2020 is reached. In general, I would have preferred just naming things so that there aren't conficting prefixes with different lengths. Because that's how it was agreed about 200 tables earlier, until we got N+1st table which did break the rule.
• What DBAs need to know about Cloud Spanner - Keys and indexes - Nothing new in that article. Hotspots, so classic. Also issues are exactly same which I wrote already about when usign Google App Engiens old native database, bottle necked sequence generation.

2019-11-17