AWS, Future, Data Structures, Microservices, Process Memory, World Views

Post date: Apr 16, 2017 4:56:47 AM

• Read about new AWS services, Lambda@Edge, AWS CodeBuild, AWS Greengrass, AWS Athena and AWS Shield. Especially Greengrass is interesting platform for processing data using IoT devices using MQTT rules. Lamda@Edge provides distributed computing on Cloud using the Edge CloudFront edge servers. Yet those can't actually call other resources, so it's just really cloud based processing of data, using RAM. There's also tight time limit for processing time which is just 50 milliseconds as well as memory is limited to 128 megabytes.
• Read long article about future visions and technologies. Not surprisingly it listed: Unlimited free cloud storage, robotic service-oriented jobs, trillions of IoT devices, lot of 3D printed machines, medical parts, implants and even intestines etc, Internet of Things (IoT) communication implants, big data managed democratic countries, lots of artificial intelligence (AI) usage, driverless cars, no need for traffic lights.
• Data structures for external memory - A very nice article. Nothing new, but it's nice to see graphs and get some background information, if you didn't know it already. - I've written so much about inefficiency, so it's good to check that you code isn't horrible. (In case performance is required)
• Read a few articles and discussed with friends & colleagues about Microservice Architecture (MSA). When the system(s) have been designed from scratch for micro services. This is great shift, because I've seen many projects aiming for hugely complex monolithic All In One (AIO) solutions. Where one system does it all, using extensive standards and logic, using staggeringly massive interfaces. One interesting solution is Micro Service Oriented Architecture (μSOA) - http://baiy.cn/doc/byasp/mSOA_en.htm -, where you'll get some of microservice benefits from architectural standpoint but much less overhead. I personally really like the fact that with this mode you can avoid networking and IPC communication overhead. Which is common reason for smaller projects not to use microservices.
• Nice post by Julia Evans - Why can't I just see how much memory my process is using? - Yes, and that's just small generic part of it. Truth is much more complex. Like memory compression and other stuff. Yet all this complexity and "relativity" causes some funny effects. You use (or write) program which 'allocates and accesses' memory, to shrink other programs memory usage? What? Or you can even manage that by just accessing suitably sized set of files stored on disk repeatedly. - Yes, that's very true. Because your application (or disk cache) puts on purpose pressure on memory management, it leads disk cache evicting pages as well as other processes to be swapped out or compressed. So, if you think you have "too little free memory". You can run very simple program to allocate N gigabytes of memory and access it repeatedly for a few minutes. After that you'll simply release that memory. And now you got plenty of free memory. - This is one of the exact reasons when people say "their system is out of memory", it just indicates that they'll completely (usually) lack understanding of what memory even is. Coming up with 'right amount of memory' is quite complex science which is so complex, that at least I can honestly say that I don't master it. It would require lot of case specific testing, and even then it would be most likely related to the question what's acceptable performance for that specific task / system. One of the ways to detect that is to of course watch performance, amount of page faults and amount of disk I/O / active disk time on Windows. But this is also one of my favorite topics, so I've written about this earlier and most likely will write in future too.
• Views of the World - Amazing site. As I wrote that geolocation aware web app, as well thought about CDN networks. I encountered a few of the matters handled on the Views of the World site, like population distribution on planet, etc. But this is much wider topic and there are so much more data available nowadays. So just take a look and learn. Accessing the site was enlighting at least to me. Nothing really new, but lot of aha moments, that's something I thought or didn't thought. Now I got the facts. - Discussed world market for business sector X with a few guy, it was clear that when they were talking about the market, they did mean EU & US. It seemed that they completely ignored Asia. So they had good view of their market, but it was clearly lacking global view. So there's potential for such product or service clearly in areas which they weren't considering.