SaaS, Python, Decorators, Databases, Superfish, Gogo, Malware, IoT

Post date: Feb 22, 2015 5:57:26 PM

• SaaS business 40% rule
• What's new in Python 3.5
• Python 3 logging - I just encountered interesting thing. If I call at any point logging.info('randomstuff') after that all calls to logging instances start to log to stdout. Why is that? Because I documentation doesn't reveal any reason for as far as I know. How did I find it out? Well, instead of the instance called logger I accidentally wrote once logging and that changed how all of the other instances the logging were working. Some discussion @ G+ - So afterall it isn't bug. It just changes the way how data being propagated to the main logger is being handled by default.
• Simple Service Discovery Protocol (SSDP) - Enjoyed upgrading firmware to firewalls which were vulnerable with SSDP reflection and amplification attacks.
• Checked out Gateway Load Balancing Protocol (GLBP)
• Did some work with MS SQL 2012 Management Studio. I just can't believe how bloated it is compared to other similar tools. Installing it took a long time.
• Asynchronous Python and Databases - Sometimes asynchronous stuff can save a lot of resources when right subsystems are used. But on the other hand it might implement things which are really hard to debug. One of my projects is now using mostly fully asynchronous processing where there is just inbound and outbound queues. But in this particular case the solution is perfect? Why? Because everything is written really asynchronously so requests are send and response might arrive or not. When response arrives it's processed. So it's similar to most of UDP related code. There's no state what so ever being maintained. Queues are persisted stored so even killing the process won't screw up things. Of course a few responses could be lost, but that doesn't matter. Data will requested again later. But same rules apply to this, asnyc isn't ultimate solution, should you always write multithreaded, multiprocessing or similar code? No? Yes? It dependes from the situation. Using these technologies where not needed can just make code seriously complex and brittle and cause totally new performance problems. [AsyncIO, Coroutines, SQLAlchemy, Concurrency, event loop, tasks, I/O]
• Similarly the processes which are extracting data received from that subsystem, is fully lock free. It runs as parallel python processes using process pool and doesn't use any application or database level locks. Providing maximal performance.
• Something different: ARX 160, A2/AD for Submarines, Anti-shock body, FN SCAR, HK416, Watched The Real American Sniper by History Channel, Air purifier, Electrostatic precipitator
• Laughed how two guys tried to set routers custom MAC address for over two hours. Then they came to me and asked if I could help, because they had invalid MAC address. I smiled and told them that there's no such thing as invalid MAC address. What about just using another browser, so the JavaScript cod wouldn't fail and you could set what ever MAC address you want to. After they stopped using Chrome which failed and started to use old IE, suddenly all of the 'invalid MAC addresses' became very valid.
• Received a few questions about how NAT works with local connections and replied to it:
  • I've encountered several different situations with that kind of configuration... It's really hard to say how certain devices work without knowing exactly what kind of configuration those are using.
  • 1) Smart routers route that traffic locally.
  • 2) With some routers it won't work at all.
  • 3) And some routers handle it like any other connection. Basically forwarding traffic to ISP router which bounces it back. Nice drawback is that you're consuming your Internet bandwidth with your own traffic.
  • 4) Proper firewalls usually allow you to configure it as you wish including all options mentioned above.
  • 5) In some cases even the Carrier Grade NAT (CGN) can prevent loopback connections to the same network using public addresses. Which is really annoying. I've encountered that a few times live. Problem was fixed by using "internal addressing", but it's still very annoying that the public addresses won't work. Now when I said it, I actually know one such network right now. It's really silly because you can use server.example.com from everywhere, except from the actual network where the server is. When you're connected to that network you'll need to use 10.x.x.x addressing. Which is as far as I know it really duh. But no can do. It's not my network to administer.
  • 6) Of course NAT configuration allows all kind of tricks, which haven't been accounted by most software or users. Like mapping ports, IP addresses and anything can be mapped on the fly. Like all outbound traffic can use different IP address than Incoming traffic. Even if computers are in same local subnet, those might have completely different external addresses which might use addresses from different subnets or even service providers.
• Explored yet another NFC based alternate payment solution. In this case the purchases are invoiced and the service provider also offer credit option. So it's just another alternate credit card provider. (Not using the common payment solution) but 100% their own private infrastructure. As I've said, payment solutions aren't that complex at all. It seems that many are just making those overly complicated or at least thinking that those are so complex, without understanding the details of the system. But when you've been implementing multiple payment systems and account & invoicing solutions, you'll notice that it's basically the same stuff just in different packet. Identifying customer, getting strong authentication to the payment, and then getting the money for the credit purchases (credit card) actually from the customer of course this step can be also made before hand aka prepaid cards.
• Thank you Microsoft for improving user experience. 2008 R2 says MSVCR100.dll is missing. But Windows 2012 gives just unclear crap about stuff not working. I just love totally pointless error messages. Things fail, live with it.
• Had to run sfc.exe /scannow to find out if some libraries are incorrect.
• Had some problems with cx_Freeze and running those binaries on Server 2012. Works well everywhere else. Fixed issue by installing x86 libraries vcredist_x86. This is related to earlier whine about MSVCR100.DLL
• Had very boring discussions with guy who doesn't believe that Samba and Server Message Block (SMB) aka TCP port 445 is only port that needs to be opened for modern file & printer sharing. Yes, there are potential security issues which are very related. I guess he has some other kind of configuration error and just believes the port stuff is what matters.
• Design & Tech of BBC's new mobile site
• Equation Group - There has been a lot of talk earlier about replacing device firmwares with malware. But everybody's been saying that "it's not possible". Even if it clearly is. This reminds me from early MBR viruses. - Link to older related stuff by a hacker.
• The USB relay (Fanny) also shows clearly why slow speed RS cable with LEDs is better option than using USB memory, which got too high speed connection & plenty of storage space.
• Received "Purchace Order.exe" - Clearly targeted attack. No anti-virus tool detected it as a virus. But yet it doesn't make any other sense to send such files. Naturally I didn't execute it. - Heuristic analysis shows that it's probably Trojan / Spyware like application containing key logger, data stealing and botnet features. How nice!
• I'm just so tired of reports which say that program is taking 100%, 50% of 25% of CPU ... Blah blah, all that BS! Those are absolutely meaningless values. If there's one thread which is in running state all the time. Then there is one thread in running state all the time. It doesn't mean that program is using 12,5%, 25%, 50%, 100% or such CPU utilization. It's totally dependent of what kind of platform it's being run. But still I see dumb people talking about such things all the time. Like Anti-virus product X is good, it's only using 25% of CPU ... Braindead. Did they ever think that Anti-virus program which is using 100% of CPU with low priority might be much faster and yet "load" the system less than the Anti-virus program which is "only" using 25%? I've seen so many darn programs which take 4 hours to process data and when you're looking for bottle neck it's locking and stuff like that. So basically there's no clear single bottleneck like disk I/O, CPU, memory or so. Programs just mostly idle because those are written so badly. But is it a good thing? Because now the programs are consuming less resources, right?
  • Btw. I see that multi-core programming has mind f... incompetent developers. They don't understand things like process priority at all. I'm so sad.
• Tuned many templates related to one project and fixed ridiculous typos. It's funny what kind of stuff you can produce when you just write out things as fast as possible and never check it. I guess you've noticed that when reading this blog. Smile.
  • Finished a few data transfer projects a work. In these cases customer needed reliable and quick way to get data from A to B. Old solution was slow and unreliable.
• 7 Key Saas Metrics You Need To Be Tracking - Old stuff, nothing to add
• A new development environment for Finnish National Service Channel (Using ESB aka X-Road technology) (Kansallinen Palveluväylä in Finnish) has been opened. - If I would be unemployed this is something I could be exploring in detail and starting to map possibilities it would offer.
• Did some work with css, png, base64 encoding and basic stuff like that. Boring daily tasks.
• A few incidents again. Data security is total joke. And cloud or private cloud doesn't have anything to do with it. Even public cloud services from major players are just simply darn secure. Compared to all kind of screw ups by helpdesk and even direct and plain unauthorized use of data for totally other purposes. I always laugh when people tell about their logging and other solutions. What if the backup administrator just simply takes full copy of the system and then does what ever he wants with the data. We have this blah blah logging. Well, nothing is visible in that logging when people with right privileges do their job. Or screw ups... Stealing data and delivering it to third parties without authorization from the rightful data owner. Many guys don't even understand that there are also 3rd party rights being violated there. Let's say that you've been buying stuff from webshop X. Then company administering the webshop just releases the whole database of that webshop. Now privacy of the customer of that webstore is also getting violated as well. Does it really matter? Most probably now, but there can be unforeseen consequences. Do people understand stuff like this? Well, usually they don't understand nor care.
• Read articles about NSA SIM key theft from Gemalto as well as several Superfish (Lenovo) related technical stories. All this is something that was thought to be "impossible", lol. The Intercept The Great Sim Heist
• Found SuperFishIEAddon.dll from a few Lenovo laptops. It seems that Windows Defender doesn't remove those nor related registry keys.
• Enterprise Aplication Integration (EAI)
• Retail, WebStores, Online marketing, how consumer shop today? This is how I do it - Depends from retail sector. But I personally always first compare and choose product online and then I might pick it up from store to save delivery costs. Of course daily grocery is different, but all other stuff is bought using this compare and select and then just pick it up. Of course some dealers offer really competitive pricing + free shipping, which basically means that I don't need to bother to pick it up and I can save time and money. That's the preferred option. Only stuff I don't usually order online are clothes and daily groceries.
• Gogo is doing MitM and serving fake SSL certs - Noticed similar issue while I took Norwegian flight. Yet I'm not sure if they were using Gogo, if not, they're still doing the same stuff. I've also seen some hotels doing it. At least one 5 star hotel in Slovakia server me invalid certs for my own server as well as broke banking security.
• Internet of Crappy Things - I couldn't agree more! I see all the time that even businesses are absolutely full of seriously flawed and fatally mis-configured systems. IoT will make things at least 100x worse, even if we don't count the NSA / intelligence agency tinfoil factor in.
  • It seems that hubiC file upload is still broken - It seems that hubic.com hasn't yet fixed the annoying UX fail. If you upload too large file. Guess what, they'll just fail it at the end and *restart* the upload because it failed. That's just so ... awww.... Uploading 2 GB file can consume 20 gigs of bandwidth when you just leave it running for several hours in background tab and wonder when it's getting completed. Absolutely horrible user experience.
• Fine tuned my bottle.py using projects to use decorators extensively for stuff like etags, user rights managements per url, etc. Bottle is very similar to Flask and other light Python WSGI style frameworks like Google App Engine SDK. Only question is what you should use to form the etag? It's easy to mask all etags to look similar using something fast like crc32. The only problem is that per function (url being server), you have to decide what's the best method to detect is page has changed or if you fully render the page and then conclude that the result is still same as it was earlier wasting a lot of resources on server side. For these shortcuts I had to add one project wide shared etag validator configuration parameter. If it's changed, it causes all all existing etags to be invalidated at once. This forces pages to be fully reloaded which otherwise use something like DB last modified timestamp to lighten etag processing. If this isn't being done correctly you'll end up having crappy site as so many others are. If the site admins say that you should empty the browser cache to get the site working, it only means that the site admins and webmasters there are just totally incompetent. If change is coming, they should shorten TTL values as well as take care of invalidating all existing etags (at least in cases where it matters) to force cache content to be properly refreshed.
• It seems that I found two computers affected by one Lenovo Yoga 2 and one tablet. Thank you . You can't simply trust anything nowadays.
• Checked out HTTP/2 draft httpbis http2 as well as related HTTP/2 HPACK for header compression. These things add just so much complexity. After a while situation is like it's now with TLS libs, Kernels and many other complex tings. Just only a few proper existing implementations because things are so complex, nobody want's to make their own from scratch. Even those can be seriously buggy as we have seen in the past.
• Excellent post about Python, ORM and peewee by Charles Leifer.
• Tori.fi seems to exclude some special characters from the password. So they don't store the password user gave. This is just as bad stuff I've been describing earlier with some sites. Like Nordnet which drops case information from username, but they forget to tell about this to the user. Or at least they drop it earlier, I haven't it tried lately.
• Dshell a forensic analysis framework by USARmyReasearchLab using Python.
• A good writing about Facebook and it's spying power. - Get your loved ones off the Facebook
• How to disable social features in Mozilla Firefox - I just wonder why every god .... application has to be so full of s... social stuff. I hate bloat.
• When going through a few competitors programs, I noticed that at least two companies have ripped some parts of my python code and used it for their own projects. Naturally without any attribution but I'm just personally happy about it. I'm glad that they liked it so much.
• Checked out Honeytoken - Nothing new, just one adaption of old stuff.
• It seems that my bank got a really bad triggers, they're sending me the same loan offer several times a week by traditional mail. It's interesting to see what kind of transactions trigger automated marketing from which entities. But triggering same trigger on 4 days during the same week and receiving same paper mail over and over again seems just so ridiculous to me. Ok, they've failed earlier too. Some times they send me targeted offers about the service I'm already using. So they're using really simple triggers and skipping all sane checks that should be done.
• Backblaze released the HD reliability report dataset.
• Memcached vs Redis comparison - Bit silly comparison, because Redis can do so much more than work as simple cache and on the other hand memcached is designed just to be expiring distributed fast key value storage.
• Very nice list of non-standard data structures for Python