Embedded Files
Post date: Jan 24, 2015 1:41:42 PM
Dumping stuff from backlog. ;) As fast as possible... (1 of 2)
One invalid path caused system to hang without any error messages, even in logs. Logging doesn't reveal when task was finished, even if it was started. Lack of proper exception logging and handling. Caused huge mess hard to debug, even if reason was slight misconfiguration. As we know, making "program that mostly works require 1 unit of time", then making program which will work and handle all kind of exceptions correctly and also give clear logs / error messages can take 10-100 times the work amount. But some cloud environments require you to write programs so that the process / server can be killed at ANY TIME without any warning, it should not cause any problems what so ever. (Amazon, Google, etc…) If program works correctly, uses journaling and transactions, this should be the end result. If program is written badly without those primitives, result will be a program which works well in test environment ,but will end up causing total disaster in production when loads go up.
How is scheduler state (done state / running state) defined? Does execution "end" when process tells it's done? What if process aborts / crashes? This actually is one of the parts what I have been especially working with my ERP projects. Because I have error handling for most basic errors which I'm expecting. But as we know, there are well, about 1000 times more errors which are possible, but I'm just not simply expecting those. I'm usually expecting errors like slight misconfiguration or invalid server name, login/password, invalid paths, file locking, network errors, data structure exceptions etc. But if there are other issues outside that circle, I'm catching ALL other exceptions and logging those. If the logged error message would have been clear enough, we would have immediately caught the problem.e
In some cases I have gone to extreme lengths with checking all those listed errors, because I have wanted to make so robust solution that even if the process is killed at ANY TIME, next run will get things right. It checks for all temp files, doesn't ever write to "final file" directly, always uses temp files & lock files and prevents parallel execution by checking lock file time stamps. Naturally even this can fail if clock is played with or the locking process becomes really exceptionally slow and then is being executed again in parallel. It's hard to make programs which work correctly in all situations.
Project budgeting, coordination and cost estimation, customers point of view, user experience, usability. Technical sales support tasks. Key customer responsibility. Customer contacts. Service desk experience. Wide ranged experience scope. Lets get this done attitude. Don't spend days thinking why we can't do it, if there's no real reason. Self guided, I'll study what ever I think I need to know to get the job done. Risk assesment.
Wondered stuff about WiFi DropBox like WiDrop designs. I guess I've blogged this already. New Yorkers StrongBox.
I should have written longer analysis about different anonymous P2P applications and what are the pros and cons of each design. Sorry, no time for that. (Freenet, GNUnet, Perefct Dark, Rodi, RShare, Share, Sumi, Winny, Nodezilla, Mute, Marabunta, OFF System, GitaTribe, Alliance, Direct Connect, Hybrid Share, OnShare, anoNet, OneSwarm, Retroshare, Galet, Turtle F2F, Waste, Kerjodando, I2P, JAP, Psiphon)
It needs to be carefully monitored when Big Data analytics is beneficial and when it isn't. Should be quite obvious to any business. Article.
Comodo provides P/ CSR S/MIME certs for free.
Played with SciKit-Learn
No no time to play with Derby.js and Meteor even if those are interesting.
Reminded my self about off the record messaging, pfs, deniable auth, deniable crypto, malleable encryption.
SSD device block generation, Could reveal enctypted hidden container(s). Because places in files are written repeatedly which aren't expected to be written often. Copy on write file systems might have similar problems.
Gitian - Compare that delivered binaries correspond to source code.
smtp_tls_policy_maps = hash:/etc/postfix/tls_policy
Content:
ferraro.net encrypt
gmail.com encrypt
mailsuomi.com encrypt
State of outlook & email serurity is here: status=deferred (TLS is required, but was not offered by host mx1.hotmail.com[65.54.188.94])
Gmail TLS cert fails.
When certificate fingerprint monitoring is on, delivery fails if fingerprint doesn't match: status=deferred (Server certificate not verified) - So email can't get delivered to wrong destination or without enryption even if there's MitM attack. Successful negotiation: xxx postfix/smtp[805]: Verified TLS connection established to s.sami-lehtinen.net[]:25: TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits) - fingerprint & Public key pinning.
We should all have something to hide.
Ad networks are delivering malware, nothing new. But still very annoying problem. Encountered malware at GrooveShark and SendSpace.
Piledriver microarchitecture, Integer Factorization Optimization, MariaDB
Reminded myself about: Mixmaster anonymous remailer, Anonymous remailer, Nym server, Van Eck Phreaking
ITaaS, Software Defined Data Center
Check your browsers Cipher Suite Details.
OWASP TOP 10 list, legendary.
Misconfigurations do affect affect all security solutions, not only computer security. One building just had it's lock system mis-configured. Anyone having key to the building, could access all areas. I really mean all areas, like telecom and power distribution rooms and any spaces occupied by other businesses. Nothing new, just a minor configuration issue. It's ridiculous to first spent a lot of money for high end security system(s), and then totally mis-configure those.
VSRE standard - Yes!
Unhosted idealogy, I kind of like it. But I doubt normal users will like it too much.
Played with compressing data on paper. Practically useless, but nice play. See data compression.
EAX mode is better than CCM yet GCM is really complex. AEAD, AE.
TLS/SSL RC4 beast ephemeral keys, ecdh edh, session resumption.
Ladder of Abstraction - Yes, check it out. It's worth of it.
SifterApp, simpler bug & issue tracking
STARTTLS - No comments, just dumping
How much does the selected algorithm change the end result? Maybe a lot. Jump point search. It just makes a huge difference!
At least I have learned a long time ago, to avoid doing something complex cool and smart. Doing the simplest working solution is usually also the most reliable solution. Any smart tricks which add any complexity to logic of program are going to cause a lot of trouble later. I have very bad personal experiences even from most simplest solutions. Like comparing a few fields to few other fields, with logic. It was too complex for programmers to implement correctly and lead to annoying bugs that persisted for months. All this could have been very simply avoided by dropping the comparison logic and using simple unique ID on every record. It's worth of noting that most probably anyone later modifying the code would break that logic again, even if it would work.
zRAM / Compcache - Compressed memory, nothing new afaik.
Chromium Disk Cache structure - Small data is but in blocks - Yet I remember seeing some other article (later?) where they implemented SIMPLE DISK CACHE which did always use files for simplicity. It's funny how optimizations can be bad at times. Just like should MORK file format (msf) be used as index file for standard mailbox data... Or is simple maildir approach better? I'm using both. Or maybe data just should be simply in SQLite database? Who knows. Does it really matter if it works out?
It's nothing new that Telcos require all confidential information to be encrypted... I wonder why, I'm sure their security stuff must have known about this. Also, encrypting everything which isn't classified as public just makes sense anyway.
Studied documentation why Skype abandoned original P2P model and started to use cloud servers instead. Everything was done in very reasonable manner from product managers and normal end users point of views. - Sorry, no link.
Securing the Cloud, Cloud Computer Security Techniques and Tactics - Vic J.R. Winkler - Cloud Security, I really liked Epic Fail sections which showed what kind of epic failures there has been with each kind of security issue type. Made me smile. - type 1, type 2 vm. VirtualBox, Parallels, Virtual PC, VMware Fusion, VMware Server, Xen, XenServer. LXC (Linux Containers), BSDjails, OpenVZ, Linux-VServer, Parallels Virtuozzo
Gallery how different machine learning algorithms detect patterns. Excellent stuff!
FileDropper advertices 5 GB files, but actually 1.7 GB file is too big. - Btw. They'll still have similar issues in 2015. Upload large files and those somehow disappear or something similar.
Cross & double monitoring of services - Always use external and independent system to monitor other systems. Also deliver alerts "out of band" if possible. I've seen it happening so many times that when systems go down, there's no alert... Because the monitoring & alerting system went down too.
That's why they take memory snapshot first, which is trivial with VPS and then pick encryption keys from it to access encrypted volumes. This is well known method and works with pure hardware machines too with physical access. It's great question when you get to server, to shut it down or leave on. If on, it could destroy data, if turned off encryption keys are gone. I think it would require some individual case analysis before deciding which one is better approach.
Actually it seems that I blogged this later. But back in 1.7.2013 I also made memo which clearly showed that when you browse using DuckDuckGo with Dolphin Browser data is flowing over non-secure (non https) channels. I do have packet logs. But because sanitizing those would take too much time, I'm not going to post those.
Excellent chart from XKCD guys. Is it worth of doing? - I actually have printed this one on my wall.
Motorola is listennng in. Android phones do spy users.
Never trust Facebook. - Uhhm? Who would be foolish enough to trust it?
Functionality and differences between BeiDou, Glonass, Compass, IRNSS, GPS, Galio satellite navigation systems, signaling and design. As well checked out legacy eLoran and TIMU aia simple Inertial Navigation.
When it comes to the price tag for BI, a lots of people are shocked to learn how much some companies charge. For large companies 50k is kind of a low end instillation, and BI can cost over half a mil. However, the question shouldn't be how much is BI going to cost me, but how much can I expect to save/increase profits thorough this tool?
Studied documented Priority Inversion software problem with NASAs Mars Pathfinder spacecraft software. - Nothing new.
USB Hell - Drivers, sockets (4 different), charging & power issues etc, different cable connector types etc.
One thing has become clear to me over time, especially in the current financial crisis,” she said. “No matter the job, most of us no longer have job security. Our labor is replaceable.” - Yet another reason to keep studying.
PGP Tutorial - Yes, you should already know everything mentioned in this document. If you don't, just go and read it.
Additional VPN transport - TCP over WebSockets over HTTPS. This is great way to access all services when guest networks have TCP port restrictions. Yet much faster and more efficient (less protocol overhead) than tunneling over DNS which I did setup earlier
SIM card insecurities
Liked Hong Kong, Singapore, Sydney, etc. Peking airport WiFi was strongly authenticated (passport), but poorly secured. Which means that I could have hijacked anyones identity and made them look guilty. Afaik, this is very bad way of doing things.
Death of public key encryption? - Sounds even more viable in 2015.
The AEAD modes combine stream cipher modes with MAC constructions that developers don’t have to think about.
In computer security, a covert channel is a hidden signaling mechanism. Attackers exploit covert channels to leak messages across security boundaries . Side channels are the flip side of covert channels; they’re actual signaling performed unexpectedly.
Bank allows reusing authentication codes in case of some errors...
Tox.im - The way of secure (?) future messaging?
Dynamic real-time pricing
GSMA:n Wallet-POS APDU, GlobalPlatform Security Domain
SSL/TLS broken - Gone in 30 seconds
Checked out Mifare Ultralight and Ultralight C variant with crypto
Enjoyed psftp key management using registry editor... So wonderful.
SQL injections - Joining Union query-based injection, squeal, error-based injection, boolean-based injection, time-based blind injection.
True Hacking, hacking hard drives! - Excellent stuff!
Private / Public surveillance relationship - Just what I have said. If something can't be officially done. You'll just find a way to get it done unofficially.
Hidden Services, Current Events, and Freedom Hosting - Multiple hidden Tor services taken down and some of those tried to plant malware on visitor systems. Afaik, some of those pages planted back then do still exist (2015) with that yellowish background and some kind of frame near the top.
Watched BBC what's killing bees
Studied & Configured firewall: ICSA certified ipsedc & firewall, USG (Unified Security Gateway), SPI, SNAT, DNAT, BWM, ADP, IDP, AV, AP, CF, AS, Routing, HA. In long format, destination nat, routing, stateful firewall, anomaly detection and preventation, application classifier, intrusion detection and prevention, anti-virus, application patrol, content filter, authentication, system management, logging & monitoring, anti-spam, threat database, high availability, vlan, ipsec, sll, l2tp, real-time, dynamic malware protection, IPV6, security zones, dual stack.
Enabled PFS for my own server https://s.sami-lehtinen.net/
Lavabit, Silent Circle, Lavaboom, Pidgin, Empathy
End to end (E2EE), pre internet encryption (pie)
Python Statistics - PEP-0450
Watched GNUnet video - Nothing new in this video, I have been liking GNUnet project for several years. But if you don't know GNUnet project, check it out.
The GNU Alternative Domain System (GADS) (GNUnet)
Of course checked out HyperLoop!
I really like teams which get stuff done. Quick iterations, meetings, let's get it really done. Push and do. Test, iterate, quickly! I hate projects where every single question takes days and test iterations and stuff takes weeks or months. I love the feeling when stuff really gets done, and everyone got the same mood. I can do it. - Not the attitude, that I don't know if I really need to get it done, if someone else could do it some day or ... Aww!
Seems that 42registry is working again. My postfix also handles mails to myaddress @ s-l.42 or myaddress @ samilehtinen.geek properly now.
I've got separate email address for GPG only messages. If any email which isn't encrypted to my public key is sent to the address, it bounces back with link to my public key.
Windows server access policy management.
Yes, not using cache at all would make everything very slow. I'm now of course talking about using in session memory cache. If it's too small you can reconfigure it using browser.cache.memory.capacity parameter with Firefox. With fiber I never use caching. But yes, with 512kbit/s connection I unfortunately had to use disk caching too, to avoid re-downloading anything I simply could. But of course in that kind of situation and configurate you're really aware that you're not destroying all data between sessions. For privacy virtual machine with hardened configuration + tor is good idea. Otherwise there's no reasonable expectation of privacy anyway, as they're saying. In technical terms, there are so many ways to track users who do not harden those, that there's no reason to expect any privacy. As we have seen with all these NSA discussions, all technical options were pre-known already. You don't know if sites use some techniques or not, but it's reasonable to expect that they do use at least all publicly known techniques. And possibly some unknown. So making attack (or tracking) surface as small as possible, when looking for privacy is reasonable. Maintaining any data between sessions is just stupid. Always boot clean virtual machine, which is similar to other virtual machines, is best approach. Otherwise there are tons of things they can do to track you.
Btw. even if browser keeps cache, you can always clear storage paths.
One of things that doesn't seem to be known to many users is that many databases contain 'deleted' data for long time. They just don't think about it. Just go thorough all files stored by browser, you'll end up finding stuff that you woulnd't expect to be there, if you're naive. Right attitude is to expect everything to be stored always, and take proper care to destroy data when it's required. This is just like the issue with SSD drives. If you write something on drive, you wan't to destroy. There's no sure way to destroy the data from drive, without totally physically destroying the drive. You simply don't know, if the controller has written data to cell XYZ, and then re-mapped XYZ to somewhere else. Just overwrite it approach does not work in this case. And you can't even guarantee that the manufacturer tool could properly erase that cell.
Just final words. Etag doesn't have anything to do with "images", it's not tied to content-type at all. Next week I could release "css" tracking exploit, which uses etags, which is kind of css checksum. Uh...
RFID readers, NFC tags
IronPigeon - Yet another secure communication protocol
Low level flaws in 3G Modems might allow hijacking computers. SMS shell fuzzing usb internet modems.
Especially traps were important when processing money. Of course you can work around those using alternate code, but why bother, when there's existing library. Python Decimal Yes. I've known this for 20+ years, but this is just reminder. Yet another coder used floats for money and as we know, it's sure way to fail.
People often laugh at bad password policies, but sometimes password policies are just catastrophically bad. Like in some cases, shared passwords are used so widely, that it becomes impossible to renew the password, because nobody knows how to inform everybody who needs the password. That's one of the ultimate fails I have seen happening. No no, we can't change the password, because we don't have any idea how to inform the people who need it. Afaik, password should have been changed ages ago, and should be changed monthly or so. Of course it would be better not to use shared password at all, but in some cases that's also too difficult goal to reach.
Otr main key collision would mean that random numbers are bad, really bad.
IT security stuff: SSH shell accounts, when SFTP should be only available. When informed about this matter, they don't care, it works. Rights elevation exploits. Passwords stored at end users workstations. Not in secure environment. Exe files which are run with system account, but all system users do have full write access to those files. etc... Business as usual.
SQlite3 - Partial Index, Next Generation Query Planner
Checked out Zoined Retail Analytics in detail (with demo account)
Bruce Schneier's blog: " We're always making trade-offs between security and efficiency. "
iDRAC IDRAC DELL Server stuff, Remote management, etc, VNLP.
IPsec is crappiest protocol ever, at least in real world.... Yes, I'm having serious problems with it AGAIN. No words for this "#¤"#4.
F-Secure securemail requires that password contains symbol, but " isn't valid symbol, how lame!
I received email from F-Secure: "
This is a secure message, click that link right now!
That's just one more reason to use individual email address for every service & communication instance. You'll immediately notice that the message is out of context of that email address it was sent to. I've been also receiving UPS can't deliver packet notifications and other scams, which are only asking you to click link provided.
In this case this message actually is legit. Anyhow the first impression is quite much, yet another scam, delete.
I just had to spent quite a while analyzing message headers, body content and link destination & source servers, AS numbers, IP-addresses, before I decided to open this link with my broser."
Watched many PyCon CA 2013 videos
Great example how accurately people can be tracked using mobile phone data, even without GPS, see the cell sectors marked on map.
Finished reading book(s) Securing The Cloud, Innovators Dilemma, Enisa Cloud Security, PCI DSS Cloud, HIPAA check list, PCI SSC Quick Reference, The Dangers of Surveillance, "
Bitmessage: A Peer‐to‐Peer Message Authentication and Delivery System Jonathan Warren", Go Faq, 57 startup lessons.
Replaced my encryption key with 4096 bit RSA key, signing key is still old 1024 bit DSA. Still waiting gor Elliptic-Curve keys to pop into main release og GnuPG.
Studied Linux Kernel 3.11, especially Zswap seems to be very good trade off between slow I/O and fast CPU.
Studied distributed file system lustre.
Found a few sites which looked like a scam. I had to use wget to examine headers and content before opening the content with traditional browser.
Nice post about using LXC for network isolation.
Played with DebianDB in VirtualBox environment. See dbtechnet.org for more information. Databases used MySQL DB2 Express-C, Oracle XE PostgreSQL Pyrrho.
A few minor fixes in one project using Tomcat / Hibernate / MySQL made whole project 16x faster.
One coder used framework he didn't completely understand. Originally there wasn't password requirement for users. But when site got more confidential data password feature had to be added. Well how it was done? When user gave login and clicked ok, came page asking for password. But if you changed url at this point, everything worked. When I checked the actual code, the user form logged user in and password form logged user out if password was incorrect. Oh boy, what a joy.
Toyed with different index types and partial indexes and stuff like that with SQLite3 and PostgreSQL just for fun. Because PostgreSQL 9.3 was released.
Blog more about ... ERP stuff like: inventory reporting, financial reporting, sales commissions reporting, inventory management, purchase orders, sales statistics, customer accounts, distribution logistics, EDI, credit checks, electronic invoicing, invoice processing, EDI, stock management, dynamic (time-based automatic) pricing, web shops, system architecture, enterprise resource planning, SaaS, delivery reliability analysis, supply chain management, SCOR-model (Supply-chain operations reference), performance measurements.
Great source for studies edX
SSEBITDA for SaaS business profitability measurement
Beyond passwords, new biometric identification methods.
Some customers just got "Total illusion" about project scope, schedule, costs, etc... But I guess this isn't news to anyone.
I'm using SQLite3 for my many of my small utility projects to track processed messages, just as Bitmessage does... Why? Well because usully the main database doesn't have any flags, tables or reserved fields in database I could use. So I use another database to track and compare changes in another database. Isn't that great? Why I won't add fields or tables? Well, it would break several compatibility things, that's why.
OpenVPN uses the OpenSSL - Security Now show said that OpenVPN is not OpenSSL, it's more secure. I guess they don't know too much about OpenVPN at all? Because it's directly based on OpenSSL. Ouch.
I don't know what Hushmail's web font does. But it doesn't idle properly. If you leave hushmail tab open in mobile browser (Android), it'll drain 50% of battery in 45 minutes.
It's better to verify certificates fingerprint than it's signed by "official trusted authority". Because fake certs could easily have trusted signature, but it's much less likely that they got right fingerprint.
Studied Ulam Spiral for network node partitioning purposes.
Checked list of goals for Phantom project and read it's white paper. Goals of the project do sound really great. [ Complete Decentralization, maximum Resistance Against All Kinds of DoS Attacks, Secure Anonymization, Secure End-to-End encryption, Isolation from the Normal Internet, Protection against Protocol Identification / Profiling, High Traffic Volume and Thoroughput Capacity ]. Documentation contains important aspects like: routing paths, addressing, network database, encryption and authentication, and technical description. As well as the most important part, which makes Phantom better(?) than I2P, Tor or other existing anonymized file sharing software.
SQLite4 LSM Design - A very interesting read.
Lightly checked out Incapsula - Yet another CDN.
Checked out zBase - Yet another large distributed key value storage.
Reminded my self about Kademlia, Chord, Tapestry and Pastry internals.
Studied: Gossip Protocol
BitMessage Streams / Partitioning, how to prevent all data spreading to whole network. Yet maintaining the benefit of broadcasting not revealing who's the actual recipient of the message.
Verified TLS connection established to s.sami-lehtinen.net [x.x.x.x]:25: TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits) - Seems to be working.
Received: from x.x (x.x) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "x.x", Issuer "x" (verified)) by s.sami-lehtinen.net (Postfix) with ESMTPS id X for <my-mail@sami-lehtinen.net>; Thu, 19 Sep 2013 21:58:07 +0300 (EEST).
Anyway without cert check / fingerprint, using starttls (smtps) still foils passive attacks.
Donated money to EFF, PBS, SomaFM. Actually it was quite funny process. Because PBS didn't allow donations outside US. I had to send them feedback that I'm having a problem. I can't donate money to you. Nor I can watch your shows legally, I have to pirate those. I'll download their shows and donate money, that should be ok, everyone should be happy.. They fixed it, now they allow global donations using credit card.
Downloaded Dell iDRAC6 software and had to run it as standalone app using webstart because it simply didn't work properly with browser. Anyway, I'm glad I got java background so it was quite trivial to start the app using the jawaws.jnlp file from command line. Main trick is that the credentials needed for access are inside that laucher packet, so it has to be fresh.
Google knows WiFi passwords. - Isn't "central command cloud just so beautiful thing?"
Interestingly if x is None is faster than if x == None when using Pyhton 3.x
Accidental downtime caused. Ouch! One image hosting site had interesting stuff, using wget was way too slow, so I wrote my own version. I had it configured to work in two steps, first I go thorough all index pages and collect image urls. And then next phase is to fetch the actual images. Simple stuff. Because I've been playing with 8 thread CPU lately (i7) I had to try what I can do. So I wrote small Python app which utilizes both, multiprocessing and threading. Only problem was that I was running 16 processes and 32 threads per process. No, it wasn't problem for my system, but it was clearly a major problem for that small image hosting site. 512 concurrent HTTP connections fetching large images over keep-alive connections shouldn't be a problem, but in this case it was clear that it brought their site down. I wonder why they don't have rate limits or limit connections / IP address etc. But that was the unfortunate result of that. Of course I stopped my fetcher and tuned it down a bit. But it was funny to notice how easily some sites simply stop working, even if you're not even trying to distribute requests causing the overload.
Thoroughly studied 1024 cores - concurrency, parallelism, wait-freedom, lock-freedom, obstruction-freedom, algorithms, waitfree, lockfree, algorithm.
Had a Google I/O 2012 and 2013 marathon. It's much more informative than watching random TV soap opera or other "junk"
Read ARM64 and you.
Studied GCM mode. AESNI
I just hate some Android bloat as much as some Laptop vendors which pre-install all kind of crap. No thank you. I want as bare system as possible, and then select just a few applications which I actually do use.
Had long discussions about developers if UI buttons should have single role or if those can be multi-role buttons depending from the user situation. Some key buttons should be static, but rest of it can well be dynamic. Just like the few key buttons on Android (back, home, properties) and then the touch screen is "dynamic multi-role button section".
Lot of discussion and articles about fingerprint security and so on. But as said this brings it all together, fingerprints alone shouldn't be used for authentication. It's only identity.
Win-Win Consulting - How to deal stuff so everyone will be happy. Been there done that.
Article how to switch to HTTPS for free. But as said StartSSL certs aren't free for businesses.
Use CheckTLS to confrim your SMTPS secure email settings, yes you can use it to check also your service provider even if you aren't administering the systems.
People who are able to manage private and public cloud as well as communication between departments and understand business needs, integration and system architecture, will be in high demand.
Privacy and Trust - Bruce Schneier
Facebook email had it's problems: conversation with msgin.t.facebook.com[66.220.155.11] timed out while sending end of data -- message may be sent more than once.
Detector.io project - Detecting possible MITM attacks with Tor
It's true, after I posted my latest blog post (to blog) and posted it publicly to G+. It was then indexed by Google in less than 15 minutes, even if I haven't updated my blog for months. It's clear that Google+ posts trigger Google crawler and indexing activity. Posted a bunch of other stuff to see if it works and yes, it seems to be working.
NSA generates map of populations social connections? - Ugh, no news. Sociogram analysis isn't anything now.
Telehash yet another P2P secure MESH protocol.
Freelan yet another VPN software.
tinc yet another VPN software. Compression & ECC.
Robots2 - Extended standard for robot exclusion - Updated my own robots.txt's accodring it.
RoundCube WebMail all contains 2700 files... OMG! No wonder there's some "random seek". That's one of the reasons why I had to get that SSD, because starting simple app like VLC took for ever, due to sheer number of additional libraries in files to load.
How NSA attacks Tor.
Installed latest RoundCube and converted it to use SQLite3 instead of needlessly heavy MySQL. - Everything works well.
Had deep technical and analytical discussions about SQRL with a few friends.
The problem with SQRL article which our discussions were based on.
Lol, found yet another case where Windows IIS FTP service were continuously used to access server over Internet world wide using Administrator account. RDP was also available. Hmm, so much security fail here it's hard to describe.
Had to tune for a while to get Windows 2008R2 virtual domain support to work with IIS FTP.
Anonymity is hard, great reminder. You're probably deceiving your self if you think you're being anonymous. OPSEC
The problem with timestamps - Nice post, time is hard, even if it's easy to think it's quite simple thing.
/dev/sdb1 on /mnt/s1 type ext4 (rw,relatime,data=writeback,commit=300) - Great way to get better than SSD burst write performance on traditional disks. Just use writeback mode and long enough commit time. Btw. This also works great for slow USB sticks and stuff like that.
Checked out NFTables which is going to replace (?) good old IPtables at some point in future.
Checked out Quantum Algirthm - Made my head hurt. No, I'm not mathematician, too deep and theoretical for me.
Great TED talk, why privacy matters.
About buggy software. I just remembered classic Laser Squad targeting fail. In the game, when you were shooting far away, it computed the hit accuracy based on distance, weapon and shooter. But main trick was that if there was straight linear line from you to the target. You didn't aim 100 blocks away. You aimed to the first available block to that direction. When the shot was fired, it continued to travel thorough that point forward, because it didn't hit anything. This made it virtually impossible to miss. Let's make ASCII demonstration.
YT------------------------------------------------------------------------E
So instead of You shooting at Enemy you shot aimed and shot for Target point, and the projectile then just traveled to the Enenemy and hit it for sure. Nice fail, right?
Added custom HTTPS rule for my own server. In this case if I forgot to write the HTTPS and HSTS isn't primed my client will still use HTTPS.
<ruleset name="Sami Lehtinen">
<target host="s.sami-lehtinen.net" />
<rule from="^http://s\.sami-lehtinen\.net/" to="https://s.sami-lehtinen.net/"/>
</ruleset>
Using the custom rules allows you to guarantee HTTPS access for any other domain you wish to.
Played with Python BitString, just for fun. I haven't needed it this far, but if I need it. It'll be nice tool. It's good for situations where there's long list of something, which is
is in contiguous space but there are different boolean states for entires. I just often use dictionary which contains integer ID with Python. But I do know it's not space efficient representation when data
set gets larger. Good part of this that it's able to handle gaps without problems in case IDs aren't in contiguous space. If space is contiguous list with booleans would work too, but it uses 8 times the memory for same representation.
This is old stuff of course, but I reminded my about it. It's still a very nice optimization. Perfect example how things can be made faster, if you just bother to think for a while.
I just noticed that LG servers accept absolutely huge HTTP post data. Several 25 gigabyte files were posted successfully in parallel. So who says that you would only send them the filename or tv channel
information. You can stream them full hd tv-shows and send full bluray rips to them in real time. curl -F "file=@your-show.extension" http://their-http-post-url/ I just sent them a lot of data and they accepted it happily and didn't even ban my IP. Which is nice, no reason even to use multiple IP's for sending. Maybe I'll send more? Or what about if million users would send them a few terabytes or even bit less like hundred of gigabytes. From now on I'm sending them everything I'm watching. So if you got seedbox with free (or near free) bandwidth, just send a few copies to them of everything you're seeding.
I personally currently paying for about 30TB of traffic per month, and I'm only using something like hundreds of gigabytes. So now I got great destination for my excess bandwidth.
This reminds me from time in mid 90's, when I asked for OS/2 from my friend, and he uuencoded whole cd image and sent it to my email box from university servers. My ISP wasn't quite ready to receive about 1 gigabyte of email, which was absolutely huge amount of data back then, they got quite mad. Btw. It was the time when even ISP were using 10 megabit coaxial ethernet networks.
Yet again I kept Internet engineers hand, and told how to configure routing table and interfaces, because "things just won't work". Of course those won't work, because you have invalid configuration. It's not so hard after all. Or it is hard, if you don't have a clue what you're doing, even if you're supposed to.
Decompiled some .class files and inserted my own parasite thread inside the program to do stuff I wanted to do. Worked great. I even added automatic scoring features, spam text messaging and so on. Because it was "flood cast" game, which sent messages to all other peers, it nearly crashed whole game because they didn't have proper rate limits and my flooding of X Mbit/s to server caused it to flood 1000's X the bandwidth out. Duh! I'm sure there were many clients which weren't able to even handle that data flow even if the server wouldn't gotten swamped. Then the queues would simply have eaten the servers resources, if those wouldn't have been properly limited. So the server itself created amplification and resource consumption attack against itself.
Well one of my friends did even better. He reverse engineered whole applications communication protocol, including some protections they had there and wrote own client using C.
It's good question which solution was better, at least mine was simpler to execute, as well as it allowed me to still use full client with just my own tweaks.
Studied covert channels.
Cloud services are natural way to go. Exactly same thing has happened in other sectors much earlier, and it's nothing new. drinking water production, electricity production, banking, etc. Self-producing something is only viable, if scale is large enough or needs are very specific and outside commonly provided services. For this reason some industrial plans which require a lot of water or electric power do have own systems for producing what's needed. But for most, it's good idea to outsource. It's just like running own data center in office closet. You can do it, but it's probably much worse option than out source it to cloud. Of course Facebook or Google could use cloud servers from Amazon, but scale is huge enough, so it doesn't make sense. At least in Finland many companies which are selling server hosting, server management etc, are currently also promoting cloud services from Google and Microsoft, even if it's direct competitor to their own services. Only difference is that Google and Microsoft probably give them economical incentive as well as, services are after this still cheaper for end user than services provided by local service provider.
2FA won't protect against cookie stealing.
Microsoft povided Office 365 is ok'ish. But the sharepoint which came with it, is ridiculously slow.
I love encryption, but I usually use it for backups or only highly private information. Because error in key or loss of keys etc, could render a backups / data totally inaccessible / unusable. But for that reason, I usually take from most very private data a backup copy to another off-site location, which is encrypted using different key & technology. So I'm hopeful that if something bad happens, at least one copy is restorable.
I never trust cloud services with the only copy of the data. Either it's local data, which is backed up to cloud (encrypted) or it's cloud data which is backed back to office (encrypted). Both backups data and connections are of course bot encrypted. And all with different keys. If data is encrypted at the original source, is secondary question, it might be or might not be, depending what it is.
Even my own email server is backed up off-site daily, with encryption, and proper backup history (for months).
One message routing protocol was written so badly that it was only able to handle one remote connectivity point. The project was really complex and basically unmaintainable. When there was need for multiple destinations what did I do? Well, of course I wrote simple message store and forward solution with multiple outputs. Now the original project just forwards the data to my python forwarder, which then pushes it as ordered but asynchronous process to multiple destinations. Everyone is happy and things work well. Related terms, SOAP, WSDL, Python, WebService, routing, router, fork, splitter.
Something different: Russian and Chinese air craft carriers and radar & weapons systems being used. Especially story of Liaoning is interesting.
Broadcom PhyR - DSL Physical Layer Retransmission, Bit Error rate (BER), Reed Solomon (RS), coding gain (CG), forward error correction (FEC), error correcting code (ECC), Impulsive Noise Protection (INP), Repetitive Impulse Noise (REIN), Interleaving, Error Propagation, IPTV.
Chocolatey - APT-GET for Windows? Yes, I've always loved proper package manager.
BoxStarter - Easy Windows environment installations.
Played with SqliteAdmin and SqliteBrowser. I think the browser one was better and clearer for my humbleneeds. Admin one is better if you got more stuff to do.
Authenticated SMTP might be blocked in some cases. Elisa (Finnish ISP) seems to be blocking even authenticated SMTP outside from their network. So operator is basically forcing customers to user their SMTP out relay.
Cryptography is hard. - So you want to crypto?
Big data quality issues ... Just so common, daily stuff, without constant tracking, monitoring, auditting, trail logs, reporting, it's going to be just garbage.
Block, hash, delta, data compression, de-duplication ... Duplicati, duplicity and generic rsync etc related discussions.
Batch, realtime, near realtime, integrations, etc. I thought I would about this... but I guess this post is already long enough.
Just want to remind people about the fact that something what is being triggered asynchronously after something has happened is NOT realtime. It seems that some people are pretty confused about this.
Had interesting discussions with ThinStuff guys. Discussions were related to update automation, security, authentication, notifications, RSS feeds, timing and so on.
Keywords: Customers point of view, user experience, usability. Technical sales support tasks. Key customer responsibility. Customer contacts. Service desk experience. Wide ranged experience scope. Lets get this done attitude. Don't spend days thinking why we can't do it, if there's no real reason. Self guided, I'll study what ever I think I need to know to get the job done.
Sorry if there's something unclear. I just dumped this stuff as fast as I could. Now my backlog contains anymore only 385 entries for 2014. ;) I'll be posting more soon.
Google Sites
Report abuse