posted Jun 19, 2012, 10:04 AM by Sami Lehtinen
updated Jul 23, 2012, 5:30 AM
Yet another rainy day, so here I am writing some light stuff.
Web cache & mirroring
One day I watched one poor guy downloading same files over and over again from slow source to same computer. I got really upset about that, how he can waste valuable time doing something so inherently inefficiently? After a while I couldn't watch it anymore, I just had to do something.
I install caching web proxy, which makes downloading files from internet much faster. After that I found out that they were downloading some files over and over again from Microsoft Windows (SMB) network share, which was also behind slow WAN link. That I solved using robocopy and scheduled mirroring. Now we got all resources we need available from one gigabit network. No need to download any files from agonizingly slow web sites. I just wonder why people didn't do anything about this earlier? If you would download files from slow site over and over again, would you bear it? I wouldn't and I didn't.
SMTPS & ISP helpdesk
My mobile checks email using IMAPS and sends email using SMTPS. That's fine. But here's story how ISP failed and failed completely to solve the situation efficiently. Here's list of things I assume they did and how they failed to understand the problem and what the original configuration was, they also failed to inform customer properly and also gave misleading tips how to fix the situation.
- Everything was working as usual, my email gets delivered using SMTPS.
- Some emails start bounce due ISP smtp server being black listed due spamming (SORBS).
- I contact helpdesk about this issue, they don't really get it and tell that I should send email again later. (duh!)
- SMTPS server becomes unavailable.
- I contact helpdesk again. Now they tell that I should use smtp.isp.fi server not smtps.isp.fi. They really do not understand at all what's the difference between smtps and smtp. I try to tell them that I want to use SMTPS. They leave ticket open to helpdesk.
- SMTPS server becomes available again, but now it requires authentication. I call them again, and tell that I'm mobile user, I don't use your email services for anything else than sending mail. I should get SMTPS account information. Yet again they tell that I should use smtp.isp.fi which doesn't use SSL and doesn't require authentication.
- I check details for smtps.isp.fi and smtp.isp.fi guess what, both servers have same IP address. It was fact that the name they instructed me to use was only CNAME to the primary server. But when I connect SMTPS server, it's SSL certificate is for SMTPS.isp.fi. So that they tell me to use smtp.isp.fi instead of smtps.isp.fi with SSL clearly reveals that they don't have any clue what I'm talking about. They didn't at any point say that I shouldn't use SMTPS protocol.
- I start to use smtps.isp.fi without SSL and it works just fine. This proves that dudes at helpdesk are utterly incompetent. It wasn't about server name at all, it was about protocol.
- After a few days ISP fixes SSL service and it doesn't require authentication anymore. Now if I would use smtp.isp.fi with SSL I would get certificate warning every time when I'm sending email.
Ok, that's what I experienced. What went wrong?
- ISPs helpdesk staff are n00bs, they don't understand their systems at all. But this is clear from discussion above.
But what caused all this? I don't know this, because as usual ISP just said that "issue is fixed" and they didn't tell how badly they failed. But with my IT experience I'm pretty sure that the story went in following manner.
- Somebody who didn't understand how the systems were configured configured their firewall. Possible reason for this, utter lack of competence, hurry or some important customer calling CEO or some other important guy and telling that their IT staff should fix the problem immediately. I assume someone was upset about not being able to send email when being outside ISPs network.
- What did they do? For some reason they opened smtps port for whole internet. Originally their service was configured so that anyone in their network could use smtp or smtps without authentication. Now because somebody opened the port for whole world, it didn't take too long until it was used as spam relay.
- Then they closed the port, fail.
- Then they started to required authentication, fail.
- Their helpdesk gave absolutely wrong information and showed that they don't know whats happening in their organization or how their systems work. FAIL!
- Finally after someone more experienced and who knew how the system was supposed to work fixed the situation to original state.
Yeah, this is pretty much how IT things get messed and customers get wrong information and nobody's happy about it. Very simple failure, due incompetence or hurry and that's all it took.