reesesWeb

Because of the insane volume of email I receive, and because of my incredibly baroque Rube Goldberg-inspired email handling system (wc -l .procmailrc returns 458, and that doesn't include the armload of perl, ruby, python, shell scripts and c programs that it calls when determining where to put my mail), and because of the volume of spam passing through the system, I was overwhelming the previous astrogoth.com. This was a box owned by an associate, for whom my small monthly fee was not covering the increasing cost of my impact. It was time to move on.

I went with Rackshack/EV1Servers because that's what every one of my friends who doesn't colo at home uses. They claim that if you needed support, you were in for pain, but everything else was great. I signed up on Friday afternoon, and within a couple hours, I had a login to a box.

Let me walk through what happened, with a little timeline.

Friday/12pm] Tried to sign up for a server online. Every Amex I gave it was unable to auth. Filled out "offline signup" form.

Friday/2pm] Received "Welcome" email from EV1 with my admin/root passwords IN THE CLEAR. Was able to log in.

Friday/2:30pm] Log into box, discover that the wrong server type was provisioned. I had requested RedHat Enterprise, got Ensim.

Friday/3pm] Decide to suck it up because the box was so insanely over spec that I didn't want to lose it.

Friday/3pm-9pm] Configuring software. Running up2date, moving web and mail over, setting up exim, etc.

Friday/9pm-11pm] Take a break.

Friday/11pm] Come back to find that my server had been hacked. Someone was creating a bunch of accounts, setting up a bunch of virtual hosts using Ensim, and not even covering their tracks. The guy was fast, but I assumed he was a spammer, and only concerned about relaying a huge quantity of spam.

Friday/11pm-1am] Fought with hacker, deleting his accounts, kicking him off, and finally changing root/admin passwords and rebooting the server. Went through and turned off all services.

Saturday/1am] Filed trouble ticket asking EV1 to re-image the server correctly, as I assumed Ensim had some exploit that hadn't been fixed.

Saturday/3-5am] Got the server back with no problems. Installed exim, mail stuff, tomcat, migrate over my wiki. Did not see any evidence of a hacker making his way back in.

Saturday/5am] Bedtime.

Saturday/11am] Wakey, Wakey. Go to log into box to check that everything is running. I cannot log in as reeses. I cannot log in as admin. I cannot log in as root on the console.

Saturday/11:01am] HACKED AGAIN????????

Saturday/11:10am-1pm] Filed another trouble ticket, went for lunch, annoyed at evil hackers, thinking about cancelling my account.

Saturday/1pm] Epiphany - I wondered if they had assigned two accounts to the same system. That would explain why the person was so easily able to penetrate my server -- the EV1 administration staff was helping them. In fact, each of us probably thought we were being attacked by super-hackers who could evade any defenses, when in fact, we were both idiots helped by a third, greater, idiot.

Saturday/1pm-8pm] Wrestled with EV1 support who kept telling me "they're resetting your passwords RSN, and you'll be able to log in before long," over and over again.

Saturday/8pm-5am] Finally got access to a refreshed RedHat Enterprise box. Installed Tomcat, Exim, Wiki, the JK2 Apache/Tomcat connector (which is a royal pain in the ass itself), and spent a few hours configuring said connector.

Sunday/5am] Bedtime.

All told, I probably wasted about 30 hours this weekend getting everything up and running. Not that I was doing anything important otherwise, and honestly, this gave me the opportunity to freshen up my Linux chops by doing the same boring things a few times in rapid succession, but jeebus, I would have prefered to spend the time reading.

Bookmark/Search this post with:

delicious | digg | reddit | technorati