Rock bottom

"I ask you, where else can you catch a behind-the-scenes glance of some very awesome people?"

Awesome" might be a bit of a stretch in the case of "Rockstartup.com This website (from where I got the quote above) claims to be a reality TV show about the infamous web startup PayPerPost.com. I can only guess that the purpose of Rockstartup is to display PPP as a hip, relentlessly honest and real hands-on startup.

But watching these two episodes only conveyed to me that PPP has to be a bunch of clueless dolts:

[youtube=[www.youtube.com/watch](http://www.youtube.com/watch?v=OwrkUKmVZe0])

The IT guys are in real stress. But that doesn't surprise me, seeing how they determine their project deadlines. This time, the change has to be finished for the board meeting. If they hold their board meetings periodically, this probably isn't a very realistic way of setting milestones.

I sympathize with how they crap their pants deploying live. It seems that there wasn't time for testing at all. In their place, I would get the heebeejeebees, too. But in their place, I would also refuse responsibility completely. I may handle my personal web server that way. But if you have a serious business, deploying untested versions is pure negligence. And they get what they deserve -- the site goes down.

It may be the way the video clip is cut, but it seems like the team lead is reporting immediately to his boss (which is the CEO of PPP) that "a network circuit blew." I'll have to watch it again to look if there's a BOFH excuse calendar somewhere...

And it gets worse:

[youtube=[www.youtube.com/watch](http://www.youtube.com/watch?v=ELea2bbL_2g])

All right, PPP has guys running around claiming the title "Code Ninja" who test a secret new feature on their public blog because that's the only one they have. And they don't seem to mind very much. Let it put me another way: they don't seem to have much of a mind.

Another brilliant idea demonstrated is executing database queries on the production database. Whoops, forgot the WHERE clause. Database FUBAR. Well, there's always the backup. Oh, there isn't? Hooray, site down again.

This time, by the way, it's a demonstration for investors that's putting everyone under severe pressure.

I wonder if these people don't know better or if they just are denied the necessary resources, mainly time and budget, to do their jobs right. Working in the IT department of one of Germany's biggest web companies, I know how doing the job right looks like:

• When determining project deadlines, you make sure everyone has the same understanding of issues and consequences. That shouldn't be difficult when the CEO resides just a few tables away.
• You employ only people that know what secrecy means and are able to read the corresponding passages in their contract.
• You don't deploy software into the production system without having it tested on a staging system. For that purpose, we've built a huge VMware farm that resembles the production environment as closely as possible.
• You have people who are responsible for operating the production system and who are the only ones having the necessary access rights. Those are different people than the developers.
• You don't do manual queries on production databases that haven't been approved by the chief DBA. If you do, you don't do them at times when they can severely disrupt the service.
• You have standby databases in the case the main one is hosed. Since unintended content changes will get replicated, you also have a backup. One that's as fresh as possible and that has been proven to be recoverable.

And as a manager, I don't think humiliating your staff by making them wear a ridiculous hat if they make mistakes betters the situation. To err is human. To make people afraid of errors means adding just another source of mistakes. Your job as a leader isn't that simple. You have to determine the causes of the mistakes your people make.

• If it's lack of knowledge, train them.
• If it's pressure, improve their working conditions.
• If it's lack of resources, get them what they need.
• And if it's negligence, hold a private feedback talk to make them understand that diligence is crucial for your operation. If they keep on being stupid, don't fool around with hats. Fire them.

I really wonder if "Rockstartup" means that this company is going to sink like a rock. At least, that's the impression I got from those videos. I really put a lot of time into deciding what I write in my blog about my work and what I don't. And hey, of course there would be many juicy bits to report every week. But if I published such proof of incompetence as these clips, I'm afraid I would not only get fired but would disappear under dubious circumstances...

(via Nik)