Software Architects Need More Respect: Hacker Culture Is Overdone

Some people would say that I am not qualified to write about software development as I have never written a line of code in my life. But I have spent over 30 years working with some great developers on a wide range of projects and I have seen what works and what does not work.

A few years ago the art of architecting systems for scalability and extensibility started to get a bad press. I think Joel Spolsky was the first to coin the phrase “architecture astronauts“, to describe those who were always designing layers of abstraction but forgetting to deliver code that actually did anything.    Then we got the lean start up movement, focusing on the constant iteration that the web enables.

This led to the hacker culture, oriented to “just do it” speed. “Hacker” used to be a term of abuse, but like “punk” it was adopted by those accused as a badge of pride.

That is all good stuff, but methinks it has gone too far. We now see too many systems where the original system has to be totally thrown away. That may be OK if you raised $ gazillions and can now compete with Google and Facebook for a big team of really great engineers. But for most startups, it is not possible. So most startups limp along with the original system. The upshot is:

1. Hosting bills that really start to bite into cash flow.
2. Systems that crash too often; a few of these and your users have clicked away to a rival. Yes, we persevered through Twitter fail whales, but is your service really that addictive?
3. New features take way too long. You either release fast or put in some QA and testing process to avoid brand-killing crashes.

It does not need to be like that. The best developers can do both. They create a scalable and extensible architecture AND deliver quick feature iterations. Sometimes you cannot get one person who can do both – it is rare. One team I worked with solved this by having a 4 person team where:

1. 1 person hacked demos, in the full knowledge that the code would thrown away. He was awesome at this and in full synch with his colleague who:
2. created the architecture AND the working code.   35 years later, the system is still being used.
3. 2 people who did the ancilliary stuff like testing, documentation and so on. Great surgeons have a team, same is true with great engineers.

I think the problem may be with the pure architect model. No engineer likes being told “here is a perfect architecture, just code it as per that spec”.  Going back to the house analogy, we need more architectural engineers who create designs that they know will work and they prove that they work by writing the code.