Defensive Design: Why Startups Fail

The idea. The passion. The hard work. The sleepless nights. The product. The goal. They all define the startup. Every one aiming to conquer and change the world. We are 50% complete in 10% of time. Then 80% in 50% of time. Then 90%. And 95%. Problems start mounting. A year ago we thought we were ready. So we did 6 months ago. And today? A month away?

I've met several stories like this one. Full of optimism. Underestimating the task. They all have one thing in common. Their development plan was built to be offensive. While the seasoned grown ups plan defensive.

A good IT company should spend about 70% of development costs on Quality Assurance - QA. I have never seen such budget allocations in startups. There are several reasons for this. Firstly, when they compete for funding, they want to look attractive. Somehow a lot of greedy investors never learn a good product has to be expensive to build. Secondly, the inventors and entrepreneurs are often inexperienced and only the experience makes you realize you have to plan a lot of money for QA. Not only for pure R&D and marketing. Some of them are even lucky to succeed with low QA budgets. But with such luck they'd better go to Las Vegas, for better return anyway.

QA means a lot of testing after the product is built. But it starts much earlier, with my favorite - a defensive design. I learned my lesson long time ago and the hard way. Back in the early 90's I realized software simply has to be defensive by design. Every function, method, every line of code I always try to imagine: what can go wrong. A database write can fail. A communications link may be noisy and broken. Power can go down any time. Interference happens unexpectedly. What has often helped me was the awareness of how things really worked under the hood. Spinning disks, electromagnetic waves, power spikes, wearing flash storage. Not every software developer is aware of them. But at least there should be some in the team, ideally those who do physical design.

Unfortunately I still come across the neglect of defensive thinking and defensive design:
- "we've built it and it works!"
- "are you sure it will work in all circumstances?"
- "why not? we've run tests!"

Such tests usually mean nothing. Defensive design should start with... well... the defensive design. Do we check for corrupt data on reads? Do we use strong CRCs, not just weak XORs? Are the algorithms mathematically proven to be correct? How do we deal with rounding errors and overflows? Has the design been reviewed by somebody? And: how thoroughly do we analyze corrupted data and isolated occurrences of unexpected behavior?

The learning curve of the civilization as a whole is impressive. But what surprises me the most, there are still individuals who (despite of the Internet) do not draw from experience of the others, often inventing new crappy wheels on their own.

If an inventor, designer or entrepreneur dreams of a real success, he has to be serious about every bit or atom layer he puts to create the product. When the structure is finished, it is often too late and expensive to start fixing the foundations or internals.