Why building software is so hard

I decided to use the same title for this post as an article I read the other day in The Age’s ‘My Small Business’ section. I just thought it was very appropriate.

I have spent over 10 years working as a Product Manager. A Product Manager essentially builds products, whether they are software, cars, boxes, food, banking services etc. During my time as a Product Manager I had the pleasure of spending more than 5 years in the technology space building software and online
services.

The article mentioned above, talks about a lot of reasons why big businesses can’t get software right prior to its release. In my experience, it comes down to a couple of key factors:
•    The nature of the beast; and
•    Management not deciding what they want.

Nature
Fixes, patches and new releases are standard when it comes to software, and customers have been conditioned to expect this. So even when a new product is released, development teams have already spent months working on the next upgrade or patch (in most cases it’s left over functionality that couldn’t fit into the original release or fixes that weren’t quite ready). Like it or not, this is the nature of software and comparing it to cars or other products that are manufactured and sold isn’t comparing apples with apples, because all products or services have their own nature and just because you can successfully apply certain standards to one product or industry doesn’t necessarily mean you can apply them to another and get the same results (not immediately anyway).

You can call it conditioning, but I believe it’s probably more to do with the maturity of the software industry as a whole. If you could send out patches electronically for faults found in new vehicles without having to recall them all, then I believe even the standards adhered to by vehicle manufacturers would also drop.

Management
The biggest and most critical issue is management not deciding exactly what is in a product’s release and what is not. A product will always begin with a concept. It could a new one, born out of another product, born out of a client’s complaint, whatever, it starts with an idea. From the concept we develop ideas on functionality, i.e. it will do this and this and produce this and that. Concepts and functionality are the responsibility of the business. The next stage is Technical requirements (essentially the rules to be followed by the technical folk). This will cover technical architecture, databases, components and other technical things. It’s the roadmap for the developers to be able to code.

From here the project is costed, a business case built and budget (resources) allocated. May not be this formal, but essentially this is what happens.

What tends to happen is that the original concept starts as one shape (square) on day one and then forms and reforms throughout the development/project lifecycle as the software is gradually made (and turns out to be a Octagon).

There are many reasons why this happens. As a Product Manager your own ideas and thoughts on a product evolve and you start to fiddle with the product and its requirements. Secondly, management starts to buy into the concept and everyone wants to be a cook. Keeping control over what the product eventually looks like is very hard work. How do you tell the Managing Director of the company you work for that his idea (good or bad) will have to wait 6-12 months, even he insists it will sell 000’s more licenses.

Then on top of that, developers start to voice opinions on the product and how it should look and work. At one of my previous jobs I had the Development Manager competing with me on ideas and putting them forward to management directly. Most development managers are great at telling you why you can’t do something or how long it will take, this guy was the complete opposite.

And I haven’t mentioned the sales team. As a Product Manger you sit between Sales and Development, receiving requests and trying to fulfil them. Each I product I have built, I have insisted on keeping the sales team well informed (good in theory but not always in practice). I demo the product to them even in its pre-beta phases so they can touch and feel it and have a better understanding for what it is they will be selling. What then tends to happen is that they will visit new or potential clients and tell them about what is coming up. This turns the ideas factory back on again and they return to you with 10 new pieces of functionality that you must build or risk losing clients. The battle is never ending.

The most important discipline in all of this should be budget. When you are given 10 development resources for 3 months, then you need to make sure that you can build everything you need and test it thoroughly within the 3 months. Simple. It starts this way and then delays occur, some issues take longer to resolve and some development estimates are found to be inaccurate. Once these issues begin to creep in, delays occur and testing gets compromised so that deadlines can be met. The release gets rushed and you begin planning for release 1.1 with patches x, y and z.

The article also compares small development teams to large ones and that claims that having more resources doesn’t necessarily result in a better product. I guess this is right, the more people you have to manage, the harder it gets. At the end of the day, however, the key in all of this is prioritising and not bloating the software with functionality that is nice but can’t be built and testing in the time and with the resources available.

To conclude, it’s a lack of management discipline that results in poor software being released. It’s over bloated requirements that never materialise into sales and could/should have been left out.

I think if Microsoft had more real competition, its products would be better, they would have to be. As long as they are dominating, the pressure on them for quality isn’t there, they are far too interested in functionality (that sometimes works and sometimes doesn’t). With the gradual growth of cloud computing and the ability to provide access to software whilst you are working online, more and more providers will find an opportunity to sell their wares and the environment will be more competitive and hopefully the quality will also improve.