I asked my son for his view. He has a first class degree in Computing but decided that he would set-up a brewery (quite good – won the IndyMan Thirst Games) but his answer was “Don’t know”.
Man (I include women) has been brewing for hundreds of years but only producing software for 70 years. Brewing is a pretty well-defined process but the process is flexible and the resultant beer can be modified by changing the ingredients and modifying the process – everything from “Alcohol free” to 12% alcohol – from light to treacle dark.
Brewers have recipes and in some countries laws govern what can be put into beer but the essential message defining the brewing process is “keep it simple, stupid” (KISS).
Much as we might like to, keeping software development simple in not always possible.
The first attempts to define the maturity of the software development process were made by the Software Engineering Institute at Carnegie-Mellon who defined the Capability Maturity Model (1990). This categorises software development process into 5 Levels ranging from “chaotic” (Level 1) to repeatable/optimising process (Level 5). It is a painful, document heavy process – I visited IBM Houston (Space Shuttle – Level 5) in the early 1990’s and my main take-away was the sheer volume of paper. Some types of software (mission/safety/security-critical) need to be developed with this degree of rigour.
CASE (Computer Aided Software Engineering) grew up alongside CMM and were mainly used on very large programmes but only simple CASE tools survive (UML, TOGAF). In parallel with CASE, structured approaches to managing projects emerged – PRINCE2 in the UK, PMP in the US – and you can get certified in them!
Agile is a KISS approach to developing software. Agile pre-dates the Agile Manifesto (2001) – it goes way back and was how software was developed on Lisp Machines in the 1980s. DSDM was also a precursor, based on Barry Boehm’s spiral model. The iterative / spiral model has always co-existed with the Waterfall approach (Specify, Build, Test, Deploy) but whilst in the 1990/200s 90%+ of software development was Waterfall – now nearly 80% of software is being produced using Agile.
If this were true it would be a huge turnaround. The reality is that many organisations, having based their development structures around Waterfall (e.g. PRINCE2) are really struggling to adopt Agile. Almost by definition Agile cannot be more than Level 1 or 2 on the CMMI scale – it is essentially chaotic and that chaos is enhanced by the emergence of DevOps and architectural approaches such as Microservices. With the software production landscape in constant flux it is impossible to evolve to a CMM Level 4 or 5 without freezing the software production process.
Just as a good brewer should be able to brew consistently and yet be able to develop new beers to accommodate changing tastes in the market (and create competitive advantage), software development operations need to have a degree of flexibility.
The dilemma CIO/CTOs have is that if you want to have a good software development operation (one that consistently delivers good quality on time, in budget) then you have to balance growing new skills with retaining the best people. If you don’t invest in your best people they will leave, taking their skills and experience of your business with them. New languages, techniques need to introduced carefully. Some are easy – for example, lots of companies have Java skills and migrating these to Scala should be relatively simple. A good software development organisation delivers competitive advantage and cannot be allowed to atrophy.
A worrying trend is the attempt to use Agile on all projects; it is becoming dogma – “Agile good Waterfall bad“. This is wrong-headed. In addition to the mission/safety/security critical ones, many applications utilise complex calculations that have been defined and tested over many years and need to be re-implemented (+exhaustively tested) very carefully. This is where the skill of chunking comes in, the wisdom and ability to determine what to develop and how.
In most organisations Waterfall and Agile need to coexist. Waterfall approaches “fit” with the way that many organisations have to work e.g. conforming to Sarbanes-Oxley and with the command and control management style still prevalent in many (most) organisations. Agile can be married with Waterfall (PRINCE2) but few PRINCE2 practitioners understand how. Scaling Agile is not difficult but trying to do so many organisations are reaching for unnecessarily complex solutions rather than using a pragmatic scaling approach.
One difference between software development and beer is that once beer is drunk, its gone but software has to be maintained. Of course, one common complaint from DevOps and support teams is that that the coders treat software like beer – “not my problem any more”. This has to change, and with it development and support processes adopted by many organisations (goodbye ITIL). Fortunately new approaches to analysing codebases are emerging – see the work Adam Tornhill (Empear.com) has been doing – that can profoundly change the applications maintenance landscape.
Similarly the way that Programme Management Offices (PMOs) work needs to change to reflect this state of flux. Old thinking has to be discarded. The fundamental problem is that most PMOs are not staffed with people who understand software development. This is one reason why projects fail – few PMOs are effective. Would a company let its finance function be run by someone who was not a qualified accountant?
Qualifications do not fare well in a state of flux. Accountancy has not changed a lot over the past 100 years – producing software has. Many qualifications are hardly worth the paper they are written on. Anyone who believes that PRINCE2 certificate qualifies a person to manage a project is deluded. Some approaches are useful – Failure Demand, KANBAN – but many are outdated and constrain thinking. They can be used to frame thinking – no more.
The huge scope and complexity of producing software dependent systems means that a massive body of knowledge (BOK) is required by CIOs, Programme Directors, Chief Architects etc. They cannot be experts in everything but need to keep up-to-date.
A. Producing software is messy – just like brewing beer.
By “software production” I really mean systems that are dependent on software and that includes architecting, specifying, user experience, coding, testing, deploying, maintaining, and managing the whole process. I use the term interchangeably with software development.
You possibly have a favourite beer from one of the big breweries. The “craft beer” revolution that is sweeping both the UK and US enables us beer fans to expand our taste horizons. Craft beers are more expensive than traditional beers because they use more, and more expensive hops. If you haven’t already, try them. https://twitter.com/fivecloudbrewco
Shibui – from Tom’s time in Japan and appreciation of Japanese culture.
Yes, I know Ada Alice Lovelace “founded” programming. More women should code.