Agile failed for Universal Credit, and perhaps is not right for government (opinion)

Computer Weekly asks why agile development failed for Universal Credit and gives a fairly powerful answer.

Its a shame that the article is so long, because the summing up points are not that complex and ones that people in IT and Government should be able to see.

Agile is a method that came out of academic, R&D and some corporate IT.  At the core of Agile is out modern age's embracing of uncertainty and change.  In an Agile project you might decide to build A for $20,000 but end up getting B.  In the past this was called mission creep, but as time evolves we are more and more likely to see it as a good thing, as Agile.

I think the first thing I learned in software as a business is that one person's bug is another persons feature.  Much of what we do in software is setting up value systems and criteria by which we are going to judge software.

Another thing I learned the hard way is that these criteria need to be appropriate for the group which is going to use the IT.

Its seems obvious that the new government that came to power in 2010 in the UK wanted to, as so many new governments do, adopt some new cost saving concepts from the business sector.  For IT the buzz for some time has been Agile as a way to save money and speed up development time.

Agile is hard to precisely define but overall its a reaction against classic Waterfall, where a set of requirements create a design lead to a build, go through testing and go live.  This fix cost, fixed outcome model has been standard for a long time and has become hated by just about everyone in IT and business.

As cheaper computers and then virtual machines came along the sector embraced more agility.  The core idea was that rather than Waterfalls fixed concept of fixed features, fixed time frames and fixed cost you could do a better job allowing one to vary.  So if you have fixed casts you can decide to get a fixed set of features but allow the project time frames to change, or you can fix time frames but review features over time.

It seemed like just a confession of reality, Waterfall projects are infamous for over running in either time or cost, or not delivering everything promised.

The problem for DWP, and any big government agency is that the role of government is not agility.  I would go beyond most critics in saying government does not get Agile,  and I would say that the role of government means they should not get Agile.

We live in a Capitalist society of constant change, our culture is reworking itself at amazing speed and with this constant innovation culture we have created great wealth but also great risks.  The financial collapse of 2008 can probably be viewed as an inability of Capitalism to self regulate the risks innovation in Financial institutions created.

So the role of government is precisely to not innovate.  Government is the gate keeper, society needs a stable fixed point in a sea of innovation.  Its the very fact that Government culture is so static, so unwilling to change and so political that gives it so essential value to the larger society.  We live in a world of innovation to the point of self destruction, and when we create chaos we need the rigid institutions of government to maintain stability.

So the deal is this, private sector can provide innovative experience and cost saving to government and it should, but IT service providers need to acknowledge what government is, and politicians in opposition hoping to take power have to understand that radical reforms in the way government work have a almost 100% failure rate.

The more we embrace change the more we need to count on institutions of stability.  Its the very rigid culture of government that makes it valuable to our chaotic economic system.