The Trouble(s) with Software Process Improvement

Fifteen or more years of software process improvement efforts have not lead to the remarkable changes that people like Watts Humphrey may have envisaged when he wrote "Managing the Software Process". The reality is that despite our efforts software development projects continue to fail either completely or to meet their intended budget, time and quality objectives. Even high maturity organisations deliver poor quality software, and the return on investment from quality and other process improvement activities remains low. That’s not to say that SPI has failed and we should give up on it, but if we continue to perform the same actions and fall into the same traps we can expect the same relatively poor results. Even companies that were early adopters of CMM and CMMI and have reached the highest levels of maturity now find themselves struggling to retain their status and suffering the mediocre results of more naïve organisations. Some of the reasons for failure are:

• Insufficient management sponsorship, ownership and responsibility - SPI is viewed as a technical activity and management directives are delegated to groups without sufficient authority to “Just go and do it”
• Software Process Groups lack the discipline and management skills required to undertake improvement projects, with the apparent effect of causing a “Do as I say, not as I do” environment
• Expectations of return are grossly over-exaggerated in an attempt to secure funding, and funding is removed before any real benefits are realised
• Supplier management is poor, and outsourcing suppliers often mismatched against the organisation’s values and beliefs, especially with respect to quality
• Focus on CMMI, maturity levels and specific goals rather than doing what is best and right for an organisation using diverse methods, tools and techniques
• An unmanaged approach to change leading to employee resistance at all levels of the organisation
• Failure to adopt a holistic approach across the organisation, with associated confusion, duplication of effort, and critical activities falling between the cracks
• Decision makers are informed about SPI activities rather than consulted and involved
• Waves of initiatives roll on without pausing to understand the effects of the previous initiatives and often undoing the good that has gone before
• Improvement plans are based on perception rather than fact - data is not used to verify that the right areas are being targeted
• Quality and process teams undermining their own efforts by failing to add genuine value to the organisation and its business

If you pick up any book on process improvement you will see that these and other issues like them have been identified and understood for many years, yet some or all of them still abound in any organisation currently running SPI initiatives. Given that those same books often provide the solutions on how to avoid the problems, I can only assume that there must be something more fundamental going on which is causing the software industry to fail to rise to the challenge, namely the people problems I described in my last post on the 7 Deadly Sins of Process Improvement (or Change Management). Next time I'll describe the first of these - Arrogance. Print this post