Getting a Grip on Governance

As a programme and project manager, particularly in large matrix organisations, one of the things that used to drive me mad was the lack of joined up governance for projects. I know I'm not alone. Whenever I've been involved in organisations improvement initiatives, this is a regular bugbear across delivery.

Project managers, who are often juggling multiple projects, have to undertake a myriad of reviews, complete multiple status reports, and submit to random, often unsolicited, data requests, and run the gamut of Delivery and Quality Assurance audits. In the worst cases I've seen, an individual PMs can lose as many as 5 days a month just providing data and sitting in review meetings, often saying the same things to different groups of people (and sometimes exactly the same people!).

Now, I'm not advocating that anyone should stop performing reviews or monitoring project status. That would be a violation of lots of things I believe in, and probably commercially suicidal. But I do believe that organisations can get a lot smarter in they way they perform their governance, freeing up people's time, reducing the burden on projects, and adding genuine value to the organisation as a whole.

In a many typical organisations the project is answerable to external and internally facing governance. The client or business wants to know how their money is being spent and when they can expect delivery, and the delivery organisation needs to understand how to manage its people, IT assets and other resources. These aren't unreasonable expectations.

What's unreasonable is when the balance shifts from a need for information to help run the business to a culture of interference, where indirect stakeholders start demanding their pound of flesh. Where bean counters get notifications from monitoring systems and demand answers as to why variances are exceeded and expect solutions to be implemented by yesterday.

Quality boards and risk review boards are set-up in addition to the weekly, monthly and quarterly status review boards. Red and Amber projects find they are spending more time explaining their status than being able to fix it.
Ultimately, the project managers spend their days repeating the same things to the same people time and time again. And the majority of the people listening are not actually in a position to do anything with the things they are told, let alone provide help to the beleaguered projects.

The sticks are all sharpened ready for the kill, but there isn't a carrot in sight.

When an organisation has a systemic problem with delivery it seems that the usual way they go about fixing it is to add more and more layers of governance. It's as if talking about it often enough will make the problems go away. A smarter fix might be to review the entire governance process, streamline it and only involve people who can provide assistance and solutions, rather than add to the problem.

Joined up governance means looking at how automated monitoring systems can be used intelligently and incorporated into standard reviews. It means looking at the people who need to be involved in those reviews, what preparation needs to be done (in my experience people just turn up and fire high level standard questions at the PM), and what the expected outcomes are going to be (a completed review checklist is not a satisfactory outcome!).

This is really an ideal role for a pro-active PMO, that acts as a conduit between the relevant stakeholders and the project. Indirect stakeholders (should there still be any need for them) should be able to get information from the PMO rather than the PM. (If the relationship between the PM and the PMO becomes imbalanced, e.g. when the PMO starts to become a self serving force, this must be redressed - see my 2008 article Project Management Office - Master or Servant?).

As part of the change to governance, it's an ideal time to review the measurement systems in place. Projects are often challenged as to why data differs across different systems. The better question is why data is being captured in multiple systems in the first place as it clearly introduces scope for error - see my 2010 post When Measurement Programmes Go Viral.

Project and programme oversight shouldn't be rocket science yet many organisations have governance systems that would not look out of place in mission control. As spring starts to take hold this year maybe it's time to take a good look at your existing governance and see where it can be cleaned up, simplified and ultimately become a valuable tool rather than a weapon to be deployed against project teams doing their best to deliver real value to the business.

Unused Information Holds Many Answers

Twitter can be a wonderful source of inspiration for a blog entry especially for an old pro like myself, who has encountered so many "coachable moments" that sometimes I forget what I want to share. So, thanks to the Standish Group for the inspiration for this entry with the following Tweet:

"44% of CIOs say it takes on average a day or less for their organization to reach a standard IT project decision"

This caught my eye and I responded by rhetorically asking how they measured that - probably a finger in the air. A few days later Standish came back to me with the response that "it was asked in our monthly DARTS survey of over 300 CIOs [which] had many questions on decision latency, complexity, & costs". It wasn't my intention to question how the Standish Group came about their data - rather how CIOs could actually provide a measured response in the first place?

In 28 years of working in the IT industry I have never seen a project, programme or business area maintain quantitative time related data on their decision making processes (except in my own projects!). If that sort of data is not available at the lowest levels of the organisation, I'm struggling to understand how a CIO can honestly answer the question on behalf of the whole business.

Standish have since tweeted lots more amazing stats based on their survey such as "39% of CIOs say it cost on average $500 or less for their organization to reach a standard IT project decision".

But how CIOs or anyone else responds to these questions isn't really the main point of this post. I'm interested in why teams (read departments/groups/functional areas as well as projects/programmes) don't record such data, and if they do, why they don't use it to better understand the way they operate.

Most projects maintain some kind of RAID log - probably using a standard template which came from the CMMI programme or PMO - and go through the regular motions of entering data and reviewing the outstanding items so they can close them. They probably prioritse each item, and assign a degree of severity. They may even put in open and close dates, but they rarely, if ever, do any analysis on the data, other than to monitor the number of open and closed actions over time (which generally tells you very little at all).

As a process management person I view these logs as an valuable source of input, if you're prepared to put in some effort and ask some awkward questions. Why do some issues take weeks or even months to close? Why should it take 10 days to reach a decision? Why don't open issues get reprioritised after a certain amount of time. Are there connections between the types of issue or decisions that cause the most problems?

These are the types of question that should be getting asked at departmental reviews, stakeholder reviews, and quality reviews but generally get ignored in favour of the familiar questions about timescales and budgets. If you ask different questions at these reviews, establish root causes and fix the problems, issues around budgets and timescales will probably start to fade into the background.

I find it bizarre that organisations spend so much time tracking code defects (rather than getting on with the business of fixing them as they arise) but seem to ignore management defects until they have actually caused operational failures.

While managemement continues to highlight time and money as the only critical yardsticks by which performance is measured, quality will always be an afterthought and the entire organisation will suffer as a result.

When Measurement Programmes go Viral...

I think it’s a fair comment to say that only a foolish manager or leader would try and run an organisation, department or even a project based on subjective judgment alone. Gut feelings and intuition should not be ignored but they need to be backed up and supported by facts and often the best facts are quantitative. I have come across plenty of managers who follow the ostrich tendency and genuinely believe that they understand how their organisations perform but reject any type of data collection. Luckily they are in a minority and they aren’t the focus of this post.

If ever you attend a course, seminar or conference presentation on the implementation of a metrics programme in an organisation there are usually several repeating themes:

1. Metrics collection is only worthwhile if you use the numbers to take actions
2. The cost of data collection must not outweigh the value of the data
3. Data quality is paramount – poor quality data is usually worse than useless
4. Start off by collecting the most useful data, based against your objectives, and build your programme on that basis
5. Continuously review the data you collect and get rid of those measures which no longer add value
6. Don’t use the numbers to beat up your people

The Measurement and Analysis process area of CMMI is one of the easiest PAs in the model to understand. In the main, it is written in jargon free English and it follows both a logical and a chronological path. In short, it ain’t rocket science.

Yet in almost every organisation I’ve worked in it brings fear to the heart of the process teams, fills project managers with dread and often has so-called metrics subject matter experts rubbing their hands with glee at the thought of the power they will be able to wield. In some organisations some managers will also delight in the prospect of new ways to command and control their workforce whilst others will react with complete apathy (“we’ve seen it all before and it won’t work”), and still others will share the same fear as their staff.

So why do so many organisations get it so badly wrong?

There are a few places that get a good balance with their measurement programmes, but too many fall into one extreme or the other. Some organisations simply pay lip service to the requirements and do just enough to think they’ll get through an appraisal. But in this piece I want to look at the organisations that go to the opposite extreme and overwhelm the organisation with useless, redundant and time consuming measurement activities – again with the simple objective of meeting the needs of an appraisal rather than focusing on the real needs of the business. These are the organisations where measurement programmes have gone viral.

Imagine the scenario; an organisation has set a goal of achieving CMMI Level 3 within 24 months (I know – it’s a bad goal, but we all know it happens). The SEPG and steering committee agree that they need to ramp up the measurement programme because they don’t have enough going on to achieve Level 3 based on arbitrary perception rather than a genuine business need. Someone is appointed to head up the programme who is a naïve but ambitious PMO manager as opposed to a management or business expert (or even a software engineer).

Within weeks, a deluge of new measures are mandated and the already overextended  project managers now have the burden of collecting and submitting each and every new measure through a system of manual data entry sheets within yet more arbitrary timescales. No explanations are available as to how the data is to be used or how it will benefit the organisation. Of course, none are necessary because collecting data is a “good thing” and a CMMI requirement, and therefore it must follow that more is better. A team is put together to generate a set of internal dashboards which are built using complex excel spreadsheet and macros and a compliance team is set up to monitor the whole activity to ensure that there are no missing values. The spreadsheets are published to senior managers and a series of compulsory review meetings is established. Each and every aspect of each and every project is examined and corrective actions are created to bring deviants back in line. CMMI requirements are therefore addressed and the outcome of the forthcoming appraisal is in the bag. Everyone is happy (except the PMs, but they don’t count).

With this success story behind them, there is only one place to go for the measurement programme – the accumulation of more data, the development of bigger and better dashboards, and the total domination of the PMO across the enterprise. Yup – we’ve gone viral.

Of course the reality is that behind the scenes, the project teams are making up the numbers to comply with the data collection process – putting something in is better than getting in trouble for failing to submit anything at all. Managers largely ignore the data, because they are overwhelmed by it, and don’t really understand what all the numbers mean anyway. Dashboard reviews become repeats of the other management review meetings, and no real or useful analysis can be performed at any organisational level because the data is just one humongous and homogenous blob.

In many cases the organisation will achieve its goal of reaching Level 3 because they’ve done “enough” to get away with it, but everyone knows that it’s a bit of a sham. Emphasis on measurement falls away over the next six months (along with all the other non-institutionalised processes), and the pre-appraisal status quo is restored, until the next appraisal in three years time when the frenzy will start all over again.

So what can you do to inoculate yourself against this behaviour?

1. Look at the M&A Process Area in detail – nowhere does it tell you that a certain number of measures should be in place to be operating at any specific CMMI Level
2. Align a few key measures against your specific business objectives, and focus on getting good quality data. Involve business leaders in the selection of the measures so that they can get nearer to the solutions to the problems that cause them pain
3. Look for measures that will help projects and project teams not hinder them, and don’t overburden projects with demands for duplicate data or data that can be found elsewhere
4. Provide explanations of how to interpret the data. If you can’t do this, you have no right to demand the data in the first place. You also need to remember that different groups of people will have different objectives so do not take a one size fits all approach
5. Perform appropriate analysis and publish findings along with recommendations of collective actions that can/should or must be taken
6. Review your measures on a regular basis and throw out redundant ones or ones that don’t add value to the organisation. Review the cost of data collection at the same time.
7. Get reactions from the “shop floor”. If your metrics programme is causing your people pain, it’s probably doing something wrong and it needs fixing
8. Align the expectations of managers and information providers – in others words perform some serious stakeholder analysis and relationship management
9. Instead of worrying blindly about compliance in providing data, concern yourself with why the data may not be being provided, and instead of analysing missing data, examine outliers and significant deviations from expected values
10. Stop treating CMMI as an objective and focus on doing the right thing for the business, with CMMI as a (one of many if possible) reference model to guide you

Getting Value from the Quality Department (Part 4)

In the last three posts I've been looking at some of the issues facing Quality Teams in an Application Development and Maintenance environment, and some of the ways that these teams and their organisations can get more value from quality related activities.

In this final part I want to consider more problem areas, management and measurement. These two areas are crucially interwoven as we shall see later in the post.

Quality activities in any organisation are always going to be at risk if they take place but no-one takes any notice of them. Unfortunately, in many of the organisations that I've been involved with, quality is all too often considered as a necessary evil, and the exploits of the quality team are left to percolate in the background.

Executives and middle managers, only get interested when something nasty hits the fan. These situations generate knee-jerk reactions such as a review of quality activities (usually too localised), a commitment to prevention rather than cure, or policy word changes (but without enforcement), but these tend to be short lived and ineffective actions which fail to address the real problems in the same way that a sticking plaster cannot fix a ruptured artery.

In almost every case where I have seen little real management commitment to quality, it turns out that managers have no realistic or measurable objectives set around quality. There are often collective objectives like "Maintain ISO 9000 compliance" or "Achieve level 3 of CMMI by quarter 3 next year" but these are fairly meaningless at the best of times. They are also Boolean objectives; "Achieved" or "Not Achieved".

More useful quality related objectives might be "Improve resolution time of quality issues by 20%" or "Participate in 50% of quality incident reviews". Of course, this makes the assumption that quality reviews take place and quality issues are identified, but crucially they put the onus of responsibility onto individual managers and bring them into direct contact with quality activities. Failure to participate will have an impact on their bonus or salary review.

Of course, an organisation that has a quality department almost certainly collects lots of data. The trouble is that this is often all that happens. Data collection is of no value unless the business actually does something with the data, and when I talk of the business in this context, I'm referring to the decision makers, not just the data collection team.

In many cases the data collected is worthless even if anyone wanted to use it because it doesn't actually address any direct business requirements, or because the quality of the data is so poor that it is of no value. Historically, data collection, analysis and data based decision making has been seen as a good thing. Sadly, many organisations collect data that they think they need collect, without understanding what it is to be used for, who it is to be used by, or how it is going to be used. Vast amounts of time are spent providing numbers because the system says you must. Often the same numbers are demanded by different people, often in different formats and at different times.

At one company I worked for we had three time recording systems, one electronic and two paper based (all of which required predicted clock-in and out times as well as actuals). To my knowledge only one of these was actually used to determine anything of any importance (namely overtime pay!), but that was the way things were done.

Regardless of whether they like it or not, executives and managers need good diverse data to make good decisions. There is still an extraordinary number of managers who are either consciously or subconsciously oblivious to this fact. The real problem is that too managers believe that the only data of any importance is financial data, they are measured on their ability to manage P&Ls or to meet their financial targets. What they fail to understand is that financial data alone is useless in getting to the root cause of problems and trying to resolve them. For that, they need other information which can then be used in the context of the financial data to better understand why there are issues and what their causes are.

So why are these two apparently unrelated issues of management and measurement related and what do they have to do with the quality department? In too many organisations I've seen lots of potentially useful quality data wasted because of a lack of imagination both on the part of the quality team and that of management.

Data is presented in drab and meaningless charts which really only try to demonstrate that the quality team is doing stuff. At the same time managers fail to ask the necessary questions to be able to understand how quality data can help them improve their business.

Quality managers need to initiate the dialogue with management and coach them into understanding how they can make data work for them. Think of different ways to present the data, and think of useful things to say about it. Quality data taken out of context is meaningless. For example present audit or defect data alongside financial data to highlight potential correlations.

Encourage managers to ask the difficult questions about your improvement or quality programmes, and be prepared to lift yourselves out of the status quo. Only then will management begin to sit up and take notice.