Agile’s Business Failure

There has been a robust discussion around some of my other blogs on the value of Agile as a software development methodology.

Accepting for a moment the potential improvements in software delivery promised by Agile are real or at the least Agile has the capability to substantially improve the success rates of IT projects (which should not be too hard given the latest Gartner report) one has to ask why its acceptance has been so low? I suggest the answer is the lack of effective communication between Agilists and Executive Management.

As significant number of Agile evangelists believe that all that is needed to make IT successful is to allow the software developers to develop code! Vasco Duarte’s blog Software Development Today is typical – totally focused on the beauty of developing code. And done well, Agile methodologies can produce very effective code. The trouble is the developers don’t run the organisation.

Executive management run the organisation on behalf of the owners. These managers develop strategies for the organisation to follow and create projects and programs to achieve the new capabilities needed to implement the strategy. Software changes or developments are simply one element of the changes needed to enable the strategy and create new value for the organisation.

No matter how badly executed these processes are, the only ethical role for any project management team is to support the strategic initiative of the executive to assist in the creation of value. This was the subject of my post ‘Value is in the eye of the stakeholder’.

Within this framework, Agile Advocates (AAs) need to understand how to communicate the advantages of the development methodology bearing in mind most executive managers see IT as a major liability that routinely fails to deliver as promised (don’t get upset, read the Gartner reports). Management’s natural reaction to this perceived failure of IT has been to enforce ever tighter controls. Of course this has not worked, project controls don’t actually control anything (ref: The Paradox of Project Control in a Matrix Organisation and Project Controls Don’t).

However, viewed from an executive management perspective, the Agile approach is counterintuitive: less controls and greater flexibility to produce better outcomes??!  Achieving the level of trust needed to allow Agile to deliver its benefits requires a structured campaign by the AAs to first build solid relationships with senior executives and then communicate enough information to develop the trust needed to allow Agile to deliver its benefits. Advising upwards is a skill that most technicians don’t have but is a core stakeholder management skill (see: From Commander to Sponsor: Managing Upwards in the Project Environment) and is critical to the success of Agile in business.

There is also a need to discipline the Agile development process. Far too many IT techo’s see Agile as an excuse for not documenting code, ignoring scope requirements, and having fun writing stuff. To succeed in business, the flexibility advocated in the various Agile methodologies has to be tempered with pragmatism. The delivered software needs to meet the business objectives set by senior management as well as the desires of the SME ‘customers’ working with the team. The key stakeholders for any IT project are the people at the top of the organisational tree. The problem is these key stakeholders are rarely involved in the project. Until AAs can develop a language and a practice that offers some certainty to senior management that their way of developing software will deliver enhanced business benefits and maintainable code the potential benefits will be lost.

The fundamental principles in the Agile Manifesto have support in a number of emerging fields of academic research including Complexity Theory and Social Networks Theory. The challenge for AAs is to communicate these ideas in business language and to temper the exuberance of many ‘Agilists’ who naively believe that simply because you have developed a better mouse trap everyone will buy it. The world of business does not work that way.

To succeed, AAs is to avoid over claiming the benefits of Agile and work the organisational systems to allow Agile to deliver its benefits. This includes demonstrating to management the project is aligned to the business objectives, that only the minimum scope needed to achieve the planned benefits is being developed and any new ideas (ie, scope creep) will be properly assessed before inclusion in the project. Achieving this involves the appropriate use of traditional project management processes linked to effective ‘upwards’ communication.  To keep Agile, agile, the application of these processes certainly needs modification and adaptation from the more heavy handed approaches of the past (eg, Waterfall and PRINCE2) but they cannot be ignored.  The PMBOK® Guide certainly advocates/requires PMs to adapt its processes to the needs of the project – I suggested a few of the key adaptations needed in The Gentle Art of Managing Agile.

Now its up to the Agile community to make Agile relevant to the businesses it is designed to serve. To read more visit my posts on the PMI Voices on Project Management blog, and join in the discussion.