Agile’s Business Failure

There has been a robust discussion around some of my other blogs on the value of Agile as a software development methodology.

Accepting for a moment the potential improvements in software delivery promised by Agile are real or at the least Agile has the capability to substantially improve the success rates of IT projects (which should not be too hard given the latest Gartner report) one has to ask why its acceptance has been so low? I suggest the answer is the lack of effective communication between Agilists and Executive Management.

As significant number of Agile evangelists believe that all that is needed to make IT successful is to allow the software developers to develop code! Vasco Duarte’s blog Software Development Today is typical – totally focused on the beauty of developing code. And done well, Agile methodologies can produce very effective code. The trouble is the developers don’t run the organisation.

Executive management run the organisation on behalf of the owners. These managers develop strategies for the organisation to follow and create projects and programs to achieve the new capabilities needed to implement the strategy. Software changes or developments are simply one element of the changes needed to enable the strategy and create new value for the organisation.

No matter how badly executed these processes are, the only ethical role for any project management team is to support the strategic initiative of the executive to assist in the creation of value. This was the subject of my post ‘Value is in the eye of the stakeholder’.

Within this framework, Agile Advocates (AAs) need to understand how to communicate the advantages of the development methodology bearing in mind most executive managers see IT as a major liability that routinely fails to deliver as promised (don’t get upset, read the Gartner reports). Management’s natural reaction to this perceived failure of IT has been to enforce ever tighter controls. Of course this has not worked, project controls don’t actually control anything (ref: The Paradox of Project Control in a Matrix Organisation and Project Controls Don’t).

However, viewed from an executive management perspective, the Agile approach is counterintuitive: less controls and greater flexibility to produce better outcomes??!  Achieving the level of trust needed to allow Agile to deliver its benefits requires a structured campaign by the AAs to first build solid relationships with senior executives and then communicate enough information to develop the trust needed to allow Agile to deliver its benefits. Advising upwards is a skill that most technicians don’t have but is a core stakeholder management skill (see: From Commander to Sponsor: Managing Upwards in the Project Environment) and is critical to the success of Agile in business.

There is also a need to discipline the Agile development process. Far too many IT techo’s see Agile as an excuse for not documenting code, ignoring scope requirements, and having fun writing stuff. To succeed in business, the flexibility advocated in the various Agile methodologies has to be tempered with pragmatism. The delivered software needs to meet the business objectives set by senior management as well as the desires of the SME ‘customers’ working with the team. The key stakeholders for any IT project are the people at the top of the organisational tree. The problem is these key stakeholders are rarely involved in the project. Until AAs can develop a language and a practice that offers some certainty to senior management that their way of developing software will deliver enhanced business benefits and maintainable code the potential benefits will be lost.

The fundamental principles in the Agile Manifesto have support in a number of emerging fields of academic research including Complexity Theory and Social Networks Theory. The challenge for AAs is to communicate these ideas in business language and to temper the exuberance of many ‘Agilists’ who naively believe that simply because you have developed a better mouse trap everyone will buy it. The world of business does not work that way.

To succeed, AAs is to avoid over claiming the benefits of Agile and work the organisational systems to allow Agile to deliver its benefits. This includes demonstrating to management the project is aligned to the business objectives, that only the minimum scope needed to achieve the planned benefits is being developed and any new ideas (ie, scope creep) will be properly assessed before inclusion in the project. Achieving this involves the appropriate use of traditional project management processes linked to effective ‘upwards’ communication.  To keep Agile, agile, the application of these processes certainly needs modification and adaptation from the more heavy handed approaches of the past (eg, Waterfall and PRINCE2) but they cannot be ignored.  The PMBOK® Guide certainly advocates/requires PMs to adapt its processes to the needs of the project – I suggested a few of the key adaptations needed in The Gentle Art of Managing Agile.

Now its up to the Agile community to make Agile relevant to the businesses it is designed to serve. To read more visit my posts on the PMI Voices on Project Management blog, and join in the discussion.

Advertisements

6 Responses to Agile’s Business Failure

  1. […] The following is a comment to Lynda that I was not able to submit to her blog. You can find Lynda’s original post here. […]

  2. […] 19, 2009 · Leave a Comment I have been involved in a series of posts on both my SRMM Blog (see post) and the PMI Voices on Project Management blog (see post) that have stirred up sections of the […]

  3. […] have been involved in a series of posts on both my SRMM Blog (see post) and the PMI Voices on Project Management blog (see post) that have stirred up sections of the […]

  4. Lynda,

    What a great post. I spoke about this a little in my post, It’s the Agile Practitioners that aren’t Ready for the Enterprise, http://www.dennisstevens.com/2009/05/11/it’s-the-agile-practioners-that-are-not-ready-for-the-enterprise/.

    Happily, as Kevin points out, there is a growing community of Enterprise experienced project and program managers who are working to help Enterprises leverage the benefits of Agile through mature Project Management practices.

    I do want to clarify one important point. There aren’t less controls or discipline in a well run Agile project. There are more controls, more feedback, and opportunities for greater learning – both about the ability of the organization to perform the project and about the product being developed. Mike Cottmeyer and I write about this in ReThinking the Agile Enterprise at http://www.cutter.com/offers/rethinking.html.

    As you point out, one of the ways Agile fails to get adoption at the Enterprise level is because flexibility is confused with irresponsible, unfocused, and undisciplined practice. Not by Executive Management but by the Software Developers attempting to apply it.

    I am so happy to have Pat Weaver and you involved in the conversation. The combination of experience and thought leadership around managing teams and managing in uncertainty are needed.

  5. Vasco Duarte says:

    @Lynda

    Nice post. I’m glad that the conversation is continuing. Both PMI and Agile people need this dialogue.

    A couple of comments on your post.

    1. I agree that Agile Advocates need to learn to speak to the business people. However,

    2. Agile adoption has been fast in many places, not slow.

    It is true that many Agile Advocates (AAs as you call them — like the analogy BTW, and think it fits! 🙂 don’t talk “business-ese”. That’s been a failure of the community, however that is far, very far from being a problem in adoption. Having been involved in two major organization change projects I can vouch for the lack of business-oriented communication, but I can also assure you that speed of adoption has been anything but slow. I know of a company that has several 10’s of thousands of employees where Agile is a topic of discussion at all levels, from top management to the team level. This is very encouraging, especially because this company has not been shy in removing some of the building blocks that are in PMBOK (and inadequate to software development) and build new processes at all levels based on the principles of Agile Software Development.

    3. Agile software development needs discipline

    I agree with this, but the lack of discipline existed in software organizations when PMBOK was the prevailing framework for organizing projects. Lack of discipline is a problem that Agile encoutered when it started, not a problem created by adopting Agile software development methods. Indeed, it is my experience that agile software development builds a much higher level of discipline into the development process than was previously the norm.

    4. Agile alignment with business objectives.

    This is a topic for a post in itself, but I’ll just refer one example of how agile brings focus and alignment with business goals. Check this post by Dean Leffingwell where he explains how requirements management in Agile can be done in a way that directly reflects the strategy of the company.

    Let’s continue the dialogue. I’ve learned a lot and hope to continue learning from it! 🙂

  6. Getting Agile into PMI is a good first step to formalizing the movement. This is happening right now. The leaders of the community are shifting their focus from coding/bottom-up agile to business/top-down agile.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: