Agile approaches are now widely used not just in mainstream software development but also in many other sectors. However, experiences working with large established organizations trying to use agile approaches highlights that their application in mature organizations remains fraught with difficulties. What are do their failings tell us about how to succeed in agile delivery?
Over the past decade or more I have worked with organizations around the world as they have adopted agile ways of working. ‘Agile health checks’ conducted at large-scale organizations reveal a common pattern: early enthusiasm and grassroots success for agile practices soon face barriers to broader acceptance when they come up against concerns about their scaling, governance, and institutionalization. Too often, an organization that rushes into a ‘corporate agile delivery program’ ends up smothering the growing enthusiasm with a blanket of management reporting and oversight, and within a short time the efforts are mired in disappointment, scepticism, and a lack of quantifiable results. This gives ‘going agile’ in that organization a negative connotation, regarded as a byword for lack of planning, poor management, and chaotic delivery schedules.
The challenges to agile adoption in large established organization have been built over long periods of time. Organizations find themselves increasingly polarized. On one side are those advocating rapid delivery cycles and close collaboration across the team, colocation of people, and speedy feature release. While on the other side are those focused on delivery governance and control, schedule predictability, and accountability for resource use. Large established organizations are often overwhelmed by the tension that is created trying to support all of these needs.
The solution is to recognize that support for a range of approaches will be required to address different cultures, contexts, and experiences that make up an organization. The fundamental questions to ask in such situations are:
- Which aspects of the organization require more agility?
- What outcome is to be achieved with this change?
- How would the current status, and the impact of this change, be measured in terms of business outcomes?
Strangely, this simple set of questions is rarely asked, and if it is, the answers are frequently unclear or unachievable. When these questions are made explicit, a debate ensues within the organization as different people seek to clarify and prioritize what they expect, and how they will ensure they succeed. The diversity and depth of this debate that this surfaces offers an important indicator as to the extent to which agile adoption is understood and planned.
Over the past few years I have worked with many large established organization adopting more agile approaches. From these experiences, I can identify four ‘anti-patterns’ for agile success; characteristics that underlie the failure of agile adoption.
Using agile as an excuse to avoid good management practices
Organizations typically spend many years adopting and refining management practices to suit the context in which they are used. Representing years of corporate history, deeply integrated with common practices and tools and embodying key aspects of the organization’s culture, these practices cannot and should not be put to one side. Many complaints about agile adoption centre on the idea that well-understood, functioning processes are abandoned without sufficient thought to their replacement, or to the impact on skills, projects, and management practices.
Adopting agile for new activities, not as an end-to-end delivery approach
Most agile adoption efforts begin with small teams introducing agile practices within tasks with limited scope and impact. In many cases these practices are not part of any formal enablement activity; they are simply local efforts by individuals and teams doing what they believe is most effective in writing and delivering good software. With broader adoption of agile practices, other software development teams quickly see the benefits and pick up these ideas as their own.
Such grassroots adoption, while valued and celebrated, inevitably introduces challenges across the organization. The dispersed and diverse nature of product and service delivery means that those involved in the core software production have much less insight and understanding of what has taken place with the agile teams. Often, they are only aware of the additional challenges that agile approaches bring to their traditional delivery mindset and techniques: no clear upfront commitments to a fixed budget, lack of medium- to long-term planning, and progress measures that do not align with company baselines. They may see agile as bringing more problems, not as a source of benefits.
In the worst cases this results in stalling agile adoption as it is stifled by project managers, business analysts, contract negotiators, and auditors who do not feel empowered by agile approaches; instead, they feel out of control. In the better cases this can lead to a clear two-tier approach to agile adoption, sometimes referred to as an “ambidextrous organization” as discussed in Chapter 2. Here, some of the teams adopt agile methods such as Scrum, while the rest of the organization remains committed to more traditional phased-based methods. While some progress is possible this way, it can be challenging to keep these two worlds interconnected and aligned.
Believing agile to be the new silver bullet
As with any new technological advance, early successes are often overhyped and under-analysed, resulting in unrealistic expectations. Agile adoption has seen its successes, with organizations such as Google, Microsoft, eBay, and Facebook all claiming that agile approaches allow them to deliver more of what customers need more quickly. Hence, expectations are naturally very high.
The reality, however, is that every organization is different in its structure, history, culture, and market position. Lessons from other organizations’ experiences are very helpful as guides, but by no means do they guarantee that approaches successful in one company will work in another. These lessons need to be analyzed, understood, and adapted. Knowing how, where, and when to apply agile practices is just as important as knowing what it is, and who else has used it successfully. Unfortunately, many organizations try to introduce agile practices much too quickly, with overblown expectations of what results will be achieved. This is counterproductive as it diminishes executive management interest in smaller-scale impacts and demotivates all other staff trying to change behaviors well established over many years.
Focusing on agile as tool-driven methodological change
There has been much interest in agile practices as a new way to design and deliver software, resulting in agile methods such as Scrum, XP, DSDM and Crystal that capture these ideas for use by software-intensive businesses. Each method has its supporters and detractors and is eagerly supported with training and consulting. These in turn are encapsulated in a myriad of open-source and commercial tools that help the application of these methods. As a result, adopting agile practices is seen in many organizations as an IT issue, and reduced to questions of which software-specific method and tools to acquire, which training classes are needed, and how do software development teams accelerate creation and delivery of more software into production.
Unfortunately, such a narrow perspective obscures the bigger questions to be addressed: how do I get my organization to think and operate from a more agile viewpoint given my technical, commercial, and economic context? Invariably, one of the main challenges to agile adoption lie in the lack of skills and education that inhibits progress when organizations overemphasize the software delivery methods and tools.
Experiences with several large agile organizations suggest that agile approaches can be successfully applied in almost every kind of development team, and in almost all organizations. While each situation has its own specific needs and variations, the core agile principles can provide significant results when applied with care, and with realistic expectations.
Summary
Adopting new digital ways of working in complex and uncertain times requires many changes to operating practices, processes, management structures, and decision making. In situations where complex, structured development and delivery practices play such a large part of the solutions, significant adoption of agile practices are at the core of these changes. Collaborative teams employ new approaches to delivery aimed at optimizing processes toward managing progress in short sprints, increasing visibility, and encouraging feedback and review.
More substantially, as summarized by scrum.org the leading proponent of agile methods, these techniques also introduce a core set of values into the organization’s culture: commitment, courage, focus, openness and respect. Far from a mechanistic process of introducing new technologies, becoming an agile organization in these difficult times will not be defined by deployment of digital tools used by workers in individual projects, nor by the grand pronouncements from high-level managers at all-hands meetings. It will be the result of well-structured change programs focused around agile values which will drive meaningful consistent change to an organization’s corporate culture.
