Skip to main content

Author: Gary Stocks

Is “Agile Governance” an Oxymoron?

Project sponsors, typically C-level business executives accountable for project spending and outcomes, can be excused for being sceptical about Agile project delivery approaches because there is a perception that they don’t support governance. Agile should not be ignored though, because it is proving to be a more effective way to deliver change faster and in a more flexible manner. In contrast, traditional methods used for delivering software solutions have not been very successful, industry studies (e.g. Standish Group research) have been reporting failure rates above 60% consistently for many years.

The challenge is to apply Agile successfully to enterprise-wide solutions with large budgets, and more stringent investment approval and governance requirements. Many successful Agile projects have been small and focused, targeting relatively simple solutions with few integration points and small co-located teams.

What is governance in the context of enterprise solutions?

Large, complex enterprise-scale programmes and projects, such as the development of core systems, are usually governed by strict controls. The controls are understandable as these initiatives involve significant investment of financial and other resources and introduce significant risk to the organization. In this context, governance – which is aimed at achieving business outcomes and essentially defines decision-making responsibilities – includes upfront investment approval and oversight of project execution.

Upfront, a business case with quantified costs and benefits is typically used to gain investment approval. The costs are estimated based on a scope of work and defined work products or outputs.

During execution, the project steering committee would hold the sponsor accountable through regular progress assessments against a plan that includes defined outputs. Significant risks and issues are reviewed for a decision on what to do about them.

There are various levels of governance, which the investment committee delegates to the project sponsor, who would delegate to the project manager in turn, and so on. The fundamental point is that funding is tied to a defined final work product.

Do Agile approaches contradict traditional governance when used for enterprise-scale initiatives?

There is usually a change required in governance processes because linear, Waterfall approaches are often tightly integrated with governance steps. However, this process change is not fundamental because the principles remain the same.

1. Investment governance 

Agile approaches aim to release working software into production more frequently to prove whether the solution will be used and provide an early indication of whether the benefits are likely to accrue. In principle, Agile approaches support the main goal of governance which is to achieve business benefit from investments. However, the fact that there is less certainty regarding the form/specification of the final solution is a cause for concern to the investment committee. They need to bear in mind that the apparent certainty provided by the rigid specification of traditional waterfall approaches is an illusion: software-based projects are notorious for being late and over budget due to inaccurate upfront estimates.

Recommendations for project sponsors planning to use Agile approaches:

    • Ask an experienced Agile practitioner to explain Agile principles and successes, and use this to educate the investment committee
    • Request a high-level effort and cost estimate based on a short analysis cycle for a subset of requirements extrapolated to the full set of requirements, and use this to secure an initial budget
    • Use the practice of continuous releases of production software to provide additional visibility of progress to the investment committee

2. Architecture governance 

Agile approaches do not naturally lend themselves to Architecture governance as the architecture of an Agile solution emerges rather than being specified in detail before building the solution. For enterprise-scale solutions, this approach can lead to significant issues due to the greater level of integration and importance of non-functional requirements, such as security and scalability.

Recommendations for project sponsors planning to use Agile approaches:

    • Facilitate engagement between the Architecture function and the Agile solution team to agree on the level of architecture detail that is required before the solution build commences and how the architecture will be defined during the build process
    • Facilitate ongoing communication and synchronisation between the Agile solution team and enterprise support functions such as Architecture, Marketing and others

3. Project governance 

The main conflict between Agile approaches and typical governance processes usually occurs at the project level. The structures and tools employed for governance are often tightly integrated with linear, waterfall approaches. For example, the project governance approach expects documented specifications, whereas Agile approaches do not track progress against such specifications.

Recommendations for project sponsors planning to use Agile approaches:

    • Use a high-level definition of the solution as the basis for the Agile solution team’s mandate
    • Insist on a detailed definition of each iteration before that iteration is commenced
    • Ask the Agile solution team to define how they will build confidence – through practices such as frequent reviews, burndown charts and visible storyboards

Conclusion

Although Agile governance is not an oxymoron, the project sponsor needs to understand and communicate the required changes to be successful. The biggest change is at the project governance level, where plan-based, Waterfall delivery approaches tend to be tightly integrated with project governance activities. Like any change, strong leadership – from the project sponsor in this instance – will create the necessary change momentum.

Enterprise-scale Agile: requires moving from ‘Us versus Them’ to ‘We’

stocks IMG001 Mar4

Organisations are increasingly experimenting with ‘agile’ approaches to software development – combining incremental and iterative ¬¬¬approaches in one. To manage the risk of this change, initial experiments are understandably small and focused¬¬, targeting relatively simple solutions with few integration points and co-located teams of five to seven people.

Many of these experiments are showing success, with benefits such as faster time-to-market and more productive teams leading to enterprise-scale adoption. Both custom-developed software as well as customisation of commercial off-the-shelf (COTS) solutions, such as SAP and Oracle, are being implemented in this manner.

However, the use of agile at such a scale is yielding mixed results, despite the emergence of numerous frameworks to address this challenge, such as SAFe (Scaled Agile Framework) and DAD (Disciplined Agile Development). Therefore the solution doesn’t involve merely selecting a method, one of the key challenges with the transition to agile is the culture change, and this is further intensified when looking at enterprise-scale agile.

The ‘enterprise-scale’ environment

An enterprise-scale project environment is characterised by multiple workstreams, sub-projects and numerous teams, often including external members and/or teams that are possibly geographically dispersed. Moving from a traditional waterfall-based approach to an agile approach involves a complex transition that potentially starts with one workstream introducing agile.

stocks IMG002 march4

Changes in the role of Business creates confusion

Arguably the biggest culture change is that business stakeholders become part of the team, notably in the form of a ‘Product Owner’ role to prioritise features / stories based on business value. While this may be intuitive for an entrepreneurial business stakeholder, such as the leader of an innovation business unit, it becomes much more difficult for senior operational business leaders that need to balance change with significant operational responsibilities. A key business stakeholder that is comfortable with the traditional role of reviewing documented deliverables will now have to play a much more dynamic Product Owner role in an agile environment. They will be called on to make prioritisation decisions much more frequently and without significant documentation to reflect on.

stocks IMG003 march4

This situation is further complicated when there are multiple teams that are delivering through a combination of agile and traditional waterfall methods. A business stakeholder may be confused by different engagement approaches across teams, one that involves frequent prioritisation based on user stories and another that involves review and approval of all requirements in a formal specification of business requirements. This situation may call for a ‘Product Manager’ role to prioritise among different teams, some of which may be prioritising continuously based on business priorities determined by a Product Owner and others which may not be sequencing development based on business priorities.

Key recommendations:

  • Review the existing project organisation structure to assess if it will support agile principles
  • Clarify the role of business stakeholders, such as the Product Owner and Product Manager roles

A focus on team success drives a different culture within an agile team

An agile team environment is characterised by a different culture, with one of the main attributes being collaboration rather than individualism. Team performance becomes more important than that of the individual, which necessitates changes in the success measures and behaviours. People handle change differently and, for those new to agile, the adoption of a different way of working takes time and motivation.

Acceptance of change is another key attribute required for successful implementation of enterprise-scale agile. As the business begins to better understand agile solutions, and in response to changing external conditions, so it will adjust the priorities of the project, which will affect the project team.

In addition to the individual changes required, many large enterprises have IT operating models that do not support lean and agile methodologies. Analysis, build, test and production teams are often kept separate with different reporting lines, which can hinder agile processes.

stocks IMG004 march4

Key recommendations:

  • Gain a concession for a team, and the individuals therein, to measure success and performance differently
  • Understand the competence fit for team members
  • Introduce team effectiveness interventions to create self-awareness and a desire to develop required competencies
  • Provide support for the behavioural change, such as through coaching

Coordination across diverse teams creates complexity

When there are multiple teams in a large, complex programme, the challenge is to resolve inevitable conflict when not all teams and stakeholders are incentivised and held accountable for team performance. For example, the integration team may be governed by 6-monthly production release cycles when agile aims to release integrated production software more frequently. If this constraint can’t be overcome, the ‘agile’ team cannot reach its goal of continuous releases of working software and so it will inevitably fall back into an incremental approach. A similar situation was encountered when attempting to scale agile at a major retailer. The integration team wasn’t ready at the end of the build cycle, which lead to a few months delay in the release of the software.

Many enterprise-scale projects employ consulting service providers and contractors. The prevailing culture can be entrenched to the extent that contracts for external service providers measure deliverables rather than working software. This was the situation at a major mobile network operator where the services agreement for an external service provider included payment conditions for the approval of Business Requirements Specification documents and the delivery of software to meet those specifications.

Key recommendations:

  • Identify components that will require high business interaction because the focus on business value is a key driver of agile approaches
  • Identify components with limited interfaces to other components and limited integration to prove the concept of agile (‘a thin slice of the cake’ so to speak, where the ‘cake’ is the whole solution)
  • Include an integration specialist and tester onto an agile team and provide dispensation for releases outside the normal production release cycle
  • Assess whether the ultimate team goal of working software can be achieved; if not, re-consider whether the organisation is ready to introduce agile

Don’t forget to leave your comments below.