The “Overhead” of Agile – Perception vs. Reality

When I first learned* about Scrum, the trainer provided us with some “rules of thumb” for planning our time during a sprint. As we started putting Scrum into practice, we applied these rules of thumb… but also encountered shock and horror from outside the team about how much “overhead” was involved. The arrival of Ken Rubin’s recent blog post, How Much Time Should Each Scrum Practice Take? reminded me of these early experiences. In this blog post, I reflect on the perception vs. reality of this “overhead”.

* Here, I am counting the first actual Agile training I attended, as distinct from the casual encounters with Agile that occurred as we interacted with various clients.

The rules of thumb that I recall from that early learning lines up fairly closely with Ken Rubin’s recommendations. These include:

  • Backlog grooming/refinement – up to 4h per week
  • Sprint planning – up to 2h per week of sprint duration
  • Daily standup – 15 minutes/day
  • Sprint review – up to 1h per week of sprint duration
  • Sprint retrospective – up to 1h per week of sprint duration
  • When planning tasks at a detailed level, assume that you only have about 80% of the sprint to work with (e.g. for a 2 week sprint – 10 work days – assume that you have no more than 8 days to allocate to sprint execution)

The shock and horror that I recall was centered around both the 80% assumption, as well as adding up the hours per week spent in the other sprint activities. Our traditional way of thinking was about “efficiently allocating resources”, and this new way of thinking seemed to involve a lot of “extra activity”.

With the benefit of time and experience, here is my perspective on these time allocations and perceptions…

Agile distributes analysis and planning over time

Our traditional way of thinking (waterfall) concentrated analysis and planning up front… for a largish project, we would spend many weeks capturing requirements, performing analysis, designing and planning… all before “development” activities began. Under Agile approaches, this work is performed incrementally over the life of the project – much of it in backlog grooming/refinement and sprint planning. Unnecessary work is avoided by deferring analysis and planning work until near to when it is needed.

Lesson learned: teams that resist backlog grooming often set themselves up to struggle during the sprint because backlog items are not sufficiently defined or ready.

Agile assumes we will learn along the way…

When we did most of the analysis and planning up front, we had a plan and approach with enough investment in it that we were reluctant to change it – even as new information was learned during the course of the project. Agile approaches encourage lightweight planning so that we can adapt as the project progresses. Early customer deliveries and feedback, frequent reviews, and knowledge “gained by doing” all contribute to better decision making over the course of the project. This means we are more likely to deliver something valuable and effective vs. delivering “what was specified when we had the least knowledge”.

…and actively encourages that learning

Time spent in Sprint Review and Sprint Retrospective is all about learning and improving – product, processes, and team. The built-in learning means that we are more likely to improve over time and to adapt to unexpected realities sooner.

Lesson learned: feedback and learning are key… teams that abandon the mechanisms for feedback and learning consign themselves to struggle when unexpected realities intrude, and to continue in a path of likely mediocrity vs. excellence. As a Scrum Master or Agile Coach, if you are encountering resistance to sprint reviews or retrospectives, look for ways to make them more effective and engaging.

Agile creates conditions for innovating and solving complex problems

Agile approaches encourage ways of working in concert where a team’s results are greater than the sum of what the individuals could accomplish alone. A strong agile team makes the most of diverse expertise and perspectives to come up with better solutions and approaches, that result in better outcomes over time. A team that knows what value it is creating – not just what tasks it is accomplishing – is better positioned to make breakthroughs that deliver value in unexpected ways.

Lesson learned: It can be difficult to overcome the individual “let me work in a box and hand off what I have done” mindset. As a Scrum Master or Agile Coach, think about “How can I set the team up to have successful experiences of teamwork? How can I help the team members to connect with the valuable outcomes we are creating?”

Agile favors effectiveness over efficiency

Our traditional mindset had us thinking about efficiency:

Efficiency = Hours Applied to Doing The Work / Total Team Hours Available

Agile approaches are not about optimizing efficiency, but effectiveness. One way of looking at effectiveness is:

Effectiveness = Valuable outcomes created / Total Team Hours Available

As illustrated in Tom DeMarco’s book Slack, striving for 100% allocation creates bottlenecks and slows down throughput… reducing the value delivered. This doesn’t mean that we expect people to sit around waiting for work – but it does mean dedicating people to teams and not attempting to schedule work so that everyone is 100% utilized. By allowing enough slack, building in activities related to learning from experience and discovering what is most valuable, and leveraging the capabilities of the team, agile approaches encourage the creation of most valuable outcomes.

Lesson learned: Signs you might be favoring efficiency – and slowing the delivery of valuable outcomes: delays getting code reviewed (everyone else is too busy); delays completing testing on new functionality (no-one available to test); team rarely experiments with new tools or techniques (barely keeping up with the work, no time to tinker); only the “obvious” solutions are investigated (no time to engage in divergent thinking).

Perception vs. Reality: The “Overhead of Agile”

Does Agile have more “overhead”? We certainly spend more time during the course of the project on these visible agile activities, compared to our traditional mindset. It is difficult to compare directly: we rarely get to perform an experiment where we perform the same project with comparable teams and different approaches. However there are indicators we can look for to see if our different allocation of time is helping us…

  • Are teams solving complex problems or making breakthroughs based on a clear vision of the valuable outcomes being created?
  • Are teams experimenting with “possibly better” ways of doing things?
  • Are teams applying experience to “learning to do better” over time?
  • Are we changing course during a project based on new information about reality and what’s valuable?

If the answers to any of the above questions are “no” or “not much”, you might look for ways that you are favoring efficiency over effectiveness, as well as ways to become more effective. 

 

Is your scrum team focusing on efficiency over effectiveness?  How has your team improved it’s effectiveness?  What are areas your team could be more effective?

Originally published February 8th, 2018 on the Innovative Software Engineering blog. Republished with permission.