Today I am reflecting on how difficult it is for individuals and organizations to deal effectively with uncertainty and ambiguity, and how the underlying belief in the possibility of certainty has such a strong hold on us. Examples of that desire for certainty that have appeared on my radar over the last few years include:
- A customer requested that a project be performed using an agile methodology – and at the same time, asked for date commitments for a specific set of functionality.
- An organization outlined a plan for a whole system culture and business change – with some very specific (and tight) timeline targets.
- A project manager expressed frustration over a project being “late”.
The challenge we face is that we often don’t have a sense of how much certainty or uncertainty is available to us for a given problem or challenge. One useful way I have found for discussing this is the Cynefin framework. Created in 1999 by Dave Snowden at IBM Global Services, the Cynefin framework is a sense-making tool.
[Source: Snowded– Own work, CC BY-SA 3.0]
A quick summary of the framework:
- Four decision-making domains are identified: Obvious, Complicated, Complex, and Chaotic.
- A fifth domain – Disorder – represents the state of not knowing which domain applies.
- In the Obvious domain, we know enough, and the situation is stable enough, that rules can be applied effectively.
- In the Complicated domain, analysis is required, but “good practice” can be effectively applied.
- In the Complex domain, the interdependence of variables and the degree of unknowns is so great that the outcome of actions is unknown and unpredictable (think: the butterfly effect).
- In the Chaotic domain, the system is in crisis; immediate action to “staunch the bleeding” is called for – only then can the problem be dealt with as aspects in the other domains.
Our assumptions that certainty is possible point back to our experiences with the Obvious domain. We can have a high degree of certainty as to how long it will take to drive from Iowa City to Naperville (west of Chicago) – the conditions are well known. However these assumptions become less and less valid as we encounter problems in the other domains.
New product development, for example, is often a mix of Complicated and Complex problems. One sign of the Complex domain is the degree to which “doing the work teaches you what the work is”. Working with a new set of technologies, for example, leads to a great deal of discovery. “We found that when we introduced the XYZ component, our earlier decisions about how we used ABC caused us unexpected problems, and we needed to go back and change what we did for ABC.”
If we have a low degree of certainty about what we are building (think: Lean Startup) we may have multiple pivots on the target product as well as on the technology. The same kind of considerations apply in the realm of organizational change – seemingly minor actions on our part or on the part of leaders can create unexpected results, to which we need to adapt. Uncertainty and complexity also creep in when we create new development teams, use multiple teams to work on a single project, and make use of mixed organizations (e.g. outsourcing) for part of the work.
Within a given system, there is a tendency to move clockwise through the domains. In the product development example, increased automation and experience makes the outcomes more predictable – we move towards the Complicated domain. Here, our build-up charts look more stable (we have less discovery, less variation in velocity). We can, of course, be pushed back into the Complex or even Chaotic domains by changes in the environment: a technology becomes obsolete and we need to take on a new one; a strong competitor emerges and we have to pivot our product and business – perhaps in a hurry.
So… what do we do with this?
- It is useful to recognize and acknowledge the desire for certainty… and then ask the question: how much certainty can we reasonably expect for the challenge we are facing? Can the Cynefin framework inform us as to what we can and can’t expect? (We can use Ralph Stacey’s Agreement/Certainty matrix as a diagnostic tool. In the realm of software, one approach is to think of agreement as applying to both requirements and technical approaches, and certainty as applying to “how tried and true is our technology approach – for us?”.)
- In the Complicated and (especially) Complex domains, recognize that we are dealing with a lot of uncertainty, and build in approaches that help us make progress and good decisions in the face of that uncertainty.
What might some of these look like?
- The agile practice of “vertical slicing” of functionality is a good example: by building a very thin, full stack slice of functionality (encompassing as much of our target technology as possible), we learn early what will and won’t work and where we will need to adapt our technology approach, and we start gaining experience and expertise.
- The agile value of “customer collaboration over contract negotiation” suggests getting on the same side of the table with all the stakeholders: “we know that we don’t know, so how will we work together to create the desired value?”
- The transparency afforded by agile methods (e.g. reviewing the build-up chart, incrementally reviewing “what’s been done so far?”) creates space to adapt as early as possible to what we discover along the way.
Much has been written regarding complexity, uncertainty, ambiguity, and approaches for dealing with them. The place to start is to recognize: when are our expectations of reasonable certainty unwarranted? And when that’s true, how will we adapt?
What experiences can you share about adapting to uncertainty and ambiguity?
Originally published January 17th, 2019 on the Innovative Software Engineering blog. Republished with permission.