The digital economy has revolutionized business models in a way that was inconceivable just a few years earlier.
Products are launched as early as possible and perfected along the way, learning from mistakes and using enormous quantities of data. Fail fast, fail often, fail better, fail forward has become the mantra of digital entrepreneurs worldwide.
The UN evaluation sector is not the digital economy. We do not take failure lightly. A lot of tax payers’ money goes into our evaluations, policy decisions depend on those results and for those of us who work in the humanitarian field, every dollar spent on evaluation is a dollar less that reaches people in dire need of aid. Failing fast and often is not encouraged. In our line of work, getting things right is considered the norm. And if we don’t get it right, we avoid drawing too much attention to it.
I was never under the impression that the UN operates like a tech start up, but it still seems odd to be at polar opposites when it comes to dealing with failure. Is there anything that we can learn from the iterative fast-failure model?
I started asking myself this question when I had to deal with my own failures. The cutting-edge evaluation I had so carefully planned and managed for an innovative pilot programme failed miserably. We could not provide a definitive answer to the main evaluation questions. Two years of work, money, effort only produced inconclusive results. The scepticism around the evaluation and, by extension, my work was palpable.
A few weeks later, I started drafting a lessons learnt document to chronicle what had happened. The more I would identify key issues and discuss them with colleagues to understand what had happened, the more I became aware of the great unexpected and unheralded results of the evaluation. When did we realize that our targeting was excluding many people who truly needed our intervention? The evaluation baseline pointed out that our targeting was flawed! What made us reconsider the transfer modality? The evaluation did! Maybe the statistical sample was inadequate to answer the main evaluation questions, but we learnt so much about the implementation of the programme that investing in the evaluation paid back several times over. Conducting an evaluation of a new programme, full of uncertainties and unknowns, was not such a bad idea, after all.
Does this mean that I am now going to recommend making spectacular mistakes in your next evaluation? Hardly. Mistakes are a costly and messy business. We want to avoid mistakes ourselves and make sure that people around us do not think that it is OK to screw up. However, at the same time, you want to encourage people to take risks, explore new ways of doing things and, when they fail, admit it and learn from it.
In 2011, Amy Edmonson proposed an approach [1] to deal with failure that would do exactly that: encourage exploration without promoting an “everything goes” mentality. She sees mistakes on a continuum that goes from routine to frontier (see table 1 below). Routine mistakes should be discouraged, quickly identified and penalized. For instance, an enumerator who cannot stick to the script when asking questions in a survey needs to be corrected immediately and should be removed if the mistakes continue. On the other end of the spectrum, frontier mistakes are made in situations that are complex or uncertain, where we are pushing the boundaries of what can be investigated. In this case, a failure is not only acceptable, but even desirable because of the intrinsic learning potential.
Table 1
Source: Strategies for Learning from Failure
Failures are inevitable. Hiding our failures is not. The only real failure that I can think of is when we miss the opportunity to learn from our mistakes and grow. Creating a team culture where mistakes are acknowledged and openly discussed is the first step towards becoming a learning organization. The evaluation sector that prides itself with its evidence generation work should lead the way in creating an environment conducive to admitting failure. I do not think that we should fail fast and often, but we should certainly learn to fail better.
[1] Amy C. Edmonson, Strategies for Learning from Failure, Harvard Business Review, April 2011