Roberto [user:field_middlename] Borlini

Roberto Borlini

Evaluation Officer
World Food Programme
    • When I was just beginning my career in the development sector, I received a strange request from a donor: can you develop the logical framework of your programme? That was indeed a weird thing to ask three months before the end of the intervention. In hindsight I recognize that they were probably trying to cover their tracks after having skipped some steps in the approval process. At the time, with some grumbling, we complied with their request producing a half-baked logframe that was plausible, but not useful to anyone. It did not help with the planning, the management or the evaluation of the project. It was just a formal requirement, a box to be ticked. Now, let’s skip forward 20 years. I am contacted by a country office that asks for support. Their flagship programme is coming to an end and they want our help in preparing for an evaluation. I ask for some documents to understand what the programme is all about, but what I get is mostly donors reports that show the intervention through pink lenses. So, I suggest “let’s develop a theory of change!”.

      No, I am not crazy and I have not become the bureaucrat that I dreaded 20 years ago (not quite at least). What I am suggesting is to sit down together and have the country office staff tell me their story so that we can piece it together to understand what they did, how and why. This is something that we do routinely in the East and Central Africa Region of the World Food Programme, where theories of change have become the bread and butter of any decentralized evaluation since 2018. The initial push to develop theories of change came from the Evaluation Unit at the Regional Bureau where we felt that having a solid understanding of how the programme works, available in an easy-to-share format, would provide external evaluators with a strong head start. Progressively, this exercise became more sophisticated and structured. We started paying more attention to assumptions as stress points where we can focus evaluation questions. We made the process more inclusive and participatory, involving external stakeholders. We added a problem tree analysis and a stakeholder mapping as propaedeutic steps before designing the theory of change. Finally, we decided to give it a name and we opted for “evaluability assessment”, as, among other purposes, we also use it as a tool to decide whether an evaluation is desirable and feasible.  

      We have conducted a number of these exercises (14-15? I lost count…) and we never stop learning and improving our approach. In one case, we conducted an Evaluability Assessment together with a country office where management was caught in the crossfire between two donors who requested evaluations of their respective grants but did not budget adequately for the exercises. When we developed the theories of change of the two interventions, we started to see linkages between impact pathways that were not visible just comparing the respective logical frameworks. The two grants could be easily captured under one comprehensive theory of change that mirrored the reality of implementation on the ground where programmes merge into each other seamlessly. We then proposed to conduct one evaluation merging the two budget lines to look at the overall intervention, highlighting complementarities and synergies. Guess what? It worked!

      One of the beautiful aspects of theories of change is that we do not need to develop them at a specific point in time. TOCs are useful at the beginning, in the middle or at the end of an intervention. Their main use and applications will be slightly different, but it is never too late to develop one. If you do it at the beginning, you can focus more on developing a shared understanding of how the programme works and pursue more active collaborations between different layers and components of the programme. It can inform the development of monitoring indicators, help in tracking risks and mitigation measures, not to mention the design of cross-sectional evaluations. Developed towards the end, it still has incredible value for informing the evaluation design and the key evaluation questions that you want answered.

      One measure of the success of this approach is that now we receive requests from country offices for support in developing theories of change even when there is no evaluation in their plans. They see value in this exercise. They understand how it helps to formalize their collective thinking into an easy-to-grasp visual representation. No matter if the request comes in before the first bowl of porridge has been delivered or three months before the end of the programme, it is never too early or too late to design a theory of change.