How useful are theories of change in development programmes and projects?

Hi Carlos,

Please find below my feedback on use of ToCs.


Essentially, ToCs are meant to capture the changes that an intervention undergoes since its planning till the time it concludes and thereafter in the form of impact. The ToC ladder is usually characterized by inputs, activities, outputs, outcome and impact. Typically, project monitoring teams engaged in day to day management of projects can only foresee the little use of ToCs in the form of immediate output (short term), whereas the difference that project is expected to make will be assessed by an evaluation team that gauges the outcome (medium term) and impact (long-term) of the interventions.

We have used this tool in designing an M&E system of our national strategy on financial inclusion since 2015 in collaboration with World Bank Group. I can say with surety, its highly effective in overall development evaluation paradigm given that proper groundwork, planning and expert choice is made before choosing relevant indicators (true and fair representation) across the ToC ladder.

In brevity, few of the takeaways that I can summarize are as follows:

* This is a tool mostly used by expert M&E professionals and probably not that much by project managers or staff involved with trivial monitoring assignments.

* It is necessary to understand the gist of ToCs which tacitly works in coherence with the principles of Result- Based Monitoring & Evaluation (RBM) as compared to conventional monitoring practices.

* Last but not the least, if you are looking to develop an M&E system that works in coherence with the results of multitude of interventions (national/global level), there is no best tool but ToC to map results across every level that can be fitted under broader and aligned evidence capturing framework.

That's why, ToC has gained quite a significance for evidence-based policymaking and so much so in the design of new interventions based on prior evidence. For review of the practitioner's work, I would suggest Linda G. Morra Imas and Ray C. Rist (2009) guide, "The Road to Results". Further, I already shared my publication, "Designing Robust M&E Systems..". I hope, here you can find the examples of ToC being mapped from RBM system.

Will be happy to assist further in case of any queries.

Regards,

Zahid Shabbir
Joint Director
State Bank of Pakistan

Thank you Jackie and Richard, and all the previous commenters! It has been an interesting discussion, with so many different points of view and insights. We will soon be wrapping this up and summarize the learning in an Evalforward's blog. Keep an eye on it!

Best to all,

Carlos

As I look at the theory of change (TOC), particularly as it applies to development and more particularly smallholder agriculture, I wonder if we are getting an accurate analysis of its effectiveness, or is it getting caught in the implementers' need to appease the donor to assure project extensions and future projects.  This results in projects appearing far more successful than they really are with a lot money invested for limited benefit and the intended beneficiaries being left with little impact.  

I think a lot of the TOC being applied to smallholder communities is derived from academia and based more on what is socially desirable than effectiveness in sustained change and implying a wider use in the donor home country than reality which is misleading to the host country. Much of the TOC being imposed on smallholder communities is based on organizing and relying on farmers organizations or peopleware instead of hardware or software. The idea is people can be organized to work in their collective vested interest, even when this conflicts with individual vested interest. Unfortunately, individual vested interest usually will eventually take priority, so these TOCs require continued external facilitation, if not direct subsidies to survive and collapse as soon as the last advisors departs, perhaps before the they clear the departure lounge for the flight home.

The best example of this is the nearly 40 years imposition of cooperatives to provide business services to smallholders in terms of consolidated inputs and markets. This is done with blanket, but never substantiated thus slanderous, vilification of private traders. This is imposed even while the cooperative movement in the USA, which had some major positive impact a century ago, has been in decline for several decades and when last reported 20 years ago represented less than 30% of agriculture business activity. I would image the same would be true for cooperatives in the EU or other donor bases. The problem with the cooperative model is that it is administrative cumbersome which translates to high overhead costs that, in the financially suppressed developing world, will quickly exceed the financial benefits of bulking inputs and marketed produce. When this happened relying on cooperatives will force smallholder farmers deeper into poverty.  The result is that cooperative based projects only attract a small percent of the potential beneficiaries, perhaps 10 to 12%, and even then, the bulk of the members production is side sold to the vilified private traders, leaving the cooperative with little more than in-kind loan repayments. Not what you could objectively refer to as a successful project. Yet, with some creative accounting they are always considered successful at least while someone is facilitating them.

Another example is the use of Water User Associations (WUA) to manage irrigation water and maintain canals. They are based on the Ditch Companies of Colorado (where I am writing from) and other western USA states. Ditch companies’ success is based on very strict and rigidly enforced water law that is not available in most irrigation schemes in which WUA are imposed. Thus, while there is a common vested interested in maintaining irrigation canals, the individual vested interest does a 180-degree shift once you pass the individual’s inlet. Again, they can exist while there is external assistance but will collapse once that ends.

Please review the following webpages including internal links:

https://smallholderagriculture.agsci.colostate.edu/appeasement-reporting-in-development-projects-satisfying-donors-at-the-expense-of-beneficiaries/

https://smallholderagriculture.agsci.colostate.edu/perpetuating-cooperatives-deceptivedishonest-spin-reporting/

https://smallholderagriculture.agsci.colostate.edu/request-for-information-basic-business-parameters/

https://smallholderagriculture.agsci.colostate.edu/mel-impressive-numbers-but-of-what-purpose-deceiving-the-tax-paying-public/

https://smallholderagriculture.agsci.colostate.edu/financially-suppressed-economy-2/

Thank you, Carlos, for bringing up another important topic in the forum and for the document links. I also thank the colleagues who shared their experience and comments.

Theory of Change, Logical Framework (Logframe) and result Chains are all methodologies for planning, measure and evaluate programs. Each has a visual representation in the form of a matrix of what happens or expected to happen as the result of the program or project or any initiative for that matter. As a past facilitator at IPDET (International Program for Developmental Evaluation training financed by the World Bank Program) and in my practice, I have found that Theory of Change is a name or title that is not easily identified as a methodology;  it is a term that can bring dread in the mind ?.  I would rather use Logic Model a term some use interchangeably to describe ToC.  However, Logic model is also synonyms for Result Chains (https://www.betterevaluation.org/en/search/site/result%20chain) and Program Theory/Theory of Change https://www.betterevaluation.org/en/rainbow_framework/define/develop_programme_theory.  

In my view, Theory of change is a more powerful tool than the two others mentioned since direct links between Activities, Output and Outcomes (Short and mid -erm and long-term often referred to as Impact) must be established and shown in the matrix. Furthermore, the ToC is not complete without assumptions. It is not a one-time matrix but must/can be reviewed and modified with time. During evaluation, the assumptions must be verified and if they do not hold or activities were modified during the course of the program, the matrix has to be reviewed accordingly.

The requirements of the ToC foster an in-depth reflection of what the program is trying to achieve. A difficulty often encountered is deciding what is an output and what is an outcome. I have found the Kellogg document a very useful Guide; it uses the term Logic model. https://www.bttop.org/sites/default/files/public/W.K.%20Kellogg%20LogicModel.pdf. Semantic is important in the developing the matrix and active verbs such as “Increased” that denote changes, help make the distinction between output and outcome.

 I also find that filling the Activity column brings discussions that often show that stakeholders are not aware or had different views of what was actually happening during the program implementation. Follow-up discussions would often happen on whether the activities will trigger behaviour change among the program beneficiaries and have spill-over effect and results  in the community or overall target population.   I find that the ToC matrix facilitate the identification of indicators which are more meaningful for measuring performance and results.

I agree that the ToC must be developed in a participatory manner. However, at the time of evaluation, it may not exist or the one available is poor/confusing. In these cases, after an initial document review and discussions with the program staff, I will design the matrix or modify the existing one and circulate it. It helps me understand the program and formulate my requests for clarification. Since the matrix is simple to read (I like left to right), it usually receives attention and feedback.  We end up with all involved having the same understanding of the program and its expected achievements. I have seen evaluation questions revised as the result of this exercise.

It would be interesting to find out how many Ipdeters who are practising evaluation, utilize ToC in their work. ToC is at the core of the IPDET.  See Road to Results, Morras and Rist 2009, the textbook for this training. https://openknowledge.worldbank.org/bitstream/handle/10986/2699/52678.pdf?sequence=1&isAllowed=y

Dear Colleagues,

I wanted to raise one aspect that has apparently become a standard procedure when doing ToCs - many colleagues have said that they use this tool largely for programme design/implementation/evaluation.  However, literature suggest that "Theories of Change may start with a program, but are best when starting with a goal, before deciding what programmatic approaches are needed." (see AEA presentation shared in a previous post)

Thus, the starting point for ToC should ideally be development/humanitarian goals in a particular theme (poverty/hunger reduction, climate change adaptation, rural development, women empowerment, saving lives) that have been identified by key stakeholders (usually Government, since they represent us all, or humanitarian actors in their absence) for a given geographical area (country, state/region, province, district), and not the programme (or project) specific goal.

As an example of this, in a recent evaluation of FAO's contributions to the development of the food and agricultural sector in Mexico (http://www.fao.org/evaluation/evaluation-digest/evaluations-detail/en/c/1202316/), we used the Mexican government's theories of change to map and then evaluate FAO's contributions. The Mexican government (an OECD member) indeed had by law to develop theories of change at different thematic and geographic levels (national/state) as part of their long-term (national development plans) and medium-term (strategies and programmes) planning process, often with CONEVAL advice (https://www.coneval.org.mx/Paginas/principal.aspx). This together with the fact that FAO had planed its programme of work along the lines of the Mexican's theories of change enabled the evaluation to assess FAO's contributions against these frameworks.

I was wondering if other colleagues have experienced developign ToC having locally-agreed/owned development/humanitarian goals as starting point (and not the specific funding agency goal in mind), and whether they think this is a feasible way forward in their own countries/agencies.

Best regards,

Carlos

Dear Elamin,

Thanks for your question. A few years ago the American Evaluation Association had a discussion on this topic (ToC vs Logic Models). Below is a link to the presentation made by Helene Clark (The Center of Theory of Change) during this session https://www.theoryofchange.org/wp-content/uploads/toco_library/pdf/TOCs_and_Logic_Models_forAEA.pdf

Best regards,

Carlos

Dear Silva

The logic model and ToC are more or less interchangeable meaning, which usually translated in the construction of the LogFrame.

Neil could help clarify this issue of Theory of Change and its linkage with the LogFrame, please.

Elamin
EU consultant
Lead Evaluation Expert
Nairobi

Thank you all for all the wonderful contributions on the topic. I have worked with projects where a ToC was drafted during programme conceptalisation and planning and i have worked with projects where there was no ToC. I have found that when there is no ToCs, project interventions tend to be more focussed getting the activities done other than on the change the intervention is expected to bring. In this case, it feels like doing the work without the vision. However with ToC present, change aspect of the intervention is much more pronounced and it affects the way things are done during project implementation. Logframes and results chanins are also more important. But they are much more useful when their design isalso informed by a ToC. 

Very useful and enriching discourse. The learning on this platform on various M&E concepts/constructs/emerging issues is tremendous.

What strikes me is that we all discuss ToCs as if they were "a thing"....

Talking about a "logframe" is easy: there is standard format to it. 

It might be slightly adapted, but it is quite clear what it is, how it looks like, how it works.

The same is not true for ToCs.  What a ToC is can be vastly different.

I feel we might all use the same word, but having something vastly different in mind...

Best

Silva

Thanks Svetlana, Silva and others for this interesting conversation. We all seems to agree that the use of ToC in programme conceptualisation and also in programme evaluation is gaining currency. Since it’s a fairly recent phenomenon, in the last decade or so, when a large number of programmes and organisations have started incorporating ToC in their way of working, it might be too early to say what difference it has made to these programmes /organisations (I have not come across a study on the impact of the use of ToC!). To my mind, based on my experience of working with several programmes and organisations, three cohorts may exist on how ToC are being used:

1. ToC have become deeply ingrained: This cohort used ToC in all aspects of programme planning, implementing and tracking. You may agree that this won’t be dominant cohort, say about 10% programmes and organisations.  
2. ToC is used sparingly /occasionally but still somewhat usefully - This is likely to be a dominant cohort (say 50%) i.e. a large proportion of programmes and organisations are using ToC sparingly but still somewhat effectively. What it means that programme design include conceptualisation of a ToC. Further annual review and programme evaluations are based on the ToC. ToC is constantly improvised as well, whenever a review or evaluation take place in this cohort. However, in this cohort, motivation of use of ToC is externally driven and programme monitoring systems are not based on a ToC, which also limit the rigour with which reviews and evaluations can be done.  
3. ToC is used perfunctorily: In this cohort, ToC may or may not exist. A ToC may be designed as some donor demanded it but not used thereafter. If there is not demand, a ToC may not exist. However, ToC may be developed when an evaluation is commissioned, generally by an evaluator. Programme or organisation still do not know about or see value in the ToC and consequently do not ‘own’ it.    

The situation will obviously change in the future as more and more agencies and programmes start gaining from use of a ToC. First and second cohort may increase therefore. However to speed up the change, I guess, two strategies might be useful:

1. Strengthen capacities for conceptualisation of programme theories of change: Demonstrating utility of ToC is incumbent upon how well a ToC is framed, capturing programme logic and realities of the context. Capacities to develop robust ToC can be strengthened in multiple way, one of which could be to initiate Evaluability Reviews (ERs) of programmes. ERs have potential to improve programme design and associated M&E systems.
2. Sensitise key constituents on ToC – As any change in thinking and working would require a ‘buy-in’, obviously ToC way of working would require that key constituents (donors, policy makers, organisations implementing programmes etc.) understand the need for ToC. Herein a community of practice such as Evalforward and others can continue to engage and facilitate conversations and demonstrate /showcase how use of a ToC can help in better programme design and in results-based management and may be other benefits as well.

Thanks and Best,

Ravi

Dear colleagues,

Thank you for your interesting insights, it is great to see an overall consensus on an important role that Theories of Change play in evaluation practice. Throughout my evaluation career, and especially more recently, I have come to appreciate the value-added of using TOC, especially if it has been co-created and/or deconstructed in a participatory manner.

Comparatively speaking, I have found TOC approach particularly valuable when evaluating cross-cutting themes, such as local stakeholder and civil society engagement, governance and gender. Even in the presence of documents, that guide related interventions (similar to sectors), their effective implementation should take into account, and mainstream civil society engagement, gender, accountability and transparency in the work of other sectors, teams, etc. Thus ToCs help inform the evaluations and facilitate exploration against the envisioned process and outcomes, against the existing framework and operational modalities, within and external to the organization.

In these cross-cutting domains and sometimes beyond, I also have found that teams that are being evaluated more often appreciate and welcome discussions of the TOC, including and sometimes with a particular appreciation of assumptions. Once posed with questions about feasibility within an enabling (or not) environment, the realization of why desired outcomes may have not been achieved becomes real. Consequently, having gone through TOC reconstructing, ambitions and targets are likely to become sharper and more streamlined next time around.

Regards,
Svetlana Negroustoueva

 

When I was just beginning my career in the development sector, I received a strange request from a donor: can you develop the logical framework of your programme? That was indeed a weird thing to ask three months before the end of the intervention. In hindsight I recognize that they were probably trying to cover their tracks after having skipped some steps in the approval process. At the time, with some grumbling, we complied with their request producing a half-baked logframe that was plausible, but not useful to anyone. It did not help with the planning, the management or the evaluation of the project. It was just a formal requirement, a box to be ticked. Now, let’s skip forward 20 years. I am contacted by a country office that asks for support. Their flagship programme is coming to an end and they want our help in preparing for an evaluation. I ask for some documents to understand what the programme is all about, but what I get is mostly donors reports that show the intervention through pink lenses. So, I suggest “let’s develop a theory of change!”.

No, I am not crazy and I have not become the bureaucrat that I dreaded 20 years ago (not quite at least). What I am suggesting is to sit down together and have the country office staff tell me their story so that we can piece it together to understand what they did, how and why. This is something that we do routinely in the East and Central Africa Region of the World Food Programme, where theories of change have become the bread and butter of any decentralized evaluation since 2018. The initial push to develop theories of change came from the Evaluation Unit at the Regional Bureau where we felt that having a solid understanding of how the programme works, available in an easy-to-share format, would provide external evaluators with a strong head start. Progressively, this exercise became more sophisticated and structured. We started paying more attention to assumptions as stress points where we can focus evaluation questions. We made the process more inclusive and participatory, involving external stakeholders. We added a problem tree analysis and a stakeholder mapping as propaedeutic steps before designing the theory of change. Finally, we decided to give it a name and we opted for “evaluability assessment”, as, among other purposes, we also use it as a tool to decide whether an evaluation is desirable and feasible.  

We have conducted a number of these exercises (14-15? I lost count…) and we never stop learning and improving our approach. In one case, we conducted an Evaluability Assessment together with a country office where management was caught in the crossfire between two donors who requested evaluations of their respective grants but did not budget adequately for the exercises. When we developed the theories of change of the two interventions, we started to see linkages between impact pathways that were not visible just comparing the respective logical frameworks. The two grants could be easily captured under one comprehensive theory of change that mirrored the reality of implementation on the ground where programmes merge into each other seamlessly. We then proposed to conduct one evaluation merging the two budget lines to look at the overall intervention, highlighting complementarities and synergies. Guess what? It worked!

One of the beautiful aspects of theories of change is that we do not need to develop them at a specific point in time. TOCs are useful at the beginning, in the middle or at the end of an intervention. Their main use and applications will be slightly different, but it is never too late to develop one. If you do it at the beginning, you can focus more on developing a shared understanding of how the programme works and pursue more active collaborations between different layers and components of the programme. It can inform the development of monitoring indicators, help in tracking risks and mitigation measures, not to mention the design of cross-sectional evaluations. Developed towards the end, it still has incredible value for informing the evaluation design and the key evaluation questions that you want answered.

One measure of the success of this approach is that now we receive requests from country offices for support in developing theories of change even when there is no evaluation in their plans. They see value in this exercise. They understand how it helps to formalize their collective thinking into an easy-to-grasp visual representation. No matter if the request comes in before the first bowl of porridge has been delivered or three months before the end of the programme, it is never too early or too late to design a theory of change.

Dear Colleagues,

Thanks for your very interesting contributions. If anyone could contribute with specific examples of ToC application, either in developing or developed countries, and highlight how this was useful for the programme under evaluation it would be great.

Best,

Carlos

Dear all,

I cannot agree more with Silva's comments, and if I may, I would like to add a few remarks.

I think it would be reasonable to maintain that the fundamental purpose of evaluation would be to ascertain whether an action, a project for instance, has succeeded in contributing to the quality of life of its target group.

This quality enhancement can be brought about by enabling them to satisfy one ore more of their fundamental needs. Nutrition is one of those needs. Its satisfaction obviously depends on the availability and affordability of wholesome local food, which in turn depends on the adequacy ofthe the local food system. The same applies to the set of needs associated with the other fundamental needs.

Often, the adequate satisfaction of a fundamental need requires the prior satisfaction of the needs associated with the satisfaction of some other fundamental need. For example, An adequate nutrition frequently depends on transport of food. Transport thus becomes a necessity not only for adequate nutrition, but also for other fundamental needs like education, health, security, etc.

So, I think a holistic empirical approach to evaluate/acertain the completeness of a project during planning stage would be critical for the success of any action. After all, we undertake a project to bring about a desired change, viz., to enhance the quality of a group's life in some way. This not theoretical, and it is eminently practical. Ideally, it ought to be pro-active, but there is too much below par today to think about that.

Target group is supposed to take over the running and maintenance of a project on its completion to some degree. They are not an isolate, but a part of a larger community. Thus, it is important to ascertain whether the target group and/or the community to which it belongs are able to continue the adequate operation of the project on its completion.

I think the variables involved here are just too numerous to be taken into account by any theoretical norm. Even after establishing its environmental and social impact with reference to its result and the tools a project proposes to use, and a community's willingness and ability to use them with sufficient skill, there are many more imponderables one will have to face. These have to be examined on the spot. 

Best wishes!

Lal

Recently I have been grappling with a similar set of questions so thank you, Carlos, for posing them. Drawing on my experience in facilitating several ToC workshops, I would say ToC is a useful approach to evaluation. Value is realized mainly in its process, of bringing participants from diverse disciplines and sectors together, of co-mapping systems change, of identifying areas where the program might influence change pathways, and of highlighting priority areas for monitoring. Some important context though, is that many of these participants have never heard of ToC before (and it doesn't help that ToC does not translate well in different languages), so some value might be attributed to its novelty. Anyway, while other planning tools might also have been appropriate, I find ToCs to be particularly helpful for programs that have multiple interacting components, diverse stakeholder perspectives, and uncertainty in outcomes, which are characteristic of many food security initiatives today.

Indeed, the use of the theory of change has come to improve planning and evaluation techniques for development projects. It took us from the logical framework (as a tool) and from Results-based Management (RBM) to Management for Development Results (MfDR) and the importance now attached to accountability. MfDR now makes it possible to focus mainly on the effects and impacts on the well-being of populations, instead of focusing just on outputs within the framework of RBM. The interest is now to focus on changes in the well-being of populations and to establish accountability, which is the obligation to account for the exercise of responsibility or the right for the beneficiaries of actions to claim and demand. The improvement in the well-being of populations is better measured with the use of the theory of change, so that the impact of projects is greater and more tangible. The use of the theory of change compels and directs efforts to respond to the growing demand for public accountability to citizens, in both developed and developing countries, so that they are informed about how aid is used, the results achieved and the extent to which these results bring about the desired changes to sustainable human development, rather than just development.

In my experience Theories of Change remains a mythical concept in the minds of several development practitioners at field level, first because it is perceived as a compliance driven tool forced by donors and funders and second it is usually developed by consultants or technocrats with little involvement of implementing staff.

Implementing staff usually takes on the role of understanding what is required to follow the ToC assumptions and rarely will you see the ToC in operation beyond siting in the Project Proposal document. The implementing teams rarely refers to ToC perhaps because there may be some difficulties in incorporating ToC concepts in day to day operations or it may be too complex for the field staff to engage with.

I therefore see a disconnect between the intended purpose of the ToC in guiding programming and impact and the realisation of the same in practice. Also, there is the One Size fit all approach to ToC presentation which I believe could be another challenge. On one hand we want the ToC to fit on one page, highly simplified and easy to conceptualise, almost too simplistic for the real world realities. But that is what makes it easier to digest and make sense of, which is great for policy makers and high level audiences. However, for implementers, detail matters so much but we often simply leave the ToC at that high level nice and glossy presentation and expect the implementers to work some form of magic to translate that into logical delivery of interventions according to the conceptual assumptions without the necessary detailed exploration and unpacking of the ToC. Donors do not request that, it is needed by implementors so it is often left undone and implementation of project goes on with little to no reference to the ToC.

The only time the ToC question will be revisited perhaps is when the project evaluation looks to test those assumptions assuming also that implementation was guided by those assumptions which we know is not always the case.

Dear members,

I am contributing to the new discussion launched by our colleague Carlos Tarazona.

Relying on my own experience, logical frameworks and result chains are planning tools that can help at the formulation phase of any result-oriented (or result-based) developmental action, albeit a policy, a programme, or a project. However, these tools require most of the time a strong technical expertise to use them in a rather professional manner and achieve a sound formulation of a given developmental action. Having that said, most development practitioners having no clue whatsoever on these planning and formulation tools may incur the risk of not understanding sufficiently the logical framework or the result chain of “their” developmental action.

Here comes then the use of “logic models” or “theories of change” especially during the implementation phase – and more importantly during the evaluation phase – of a given developmental action. In this case, members of the implementing team will sit together at the start of the implementation phase to “draw” a “logic model” or a “theory of change” in order to understand how the developmental action will evolve in its implementation area and how the “logical framework” or the “result chain” of that developmental action will unfold in reality in a series of cause-effect relations between its different elements, moving from “resources/inputs”, to “activities” to “outputs”, to “outcomes” and then to “impact”. Drawing the “logic model” or “theory of change” of a developmental action – either expressed in a drawing or in text – will help development practitioners in translating the “logical framework” or the “result chain” into a more expressive and easier way to unveil and understand the “change strategy” of that developmental action. The consequences of such an endeavour are: (1) a better understanding of the developmental action implementation strategy; (2)a lot of information for a better programming of the developmental action activities; and (3) the setup of a sound monitoring & evaluation system of that developmental action.

However, this is not the general case that a “logic model” or “theory of change” is made ready at the start of the implantation phase; some developmental actions have taken so much time during the formulation phase that the recipient agency would rush to start the implementation. And here comes the second situation of the use of “logic models” and “theories of change” at the evaluation phase. A sound evaluation exercise for a given developmental action would certainly rely on a “logic model” or “theory of change” that can help evaluators understand what that developmental action was supposed to do – at least in the heads of the formulating people – and compare it with what the developmental action did really. If a “logic model” or “theory of change” of a developmental action was drawn at the start of the implementation phase, it should be used and maybe improved on the condition that it is validated by the implementing team. If not, then the first task of the evaluators would be to elaborate a “logic model” or “theory of change” for the “evaluand” developmental action in order to define the different avenues that should be looked at during the evaluation exercise (parameters, indicators, data to be collected, etc.).

At the end, I would say that the debate should not be whether to use a “logical framework” or a “result chain”, on one side, or a “logic model” or a “theory of change”; the debate must be on the added value by using different methods and techniques to ensure a good implementation and a sound evaluation of a given developmental action. In brief, it is not a THIS OR THAT issue, but rather a THIS AND THAT one.

Hope this helps…

Mustapha

Mustapha Malki, PhD

Hi

The influx of ToCs has seen non-professionals in agriculture dominating the landscape of agriculture where qualifications and backgrounds that are not connected to agriculture decide and control agriculture. This is more worrying in the agricultural extension and technology dissemination arena where so called professional just gather people together to claim it is agricultural extension service delivery.

I hold thorough Technical knowledge (Agricultural qualification) sacrosanct in project management as well as monitoring and evaluation without which the project will never realise the full potential impact.

A question is that in the developed countries' sector of agriculture and food security as well as natural resource management, are there cacophony of ToCs noises like this to get them to where they are and sustaining them?

Dear Carlos and Dear members,

Thank you to receive my modest contribution.

What is your experience with the use of theories of change?

I have just done 1 year at FAO as a monitoring and evaluation officer under the GAFSP Missing Middle Initiative program. I have learned many things at you@FAO in the training of M&E guideline FAO. I think FAO has a solid experience in evaluation with many practical cases. if I'm not wrong the ToC is not a very popular tool.

In my previous experiences as a follow-up and evaluation officer, we used TOC for a smart project or program communication but also to share the project's intervention logic with stakeholders (partners, beneficiaries, etc.) in order to receive to promote the critical review focused on the key points of the project. It is a tool requested by several donors with different formats not far away, in the form of a diagram or table that always respects the same logic in the short term for direct results, medium term for effects and long term for impacts. The expected purpose would be to develop a resulting sentence i.e. the theory that in a simple text explains the project or program in complexity

What are the main added value of these theories, from your own perspective?

At FAO, I discovered in the project documents, the focus was more on the logical frameworks and results frameworks, which are also as relevant tools as the ToC but very complex for communication. ToC can be a tool that encompasses these other two tools while allowing a reading on the logic of intervention and the relevance of the activities or actions to be carried out. ToC traces the response from problems to resolutions. If it is a program the ToC is the best tool to define a vision but also to plan a participatory scaling up.

As part of GAFSP MMI Senegal, I had taken the initiative by drawing inspiration from GAFSP MMI's Global ToC to define a project-specific ToC that aligns with this global program while registering with PNUAD (National program of UN for development) and CPP 2019-2023. in the form of a table, the tool had facilitated the reading of the intervention and we also led to a revision of the logical framework with the integration of new results that take into account the gender dimension. shared with the partners, in a single sentence, we harmonized our understanding of the project's intervention logic. this tool had also led to the development of a M&E guideline that takes into account all the project's outcome dimensions

Have theories-of-change in your view made a difference in the programmes and projects that you have evaluated, especially when compared to other planning tools like log-frames and result chains?

Toc is a strategic tool for project or program evaluations. beyond the results defined in the logical or outcome framework, in an evaluation, the ToC allows contributions to be defined to macro indicators referring to broader policies or programs that include intervention. with Toc, we have often identified factors for successful intervention beyond the expected outcomes in relation to the baseline situation in the area of intervention. it is a tool that in an evaluation allows to see the cross-cutting themes and the chain justifying the contribution of the project

In my humble opinion, I think FAO needs to define a zero hunger change theory ( if not existing). This theory will draw inspiration from the representations to define the theories of their country programs and finally from these country program theories will draw inspiration from the theories of change specific to each project in the representation. This work will help to improve the logic of intervention and greatly facilitate evaluations at all levels. Like FPMIS for budget monitoring, such a portal could be developed to track changes.  All programs and projects can be included in this portal with an alignment of expected results and expected changes to the changes that define zero hunger in its globality.

It depends on what is a Theory of Change, and how it has been generated/shared.

If it remains the same as a big logframe, hidden in some proposals... it does not add much value.

If it is co-generated and owned... possibly EMERGING from the process of change, then it is an added value.

As an evaluator, I see that staff on the ground welcome discussions at theory of change level when they help to systematize experience.

But they might be clueless and confused by TOCs as proposal annexes.

So, if the Theory of Change is just bureaucracy it is actually a complication.

If it is a process of systematizing experience, owned by these involved in making change, it is super useful.

Unfortunately, the latter are very rare.