Thank you, Carlos, for bringing up another important topic in the forum and for the document links. I also thank the colleagues who shared their experience and comments.
Theory of Change, Logical Framework (Logframe) and result Chains are all methodologies for planning, measure and evaluate programs. Each has a visual representation in the form of a matrix of what happens or expected to happen as the result of the program or project or any initiative for that matter. As a past facilitator at IPDET (International Program for Developmental Evaluation training financed by the World Bank Program) and in my practice, I have found that Theory of Change is a name or title that is not easily identified as a methodology; it is a term that can bring dread in the mind ?. I would rather use Logic Model a term some use interchangeably to describe ToC. However, Logic model is also synonyms for Result Chains (https://www.betterevaluation.org/en/search/site/result%20chain) and Program Theory/Theory of Change https://www.betterevaluation.org/en/rainbow_framework/define/develop_programme_theory.
In my view, Theory of change is a more powerful tool than the two others mentioned since direct links between Activities, Output and Outcomes (Short and mid -erm and long-term often referred to as Impact) must be established and shown in the matrix. Furthermore, the ToC is not complete without assumptions. It is not a one-time matrix but must/can be reviewed and modified with time. During evaluation, the assumptions must be verified and if they do not hold or activities were modified during the course of the program, the matrix has to be reviewed accordingly.
The requirements of the ToC foster an in-depth reflection of what the program is trying to achieve. A difficulty often encountered is deciding what is an output and what is an outcome. I have found the Kellogg document a very useful Guide; it uses the term Logic model. https://www.bttop.org/sites/default/files/public/W.K.%20Kellogg%20LogicModel.pdf. Semantic is important in the developing the matrix and active verbs such as “Increased” that denote changes, help make the distinction between output and outcome.
I also find that filling the Activity column brings discussions that often show that stakeholders are not aware or had different views of what was actually happening during the program implementation. Follow-up discussions would often happen on whether the activities will trigger behaviour change among the program beneficiaries and have spill-over effect and results in the community or overall target population. I find that the ToC matrix facilitate the identification of indicators which are more meaningful for measuring performance and results.
I agree that the ToC must be developed in a participatory manner. However, at the time of evaluation, it may not exist or the one available is poor/confusing. In these cases, after an initial document review and discussions with the program staff, I will design the matrix or modify the existing one and circulate it. It helps me understand the program and formulate my requests for clarification. Since the matrix is simple to read (I like left to right), it usually receives attention and feedback. We end up with all involved having the same understanding of the program and its expected achievements. I have seen evaluation questions revised as the result of this exercise.
RE: How useful are theories of change in development programmes and projects?
Thank you, Carlos, for bringing up another important topic in the forum and for the document links. I also thank the colleagues who shared their experience and comments.
Theory of Change, Logical Framework (Logframe) and result Chains are all methodologies for planning, measure and evaluate programs. Each has a visual representation in the form of a matrix of what happens or expected to happen as the result of the program or project or any initiative for that matter. As a past facilitator at IPDET (International Program for Developmental Evaluation training financed by the World Bank Program) and in my practice, I have found that Theory of Change is a name or title that is not easily identified as a methodology; it is a term that can bring dread in the mind ?. I would rather use Logic Model a term some use interchangeably to describe ToC. However, Logic model is also synonyms for Result Chains (https://www.betterevaluation.org/en/search/site/result%20chain) and Program Theory/Theory of Change https://www.betterevaluation.org/en/rainbow_framework/define/develop_programme_theory.
In my view, Theory of change is a more powerful tool than the two others mentioned since direct links between Activities, Output and Outcomes (Short and mid -erm and long-term often referred to as Impact) must be established and shown in the matrix. Furthermore, the ToC is not complete without assumptions. It is not a one-time matrix but must/can be reviewed and modified with time. During evaluation, the assumptions must be verified and if they do not hold or activities were modified during the course of the program, the matrix has to be reviewed accordingly.
The requirements of the ToC foster an in-depth reflection of what the program is trying to achieve. A difficulty often encountered is deciding what is an output and what is an outcome. I have found the Kellogg document a very useful Guide; it uses the term Logic model. https://www.bttop.org/sites/default/files/public/W.K.%20Kellogg%20LogicModel.pdf. Semantic is important in the developing the matrix and active verbs such as “Increased” that denote changes, help make the distinction between output and outcome.
I also find that filling the Activity column brings discussions that often show that stakeholders are not aware or had different views of what was actually happening during the program implementation. Follow-up discussions would often happen on whether the activities will trigger behaviour change among the program beneficiaries and have spill-over effect and results in the community or overall target population. I find that the ToC matrix facilitate the identification of indicators which are more meaningful for measuring performance and results.
I agree that the ToC must be developed in a participatory manner. However, at the time of evaluation, it may not exist or the one available is poor/confusing. In these cases, after an initial document review and discussions with the program staff, I will design the matrix or modify the existing one and circulate it. It helps me understand the program and formulate my requests for clarification. Since the matrix is simple to read (I like left to right), it usually receives attention and feedback. We end up with all involved having the same understanding of the program and its expected achievements. I have seen evaluation questions revised as the result of this exercise.
It would be interesting to find out how many Ipdeters who are practising evaluation, utilize ToC in their work. ToC is at the core of the IPDET. See Road to Results, Morras and Rist 2009, the textbook for this training. https://openknowledge.worldbank.org/bitstream/handle/10986/2699/52678.pdf?sequence=1&isAllowed=y