Hi! I'm Natalia Kosheleva, independent evaluation consultant. I regularly do evaluations commissioned by country offices of UN agencies. These evaluations are decentralized, which means that they are managed by M&E offices or project/program staff members who don't have much experience with evaluation and rely on agency Evaluation Handbook to prepare evaluation ToRs.
Evaluation Handbooks use the common definition of outputs as deliverables/immediate results of the project/program activities and hence production of outputs is under direct control of the project staff. And outcomes are defined as results further down the change chain achieved with the use of outputs by other stakeholders, including target beneficiaries and government.
Based on these definitions, the evaluation questions recommended by Handbooks and consequently put in the ToR are usually formulated like:
* Have the outputs been delivered in a timely manner?
* To what extent the project/program outcomes were achieved?
But once I turn to the description of the evaluated project/program, I often find out that outcomes are not framed in a way that you can easily attribute them to the project.
From my experience, people who manage decentralized evaluations are not very open to questioning project outputs or outcomes. In one of my recent evaluations I resolved the situation by making an agreemet with members of the Evaluation Management Group and going through the full chain of events and changes created by the project.
- Did you experience the similar challenges due to differences in the use of terminology between people who planned the project and evaluators when doing evaluations?
- And if you did, how did you handle them?
Thanks in advance for sharing your experience!