Using synthesis and meta-analysis to make the most of evaluative evidence: what is your experience?
In the past few years, there has been an increased availability of, and easiness of access by evaluators around the world to potential sources of evaluative evidence (evaluation reports, research papers, surveys) on a myriad of topics and contexts. Prospective users of such information, however, have become less capable to take full account of this development, in part due to lack of time and capacities to absorb all the information. Decision-makers in particular demand and expect from researchers and evaluators to provide them with products to facilitate the uptake of such knowledge. Evaluation synthesis and meta-analysis can fill this gap by building on findings from different sources to get a better understanding of the effectiveness of a program or policy.
In FAO we generally conduct meta-analysis as part of background research, prior to starting an evaluation. We also use them while carrying out evaluations to identify supplementary evidence and data for triangulation. In these cases the ultimate objective is to strengthen the evidence base of new/ongoing evaluations. When sufficient evidence exits, we prepare synthesis reports that are largely based on secondary information and are meant to immediately feed into programs and project improvement.
- What is your experience in conducting synthesis and meta-analysis?
- What were the objectives and the methods you used for undertaking them?
- What were the main benefits, challenges and lessons learnt?
- How can they be more used/applied in your country and/or organization?
For those of you interested, below are some links to examples of synthesis and meta-analyses carried out by FAO in the context of global and country level evaluations.
- “Evaluation of FAO’s programme in Mexico”
(http://www.fao.org/evaluation/evaluation-digest/evaluations-detail/en/c/1202316 – in Spanish): annex 2 outlines how meta-evaluations were used for better substantiating the assessment of FAO’s contributions to climate change adaptation, poverty reduction and food security in Mexico
- Evaluation of selected biodiversity projects in Ecuador (http://www.fao.org/3/CA1288ES/ca1288es.pdf - in Spanish): annex 2 outlines lessons learnt to support the improvement of ongoing/new biodiversity projects in Ecuador
- “Synthesis of findings and lessons learnt from the Strategic Objective Evaluations” (http://www.fao.org/evaluation/evaluation-digest/evaluations-detail/en/c/1184370/) and “Synthesis of lessons learnt in the application of the Country Programming Framework” (http://www.fao.org/evaluation/evaluation-digest/evaluations-detail/en/c/1128251/)
Many thanks in advance for your inputs!