My contributions
However, evaluating the quality of STI is not straightforward for development organizations. Here, I would like to share my thoughts on how development organizations could meaningfully evaluate STI in their work.
The challenge
Many development organizations do not generate STI but gather and share STI knowledge. They provide platforms for knowledge exchange and disseminate the knowledge through knowledge products, such as publications and databases. Evaluating the technical rigour of headquarters-based knowledge products is relatively straightforward. However, focusing on the technical rigor of the headquarter-based knowledge products alone is not adequate in evaluating the STI in the development organizations. This is because
Posted by
Prior to the pandemic, traveling and face-to-face meetings with stakeholders allowed evaluation managers a degree of flexibility and control over evaluations. Since then, managers have had to adjust evaluation design to ensure remote oversight and quality control. I recently conducted a complex evaluation without travels or face-to-face meetings with stakeholders and evaluation team members. Since the restrictions are likely to continue, I would like to share my experience.
About the evaluation
The evaluation looked at FAO’s Technical Cooperation Programme. This programme allows FAO to draw from its regular programme resources and respond to the emergency needs of member countries. It
Posted by
I had two related questions before attending this workshop and learning exchange on users of outcome mapping and outcome harvesting:
1. What are the similarities and differences between the outcome-based approaches and other conventional approaches such as result-based and log-frame?
2. Can we use outcome-based approaches in evaluating complex development programmes?
I found that the outcome-based approaches are not very different from the other approaches at the concept level. However, there are some differences at the methodology level. I think this is because outcome-based approaches are adaptive approaches while conventional theory-based approaches are based-on deductive reasoning. More specifically:
The concept
Posted by
What is a Theory of Change?
I looked around the web to find out what definitions are used.
Below is my favourite, from betterevaluation.org:
"A theory of change explains how activities are understood to produce a series of results that contribute to achieving the final intended impacts. It can be developed for any level of intervention – an event, a project, a programme, a policy, a strategy or an organization".
At our office, a Theory of Change typically visualizes ‘how the project is supposed to work’.
Is our Theory of Change useful?
The final output of our evaluation exercise
Posted by
Nanae Yabuki
Evaluation Officer FAOThank you for the invaluable insights. The discussion has confirmed that a mix of qualitative and quantitative methods would be necessary to evaluate the Quality of Science (QoS) in the development context.
The bibliometric analysis could be useful in assessing the legitimacy of the research findings and the credibility of the knowledge products. FAO undertakes applied research and produces knowledge products such as publications. Bibliometric analysis can help assess the quality of our knowledge products.
FAO undertakes not only knowledge work but also field activities. We use the scientific evidence for enhancing the impact of the interventions, solving specific development problems etc. For assessing these aspects, ‘utility’ of the research findings is more relevant than ‘significance’ of research findings. Relevance of Science Technology and Innovation (STI) is context specific. How STI triggers a transformational change is also context specific. Therefore, we need to come up with appropriate criteria for each evaluation.
Though the standardization of evaluation criteria may not be practical, it would be worth making efforts to design the STI evaluations comparable at the institutional level for system-wide learning.
Various development organizations use STI in their work and evaluate it. I think we can learn from each other’s experiences.