I was the Director of the CGIAR Research on Roots, Tubers and Bananas until December 2021 and participated in and benefited from two rounds of CGIAR Research Programmes (CRP) reviews. I believe its really essential to build a stronger evaluation culture in CGIAR and learn from what we do.
Echoing some earlier contributions below, for a balanced evaluation its vitally important that the CGIAR sustains its investment into outcome case studies and in particular in impact studies. The latter have declined significantly in recent years in CGIAR and are not produced ordinarily as a part of routine projects or end of intervention surveys but need specialized methods and their own dedicated funding. Otherwise there is a risk with the ideas in the technical note that we use state of the art methods and increased precision of bibliometric analysis which is what we have available and always continues to be produced but we miss the rounded picture that these other studies provide. Its reminiscent of the the anecdote about someone looking under the lamp post for something they dropped when actually it fell elsewhere because this is where the light shines most strongly.
Its also possible to use social network analysis of publications, as one tool for qualitative analysis, to explore who collaborates to publish and what is their social and organizational context as a complement to bibliometric analysis, particularly relevant to the legitimacy dimension. We have a publication on this in the pipeline with experts on social network analysis at University of Florida.
Graham Thiele
International Potato CenterI was the Director of the CGIAR Research on Roots, Tubers and Bananas until December 2021 and participated in and benefited from two rounds of CGIAR Research Programmes (CRP) reviews. I believe its really essential to build a stronger evaluation culture in CGIAR and learn from what we do.
Echoing some earlier contributions below, for a balanced evaluation its vitally important that the CGIAR sustains its investment into outcome case studies and in particular in impact studies. The latter have declined significantly in recent years in CGIAR and are not produced ordinarily as a part of routine projects or end of intervention surveys but need specialized methods and their own dedicated funding. Otherwise there is a risk with the ideas in the technical note that we use state of the art methods and increased precision of bibliometric analysis which is what we have available and always continues to be produced but we miss the rounded picture that these other studies provide. Its reminiscent of the the anecdote about someone looking under the lamp post for something they dropped when actually it fell elsewhere because this is where the light shines most strongly.
Its also possible to use social network analysis of publications, as one tool for qualitative analysis, to explore who collaborates to publish and what is their social and organizational context as a complement to bibliometric analysis, particularly relevant to the legitimacy dimension. We have a publication on this in the pipeline with experts on social network analysis at University of Florida.