What are your evaluation challenges?

@FAO/Eoghan Molloy

From the EvalForward community What are your evaluation challenges?

The latest discussion in the EvalForward CoP covered challenges that evaluations encounter in the agriculture, rural development and food security sectors and beyond. Here is a summary of the issues emerging and ways forward suggested.

Participants in the discussion identified the following challenges and possible solutions in conducting evaluations of agriculture, rural development and food security programmes:   

  • Lack of or limited availability of baseline data and information. This problem can be addressed by reconstructing baselines for example using “recall” techniques.
  • Gender aspects are often treated as an afterthought, with no or very limited data and indicators to measure progress.
  • Limited accessibility of project sites due to either remoteness or security threats: this affects the selection of evaluators and the methods to use. In a recent evaluation of an FAO programme in Afghanistan, the team used open source data from Google and Google earth maps to mitigate this constraint.
  • Time required in agriculture to produce productivity gains.
  • Adoption capacities of local communities.
  • Social stratification of rural communities.

Some of these challenges also apply in other sectors of work. However, as one participant noted, the assessment of results in agriculture, rural development and food security are inherently complex as they are affected by a multiplicity of biophysical, economic and social systems and factors.

Discussions also evoked other challenges applicable across sectors: 

  • Limited time and budget for monitoring and evaluation, which affect the usability of evaluation results to improve interventions.  
  • Poor design of projects and their results frameworks and indicators, which undermine the project / programme evaluability.
  • Disconnect between monitoring and evaluation, while both should be considered as the two legs of a system, from which development actions can learn.
  • Lack of an evaluation culture and enabling environment for evaluation: several participants    stressed that there is often confusion between evaluation, inspection, auditing and investigation, which tends to position many evaluands in a defensive mode, instead of understanding the possible benefits of evaluation for them. The tendency of some managers to identify themselves to the programme they manage, compounds this latter effect, and leads them to feel personally threatened in their professional credibility.
  • Other types of confusion were highlighted, between evaluations and other types of studies such as self-assessments, socio-economic analyses, baseline studies etc. 
  • Evaluation commissioners that do not clarify evaluation process and methodological approaches from the outset, via clear ToRs, lead to unrealistic expectations from evaluations. 
  • Some mentioned the risks related to evaluators’ self-censorship, in front of difficult political contexts or when facing interferences or pressures.

These more general challenges call for evaluators to adopt a good soft skills mix, such as communication, negotiation, people management and creativity.

Last, participants shared some suggestions for moving way forward:

  • Evaluation networks should agree on clear sets of recommendations to enhance evaluation culture and a common understanding of the M&E on both supply and demand sides.
  • Advocacy for establishing clear links between Monitoring and Evaluation functions should continue at all levels.
  • It is important to support the development of an evaluation culture through awareness raising, including by evaluators at the outset of the evaluation work, and promoting an enabling environment for evaluation.

Most certainly, the question of challenges and solutions to address them will continue to nurture discussions in the EvalForward Community of Practice. We will continue to facilitate the sharing of experiences and voices of practitioners to help advance the evaluation culture and knowledge.

This topic was raised by Hynda Krachni, and contributions were received by Serdar Bayryyev, Georgette Konate Traorè, Raymond Erik Zvavanyange, Isha Miranda, Mustapha Malki, Naser Qadous, Raoudha Jaouani.