Will Covid-19 spark change in the conduct of evaluations?  

©WFP

Evaluation in times of Covid-19 Will Covid-19 spark change in the conduct of evaluations?  

In the third and final part of our interview with leaders and managers currently engaged in humanitarian-development evaluations, we asked how the conduct of evaluations was evolving as a result of Covid-19, and what opportunities it might now usher in? (see part 1 here and part 2 here). 

Dr Robina Shaheen, Global Head of Monitoring, Evaluation and Learning Services, Action Against Hunger, UK : The conduct of evaluations will continue to change with the ongoing need for social distancing.  We are going to have to build in hygiene and social distancing messages and personal protection equipment requirements both for evaluators and respondents. This means new budget lines and expenses which evaluation commissioners will need to allow for. Evaluation managers at the commissioning agencies will need to manage their expectations about what is and what is not possible and this requires closer collaboration with in-country offices as well as evaluation service providers and consultants. There is a risk that we will fall into adopting double standards, with one rule for international evaluation staff and another for national in-country consultants and data collection agencies. In order to avoid this, evaluation commissioners will need to have clear protocols that evaluation service providers will need to adhere to and be accountable for.  

COVID-19 also presents opportunities for localizing evaluation work, working harder to identify, draw upon and more importantly trusting local expertise, rather than replacing them and building long term partnerships and relationships based on comparative advantage rather than sub-contracting relations. With more work localized, the carbon footprint would also potentially be reduced and budgets allocated for flights/DSAs of international evaluation staff could reduce evaluation costs and/or directed to providing for needs of in-country teams.  

Muriel Visser,Team leader for the evaluation of WFP school feeding contributions to the SDGs : Learning from experience about more and less effective ways of remote working will undoubtedly carry forward to post Covid-19 approaches. We hope that the experience of remote working will lead to a much more careful consideration of when air travel is really essential and that we will no longer see travel being required to participate in short meetings which can be conducted through alternative means.    

Gaby Duffy, WFP Senior Evaluation Officer : Many. We are experimenting with virtual webinar consultations with stakeholders at the end of the evaluation for example, instead of conducting a face-to-face country learning workshop.  This potentially could be a new way of working, particularly for countries where organizing a workshop is complex due to conflict, visa restrictions, etc. and it is certainly more cost-efficient.  The use of remote data collection tools may not necessarily be more cost-efficient, but it does push us to think more about our approach to engaged affected people in the evaluation process.  With these tools we can collect a broader range of data to capture voices from people and communities that are inaccessible, even if the teams could go in-country 

Martine Van de Velde, Leader of the team evaluating WFP’s country strategic plan in the Lebanon : The overall quality of the monitoring and evaluation (M&E) work done in-country is becoming even more important. Country offices need to be sufficiently equipped with resources to monitor, analyse and support the M&E capacity of national counterparts and cooperating partners. A joint approach among UN agencies working with same counterparts would be helpful for data collection and analysis—achieving efficiencies with resource allocation is a benefit of joint programming. A second important element is having evaluation teams comprised of nationals and internationals from the outset, especially in locations where there are language or connectivity barriers. This experience has also pushed us to consult with and learn more from other evaluations and expanding literature reviews.  

Marie-Hélène Adrien, Team leader of the evaluation of WFP Policy on South-South and Triangular Cooperation Policy : Covid-19 is showing that a lot of credible data can be collected remotely. This may reduce unnecessary travels. We found that we got more dedicated attention from senior staff through zoom calls than in-person. Evaluation may include greater mix of national evaluators (since international evaluators may not always be necessary).