How do we adapt our evaluation approach to the impact of the Covid-19 pandemic?
Tasked with leading a number of food-security related evaluations, I have been grappling with the challenge of how to adjust to the impact of the Covid-19 pandemic on my work. The challenges and solutions have varied depending on the stages at which the evaluations are.
Finalising and disseminating the evaluation results appear to be the easiest with dissemination events switched to online platforms, hoping that participants adapt to the technology.
More challenging has been the rapid adaptation of the approach for an evaluation in mid-process. Clearly, the ability to adapt will depend on the specific characteristics. In our case, a strategic evaluation supported through a number of country case studies, we adapted fairly smoothly, switching all interviews to a virtual format supported by technology – Skype, Zoom, WhatsApp or phone.
Although many of our interlocutors in countries were slower to respond when reached at the time of managing re-deployments required by country lockdown, we were able to reach a wide range of stakeholders, including Government, UN agencies, donors, civil society and others. Good forward planning of how to structure and conduct team interviews remotely and contextual knowledge and networks of the evaluation team have played a big part in the feasibility of utilising a virtual approach.
What is less clear is how the effects of the pandemic may influence the framing of the evaluation. Should we adjust the analysis and shape recommendations so that they remain relevant, given that the uncertain, but potentially major and long-lasting, impacts on development aid?
The choices in how to progress an incipient evaluation are even more drastic, and options have to be weighed between either proceeding virtually or waiting to launch the evaluation when the situation stabilises. Much depends on the context and urgency of the specific evaluation. Another deciding factor will be the extent of engagement required at community level. In our case empowering and supporting national consultants to a leading role is being explored as a way forward. There is a rich literature on conducting evaluations in inaccessible areas that I am looking to draw on and adapt.
I would be really interested to hear the experiences of others and their thoughts. How do we adapt our evaluation approach to this new reality in the short- and medium-term?