How can evaluation help improve data quality and policies on food security during Covid-19 pandemic?

Dear EvalForward members,

Thank you all for posting beneficial innovations and experiences on maintaining data quality during this period.

The experiences and lessons on how different actors and organisations are adapting their data collection plans are essential in maintaining the gains for utilising evidence to inform policy.

I am thrilled that we can continue to learn how to work around challenges while maintaining the credibility of the data we collect. The next step is to share this data as widely as possible and share our experiences as wide as possible to ensure that the region benefits significantly from policies informed using credible and reliable data.

Kindly keep sharing your experiences through this platform!

Kind regards,

Tim

Thanks very much Talent. Yes, the adaptation of the food insecurity experience to the COVID-19 pandemic is a great innovation especially as to attribute the insecurity that was already existing and that due to the pandemic. we look forward to the early results from these innovations. please share some of the experiences from what you are seeing so far.

Thank you very much Ethel for your contributions. I could not agree with you more. the utilisation of past data at this time is very useful. in addition, many organisations have made their past data available for utilisation. Analysis of the data, including being able to match and combine data is an essential skill to put into use at this time. Expert opinions are also important for contextual interpretation. There are a few models at this time such as those by IFPRI, EU, AfDB for analysis and evaluating scenarios, however, there is need to enhance capacity for modelling among evaluators.

Thanks a lot Tim. Very interesting exchanges. I do think there is a lot that can be done with existing data sets and secondary data, but much depends on their quality as well. A few thoughts come to mind

1. I realize from experience that quite often evaluators collect a lot more data than is required, so the current Covid- 19 situation forces us to streamline approaches and collect only very necessary data. We need to be more aligned to utility. For instance, ask ourselves, what information is required by decision makers therefore what is the minimum data required to address that at the time it is required? What data can be collected remotely and what can be supplemented through existing data?
2. Where good quality data exists, particularly with unique identifiers, efforts can be made to match data with light touch remote primary data collection, possibly using mobile technology, given travel restriction posed by Covid-19.
3. I do think more that ever before, enhanced quality measures at all levels are increasingly important. While we lose all the benefits that come with  face to face interaction in data collection, there is still much that can be done to enhance quality of outcomes- For instance, more needs to be done to engage people with the right technical expertise to mine existing data and understand trends; use existing data for modelling what the future situation might look like in view of Covid-19; and use qualitative approaches such as direct observation of crop quality and food availability in markets, where applicable to mention a few examples.

Lovely work you are doing, cannot wait to take a look at the data it will certainly be a wealth of information. On another note, I guess you are aware of the new FIES Survey Module adapted to the Covid Crisis? Its a very impressive module and even when adopted at a later stage in your study may bring additional information as it attempts to separate Food Insecurity due to the Covid-19 Crisis and habitual, engrained food insecurity due to persisting conditions at household and/or individual level. This is the module we are using in all the FAO led rapid assessments for comparability purposes. If this is something of interest, please do not hesitate to link up.

Dear Njagi,

Thank you very much for this link shared. The initiative is actually ongoing in Senegal the world Bank is in collaboration with the National office of Statistics to drive the evaluation between June to April 2021 and the FAO particularly the Statistics Division contribute by training the office members on the FIES tool adapted to Covid-19.

This type of evaluation are a macro level evaluation at national level and we take the results to be aligned neatly in national policies responses. However we need to have more specific rapid evaluation in the project areas with micro data in relation to the project goals and indicators. We are in a state for proposal for this type of specific rapid evaluation but our approach will be for a scope proposed by FAO for rapid assessment without the phone surveys.

Thank you for your feedback

Dear Diagne,

Thank you very much for sharing your feedback on rapid response. we look forward to learning more about your study and approach and experience from Senegal. On a related note, I came across high-frequency rapid response data collection being undertaken by the World Bank on the COVID-19 in a number of countries, see more information here http://surveys.worldbank.org/covid-19.

My Dears,

I’m joining this interesting discussion, occasion for us to share from the ongoing rapid evaluation assessment proposed by the Coordination Unit for the GAFSP MMI Senegal. Today in this pandemic situation, the rapid evaluation is an effective support for pertinent policies decision and programmatic action for most affected people. In December, in the baseline study, we had used the FIES (Food Insecurity Experience Scale) to evaluate the prevalence of food insecurity, at the time we had a database in FIES module for household and individual level. From this situation, we try to use evaluation with this tool to estimate the impact of the Covid 19 on food security and rural vulnerability. A comparative analysis from data FIES December 2019 to data FIES for June 2020 will give use an effective information on the impact of the covid 19 on food security. In parallel we will use the socio-economic tools to produce more evidence in the rural situation in this pandemic. Our data covers the intervention areas of the project. The Tambacounda region are defined like the more poor region in Senegal. Now from extrapolation the data can be used to estimate the actual situation of prevalence of food insecurity. The results will be used by the project to prepare the post pandemic planification and necessary action and too the more affected sub-areas where producers and households in general need more support.

This rapid evaluation exercise  are in process, so this is a short extract to explain in the urgency situation if you have baseline data how rapid evaluation with practice tool, can help to improve data quality and policies on food security....

I promise after the realization to produce a capitalization document if necessary to share with members.

Great suggestions Emile. The use of scientifically established samples greatly makes the data credible. The common practice of talking to a few people is biased and not credible. however, a key challenge remains to reach these respondents. Phone surveys are now the common approach, however, the desired information should be collected should be very short because it is not possible to keep a respondent for a long period over the phone. The other means you have proposed are ideal but may be unsuitable for people in rural areas without access to the internet and electricity. SMS is also a great option but the information collected through this method is also very short. in addition, incentives must be provided to enhance participation. This also needs careful consideration so as not to bias response.

The additional constraint of the COVID-19 pandemic is the inability to meet physically the actors who need to provide the information. They cannot be encountered at the risk of contamination. It is therefore necessary to contact them remotely, and then collect the required information and data depending on the specified periods. To this end, the database of the institutions usually in charge of collecting information and data on food security could be used. This database would provide a list of previously surveyed households and resource persons and their contacts. Once this list is obtained, it would be enough to launch the information and data collection excercise and invite the households on the list. The data collection could be launched electronically (e-mail) or through the press (newspapers and television). Households interests to participate in the data collection should be invited to contact the institution in charge of the activity in order tto contribute. Households and interested individuals could provide regular information and data by email or social networks (whatsapp, facebook, twitter, etc.). Automatically, the data collected will be reliable. As the data collected is reliable, it is possible to infer relevant calculations and analyses that can be used to define effective food security policies.

[post translated from French]

The Covid-19 pandemic has disrupted the norm in all sectors and professions. The lockdown and other measures to ensure health and safety of citizens have meant that collection of data, especially that which was collected through person to person interactions was not possible.  To the credit of many researchers, evaluators, data collectors, firms and institutions, there have been many and great innovations over the past two months to bridge the gap and ensure that there is data that can inform policy and decision-makers. However, it is important to ensure that the data that comes through is credible and reliable. Wrong, imprecise, incredible, unreliable data can undo the gains to have evidence-based policymaking. Also, the pressure to have data can lead to "bending" of procedures and protocols for data collection, analysis and inference. How then can we ensure the quality of the available data that is coming in? How can we evaluate the quality of analysis and subsequent inference to ensure that we influence the correct policy prescriptions?  How can we promote collaboration and lesson sharing during these times?