What can we do to improve food security data?

I sincerely thank the participants and the various contributions to the debate launched since June 4, 2019 on "What can we do to improve food security data? ". The harvest was good because a number of deplorable national situations were reported and, fortunately, an interesting solution was revealed.

Indeed, from the Nema program in the Gambia, Paul Mendy informs that the ability of evaluation staff to collect and report on food and nutrition security is not up to the task. The same is true in India where Archana Sharma reports that most enumerators, surveyors and field workers not only work in poor conditions, but also live in adverse conditions, are not sufficiently problem-oriented and do not have access to any type of training in tools, techniques, methodologies, approaches and processes involved in data collection. These are poorly paid workers. The consequence is that investigators complete surveys based on their bias or expected investigation bias, as pointed out by Richard Tinsley. The result is that the data are biased and unreliable, but consistent with the country's financial situation. Tinsley suggested extrapolating from projects run by donor-assisted NGOs with a sufficient budget to manage a reliable survey, while Sharma found that the research or evaluation agency should invest proportionately in high-end surveyors and field staff for quality data collection.

There is therefore a need for our agriculture and food distribution policies to be based on common sense and for nutrition standards to be founded on the food needs of local people in line with their food culture (Lal Manavado). In the same vein, Kebba Ngumbo Sima warns that it is high time that attention be given to the context of local communities or indigenous peoples and their perceptions and understandings related to food security issues. However, because of the volume of work that the 2030 Agenda imposes on countries, the outcome will really depend on (a) the willingness of governments to invest in data collection, (b) the financial assistance that national statistical offices will be able to obtain from regional and international organizations (Filippo Gheri). Many factors must be taken into account when choosing the indicator to use to monitor food insecurity, as well as the type of data to be collected to obtain the indicator. An indicator should be easy to use, provide timely information, and be informed by data that is easy to collect (cost effective). It should also provide valid and reliable information. These characteristics are very difficult to find in indicators aimed at "measuring" food insecurity. For this reason, the Food Security and Nutrition Statistics Team of FAO launched the Voices of the Hungry project in 2013 (http://www.fao.org/in-action/voices-of-the-hungry/en/), which led to the development of a new tool called “Food Insecurity Experience Scale (FIES)", which has become indicator 2.1.2 of the Sustainable Development Goals. This new tool is according to Filippo Gheri direct, easy to use, low cost and statistically valid. It also helps to distinguish levels of severity, subdivide results and compare results across countries and over time. The FIES-based survey module has already been included in more than 50 nationally representative surveys around the world and another 60 countries have already planned to include it in their national surveys.

In short, the problems of reliability of food security data are everywhere in underdeveloped countries. This is due to the low investment of the states in this activity; which results in the use of agents of low quality. But, the FIES tool is an interesting solution to correct the situation, because it allows to significantly reduce the collection bias. The FIES tool is applied to the month, quarter or last 12 months; this allows to correct biases raised about changes in the food situation of households and individuals between seasons of the year. It is therefore imperative and urgent that many food security and development assessment specialists are trained in the effective use of this important tool for improving indicator data collection practices and improving the quality of the data for the indicators, and for the achievement by 2030 of SDG 2 in particular.

Dr Emile N. HOUNGBO

Dear Filippo,

thank you very much for sharing this easy to use and powerful tool. it can be used in all countries, it even adds value to official statistics in developed countries like Germany where the tool shows that among the poorest sectors of the populations prevails moderate and once a month severe food insecurity. 

Kind regards 

Ines  

Dear Emile, You are raising two extremely important issues. Regarding your second question, I think that the situation is heterogeneous. Weakness and challenges vary from country to country, but there are certain specificities that have lot to do with lack of financial and well trained (and incentivized) human resources, as you correctly pointed out in your blog. In countries were nationally statistical offices are weaker, the challenges are quite important, especially with respect to the amount of work that the Agenda 2030 is putting on them. According to me, the result will really depend on a) the possibility and willingness of the Governments in investing in data collection, b) the financial and technical assistance that national statistical offices will be able to attract from regional and international organizations. With respect to this second point and your first question, I would like to share with you my personal perspective as food security monitoring expert. I am working in the Food Security and Nutrition Statistics team, part of the Statistics Division of the Food and Agriculture Organization (FAO). The objective of the team is to provide technical assistance to countries in monitoring food security and nutrition, including two SDG indicators for which this team is the focal unit, namely the “Prevalence of Undernourishment (SDG 2.1.1)” and the “Prevalence of moderate or severe food insecurity in the population, based on the Food Insecurity Experience Scale (SDG 2.1.2)”. There are many considerations to be made when choosing which indicator to use for monitoring food insecurity, thus what kind of data to collect to derive the indicator. An indicator should be easy to use, provide timeliness information, be informed by data that are easy to collect (cost effective). But not only. As you rightly mentioned, it should provide valid and reliable information. These characteristics are very difficult to find in indicators aiming to “measure” food insecurity. This is the reason why the Food Security and Nutrition Statistics team of FAO in 2013 launched the Voices of the Hungry project (http://www.fao.org/in-action/voices-of-the-hungry/en/#.XRXngegzbcs) that brought to the development of a new tool called the Food Insecurity Experience Scale (FIES) (now SDG indicator 2.1.2). This new tool is according to me the best answer to your question. It is direct, easy to use, low cost, and statistically sound. It also allows distinguishing between severity levels, to disaggregate the results and compare results among countries and in time. But what if the indicator is informed by bad quality data? This is the main advantage of this tool, it is statistically sound! It allows performing a statistical validation of the collected data and – only if the information passes the validation – it can be used to derive estimates of prevalence of food insecurity in the population. This indicator is relatively new, but in less than two years the FIES module, 8 questions on access to food informing the tool, has been already included in more than 50 nationally representative surveys worldwide and 60 more countries have already plans in place to include it in their national surveys. This is according to me the demonstration of the power of this tool and an example on how we can monitor food security in SDG era producing valid, reliable, timeliness and cost effective information. Filippo

I think this is an excellent example of the consequences of over extending the financial resources of a host country. As I look at the overall economic conditions that are common to and effectively define developing countries, it is they represent financially suppressed economies. By this I mean economies that serve a generally impoverished population in which the bulk of the populations spend up to 80% of income or farm production on basic subsistence needs such as marginal food, clothing and shelter. While this can put tremendous downward pressure on consumer prices, there is also no discretionary funds to form a tax base that will fund civil services such as statistical analysis. Sorry, but no taxes; no services. Thus, governments are for all practically purposes financially stalled, barely meeting personnel financial obligations of salary plus benefits with virtually no operating funds for implementing programs, such as statistical surveys. This results in surveys being filled in by the enumerators in accordance with their biases or expected survey biases.  However, the results are accepted as accurate by the government with policy decisions being based on the results. Governments then heavily rely on external donor funded NGO to do much of the development activities. Such NGO often have their own objectives or are contracted to impose donor motivated solutions that may or may not be consistent with the needs of the beneficiaries such as smallholder farmers.  Please review the following webpages and links: https://smallholderagriculture.agsci.colostate.edu/financially-suppressed-economy-2/ and https://smallholderagriculture.agsci.colostate.edu/financially-stalled-governments/

Looking at the specific Benin survey case, I sense that being somewhat aware of the budget limits the organizers tried to compensate by making the survey more complex and more demanding on the enumerators. As an agronomist my discipline does not normally rely on survey information and are not fully familiar with survey technology. However, I am somewhat unusual for an agronomist and have often relied on survey information to get a fuller understanding of farming systems. Thus, I am familiar with some of the guidelines for conducting a quality survey. My normal guidelines are that a survey should take no more than an hour, an enumerator should conduct only 4 or at most 5 interviews per day. Thus, expecting them to do a 15-page survey would be very difficult in an hour and 10 per day virtually impossible. Thus, the enumerators will have to resort to group interviews that I tend to dismiss as having individual biases of outspoken minorities in them, or simply fill out the surveys on behalf of the interviewees. If you are expecting results from remote areas and not able to provide transportation, is there an alternative to having the enumerators fill out the surveys and still meet the limited deadlines for completing the survey? I don’t think so.

The result is the data is full of bias and unreliable, but consistent with the financial state of the country. It also means that the government is substantially out of touch with most of the populations. I recall hearing a few years ago that about 90% of smallholder farmers have never had any contact with agriculture officials and they are perhaps better off because of that.

A more specific case that I am personally knowledgeable of would be the Ruaha river basin in Southern Tanzania.  Here there were many complaints that the river was drying up earlier each year and endangering the wild life in the Ruaha National Park, which was blamed on climate change. According to the government there were 4 rice schemes above the National Park totally some 11,000 ha. However, when they got an aerial or satellite view the total rice acreage was some 40,000 ha. The difference was many small illegal informal schemes build by the farmers even including some permanent concrete abutments to divert the water.  Now what civil officers is going to wonder through the area on a rough road that will take at least 6 hours to fully transvers the 200 km and tell the poor smallholder farmers that must stop growing rice so foreign visitors can enjoy the wild life in the National Park. More likely they will visit the area, inform the farmers the rice cultivation is illegal, but with a modest gratuity allow them to continue until they return in 3 or 4 years. What government official will acknowledge this? Easier to simply continue to blame the river drying on climate change.

So, what can be done to get more reliable survey information? I don’t really know. I would suggest extrapolating from donor assisted NGO administered projects where they do have enough budget to manage a reliable survey. This would make for a patchwork of quality information across the country, but I would expect it to be fairly representative. Until the economy can expand to provide a reasonable tax base to finance civil services I fear this is the best means forward.

I hope this is helpful.

Thank you.

Dick Tinsley

Data Quality Management and Surveys

The General Policy for Data Quality Management in Agriculture provides for the development of a Code of Good Practice in Surveys for Harmonization of Approaches. during the design and production phases. Here, the term "survey" refers to any activity aimed at collecting or acquiring data for statistical purposes. This includes censuses, sample surveys, and the production of statistics from data from administrative records produced by agents. The creation and maintenance of an administrative file for statistical purposes does not fall into this category, only the exploitation of such a file for statistical purposes belongs to the field of surveys.

The collection of good data collection and analysis practices is one of the internal mechanisms contributing to the quality of processes, a prerequisite for the quality of products and services. Distributed to the staff of the practical guides while directing the responsibility since 2016 the GIZ initiated an institutional mechanism of collection of the data and on the other hand a measure of Audit (against verification) of the quality of the data collected by the team of responsible for the production of the data. However, GIZ's desire to reach out to a broader audience of customers, users and partners led to the development of the Policy Statement on Quality in Surveys. It consists of sections of the Code of Good Practice in Surveys whose content is less specialized.

The definition of quality

Like many statistical organizations, the Institute defines the quality of a product by all the characteristics that influence its capacity to satisfy a given need, to allow a planned use. GIZ uses six dimensions as criteria of quality: relevance, reliability and objectivity, comparability, timeliness, intelligibility and accessibility. Table 1 summarizes the definition and general guidelines for quality assurance for each of these dimensions. During the realization of a survey project, the dimensions are targets to be met to ensure the quality of the statistical information resulting from this survey. After its realization, they represent the criteria that make it possible to evaluate the quality of the statistical information produced. In the process of being implemented, the decisions made regarding the procedures and their implementation must take into account all the dimensions of quality. The quality assurance of a survey product therefore depends on the work done by the implementation team and all the staff involved, at all stages of the survey.

 Hi Emile,

How to measure food and nutrition security is a critical one which must be effectively addressed if we must adequately determine the effectiveness of the huge investments in this area and track the global efforts in achieving the respective SDG indicators.

As for us at the Nema Program in The Gambia we use three mechanisms, namely: 1: Number of hungry months in a year per household, 2: Amount of rural income and 3: Food and nutrition security scores provided by the National Nutrition Agency which publishes these data annually.

The number of hungry months is counted as the number of months in a year when farming households run out of stock of food they produced by themselves and have to resort to other means to acquire food for the family. In other words it is the number of months in the year when rural farming households have to buy rice from a shop. In The Gambia in a baseline study of 2013/14 revealed that this was at 5 months on average and the project target was to reduce this to two months by the project end.

Rural incomes are measured by the Bureau of Statistics and Planning and it compares the average annual incomes generated by rural households as compared to urban dwellers. Let me however state this one is particularly very difficult to define and as such we have tried as much as possible to avoid using this method.

The annual data produced by the National Nutrition Agency is promising one except that the agency needs to be well funded to conduct the survey in regular frequency. This has been a challenge over the years hence it’s not been entirely depended upon by projects.

In terms of general challenges food and nutrition data collection have cultural implications as Gambia rural dwellers hold this information dear to their heart, it relates directly to their prestige and so do not willingly give it out to. Aside from that other methods of measuring yield and productivity data are costly as it requires field presence of extension service personnel to ensure local secretaries record yield accurately. Extension coverage is very low in Gambia hence the data recorded by local community based secretaries can be subject to quality opinions. Then there is also the disharmony in food and nutrition data reporting requirements by the various players and donors. This does not help the situation either. And finally the capacity of evaluation staff to collect and report on food and nutrition security is not up to scratch in many developing countries, Gambia being no exception.

Looking forward to hearing your stories too!

Dear Emile

Answering to your questions is highly complex in terms of collecting data and collecting quality data.

See below my answers, which I hope will somewhat help you or that we can come together to improve the data collection.

• How can we monitor and evaluate efficiently progress towards the SDG2 – End Hunger if we cannot count on reliable data and consequent statistics and indicators?

All SDGs are complex, unable to be determined through direct data and most data vary from community to community.

• Do you think there are also weakness and  challenges in data collection in your country?

Yes we do have same issues. Departments of censers and statistics are not able to identify the data collection methodologies for SDGs. Practically it has been a nightmare to them.

Another issue is that we could be able to gather many quality data could through CSO's. But those data are not being sharing at national level and within the organization. Therefore, we are losing quality data and useful data.

Therefore, how do you address this issues?

Then it comes to "Big Data". How accurate and viable when you know that we are missing some data?

Dear Dr Emile N. Houngbo,

I am glad that you raised this important issue. The authenticity, credibility, reliability, accuracy, relevancy and right interpretation of data all are crucial not only to improve the quality of food security but also to address other development cooperation areas effectively as it feeds into decision-making processes. Unfortunately, in most of the underdeveloped and developing countries, poor quality of data poses a real problem and the projections made on the basis of these data are far from the reality.

Here, in India , data collection is a big challenge as most of our enumerators, surveyors and field agents not only work in poor working conditions, they live in adverse conditions, are not oriented to the issue appropriately and do not have access to any kind of training in tools, techniques, methodologies, approaches and processes involved in data collection. Most of our field staff are either NGO Workers already over burdened with other responsibilities in voluntary sector or recent graduates with no knowledge of the subject, no time to spend on the work and the low interest in data collection activity. In both the cases, they are lowly paid workers in the non-profit sector with no means of transportation to reach out to the remote areas and difficult terrains and badly lack basic training in application of the right tools and methodologies. Therefore, the Monitoring & Evaluation Frameworks, Key Performance Indicators, Assumptions, Project Findings, Recommendations and the End Product all may not be the representative of the real picture.

It is highly recommended that the research or evaluation agency invest in proportion in the high-end surveyors and the field staff for quality data collection as the agency invests in the desk job data analysts, data interpreters and the evaluators. Depending upon the complexity of data, the field staff should also undergo 1 to 2 days training before they embark upon the journey of data collection in the field.

Thanks,

Archana Sharma
Director                                                                                                

https://www.youtube.com/watch?v=yARyvU9ijAQ

Hi Lal,

I entirely agree with you!!! In fact Food Security - using who's lens or measurement??? This can really be contestable. It is high time we pay heed to large extent to the context of our local communities or indigenous people and their perceptions and understandings related to food security issues. We have carried away by scientific beliefs for too long now and we have not made much headway. It is high time we include local communities/indigenous people in our discourses on food security and poverty issues to inform policy since they are the most affected when it comes to such issues. They have been living and struggling for centuries with food security and poverty issues and thus are in much a better position to deep dive into these issues with more realistic information and confidence.

Just food for thought!!!

The debate continues!!!

Greetings!

I think an answer to Dr. Houngbo that reflects reality as it is, and not what we would like it to be, would sound very discouraging. I  believe that contrary to the common belief, it would be unwise to trust implicitely data on nutrition even from the affluent and technically advanced countries.

Perhaps, this question may underline with sufficient force the limitations one would have to face in policy formulation in general and that with respect to nutrition in particular. Consider the general methods in use to ascertain the adequacy of nutrition even in a small area.

1. Bio-metrics with reference to age, sex, etc.

1. Does one has a ‘valid’ baseline for comparison?
2. How long should one monitor to arrive at such a baseline?
3. What guarantee does one have that food intake of the participants could or would remain constant during the monitoring period?
4. In the absence of prior group specific values, how does one determine what would be an adequate diet for each test category during the monitoring period?

Well, I could describe some more difficulties, and this is only on establishing a baseline for comparison.

Then of course, we have the usual difficulties regarding transport, monitor competence, inadequate numbers, not to forget people’s willingness to participate.

Some have proposed a work-around or an indirect method, which would have been amusing had it not been put forth as a solution to an important problem. It is to monitor the consumption of various food items in an area. The untenability of this method is too obvious to be elaborated.

I know that I sound most discouraging. But, haven’t we perhaps placed too much trust in numbers because of their ‘seeming’ objectivity? After all, numbers are no more objective than any man-made symbol. Do we think something is automatically the way forward, because it looks like being scientific?

Our notion of science can totally mislead us just as any other belief system could. As soon as we say, X is f and that is an absolute fact with the firmness of any fanatic, out flies the science.

So, can we think of some other approach? It could use statistics for what it is worth as an adjunct, but let us base our policies on agriculture and food distribution on common sense and the applicable norms of nutrition. By applicable, I mean food needs of the local people in line with their food culture as much as possible. Then of course, the powers that be should do all they can to ensure that the facilities are available to local people to produce enough food, a fair distribution and most important making it available at affordable prices. A well-planned and sustained cooperative endeavor free from monopolies seems to be the only way forward, if none is to be left behind starving.

Value of food stems from it being essential to life, not because  it forms ‘value chains’ that enrich a host of intermediaries.

Best wishes!

Lal.