What can we do to improve food security data?

Je remercie sincèrement les participants et les diverses contributions au débat lancé depuis le 4 juin 2019 sur « What can we do to improve food security data ? ». La moisson a été bonne, parce que plusieurs situations nationales déplorables ont été rapportées et, heureusement, une solution intéressante a été révélée.

En effet, à partir du programme Nema en Gambie, Paul Mendy informe que la capacité du personnel d’évaluation à rassembler et à rendre compte de la sécurité alimentaire et nutritionnelle n’est pas à la hauteur de la tâche. Le même constat est fait en Inde où Archana Sharma rapporte que la plupart des recenseurs, enquêteurs et agents de terrain travaillent non seulement dans de mauvaises conditions, mais également ils vivent dans des conditions défavorables, ne sont pas suffisamment orientés vers le problème et n’ont accès à aucun type de formation aux outils, techniques, méthodologies, approches et processus impliqués dans la collecte de données. Il s’agit de travailleurs mal rémunérés. La conséquence est que les enquêteurs remplissent des enquêtes en fonction de leurs biais ou des biais d’enquête attendus (Richard Tinsley). Le résultat est que les données sont pleines de biais et peu fiables, mais cohérentes avec la situation financière du pays. Il suggérait alors d'extrapoler à partir de projets gérés par des ONG assistées par des donateurs et disposant d'un budget suffisant pour gérer une enquête fiable, pendant que Archana Sharma trouve que l'agence de recherche ou d'évaluation devrait investir proportionnellement dans les enquêteurs de haut de gamme et le personnel sur le terrain pour une collecte de données de qualité.

Il y a donc besoin que nos politiques en matière d’agriculture et de distribution alimentaire soient fondés sur le bon sens et les normes de nutrition soient fondées sur les besoins alimentaires des populations locales conformes à leur culture alimentaire (Lal Manavado). Dans la même logique, Kebba Ngumbo Sima alerte qu’il est grand temps qu’une attention soit accordée au contexte des communautés locales ou des peuples autochtones et à leurs perceptions et compréhensions liées aux questions de sécurité alimentaire. Or, à cause du volume de travail qu’impose l’Agenda 2030 aux pays, le résultat dépendra réellement a) de la possibilité et de la volonté des gouvernements d’investir dans la collecte de données, b) de l’assistance financière et technique que les bureaux nationaux de statistique pourront obtenir des organisations régionales et internationales (Filippo Gheri). De nombreux facteurs doivent être pris en compte lors du choix de l'indicateur à utiliser pour suivre l'insécurité alimentaire, ainsi que le type de données à collecter pour obtenir l'indicateur. Un indicateur doit être facile à utiliser, fournir des informations en temps voulu, être éclairé par des données faciles à collecter (rentables). Il devrait aussi fournir des informations valides et fiables. Ces caractéristiques sont très difficiles à trouver dans les indicateurs visant à «mesurer» l’insécurité alimentaire. C'est pourquoi l'équipe des statistiques de la sécurité alimentaire et de la nutrition de la FAO a lancé en 2013 le projet « Voices of the Hungry » (http://www.fao.org/in-action/voices-of-the-hungry/fr/#.XR8CpeQ8Too) qui a permis de mettre au point un nouvel outil appelé « Echelle de mesure de l'insécurité alimentaire vécue » ou « Echelle de mesure de l’insécurité alimentaire basée sur les expériences (FIES) », devenu l’indicateur 2.1.2 des objectifs de développement durable. Ce nouvel outil est selon Filippo Gheri direct, facile à utiliser, à faible coût et statistiquement valable. Cela permet également de distinguer les niveaux de gravité, de subdiviser les résultats et de comparer les résultats entre les pays et dans le temps. Le module de l’enquête fondée sur la FIES a déjà été inclus dans plus de 50 enquêtes représentatives au niveau national dans le monde et 60 autres pays ont déjà prévu de l’inclure dans leurs enquêtes nationales.

En somme, les problèmes de fiabilité des données sur la sécurité alimentaire se posent partout dans les pays sous-développés. Cela est dû au faible investissement des Etats dans cette activité ; ce qui se traduit par l’utilisation d’agents de peu de qualité. Mais, l’outil FIES est une solution intéressante pour corriger la situation, parce qu’il permet de réduire significativement les biais de collecte. L’outil FIES est appliqué tant sur le mois, le trimestre ou les 12 derniers mois ; ce qui corrigerait les biais soulevés au sujet des variations de la situation alimentaire des ménages et des individus entre les saisons de l’année. Il est donc indispensable et urgent que beaucoup de spécialistes de la sécurité alimentaire et de l’évaluation du développement soient formés à l’utilisation efficace de cet important outil en vue de l’amélioration des pratiques de collecte de données sur les indicateurs et de l’atteinte à l’horizon 2030 de l’ODD 2 notamment. 

Dr Emile N. HOUNGBO

Dear Filippo,

thank you very much for sharing this easy to use and powerful tool. it can be used in all countries, it even adds value to official statistics in developed countries like Germany where the tool shows that among the poorest sectors of the populations prevails moderate and once a month severe food insecurity. 

Kind regards 

Ines  

Dear Emile, You are raising two extremely important issues. Regarding your second question, I think that the situation is heterogeneous. Weakness and challenges vary from country to country, but there are certain specificities that have lot to do with lack of financial and well trained (and incentivized) human resources, as you correctly pointed out in your blog. In countries were nationally statistical offices are weaker, the challenges are quite important, especially with respect to the amount of work that the Agenda 2030 is putting on them. According to me, the result will really depend on a) the possibility and willingness of the Governments in investing in data collection, b) the financial and technical assistance that national statistical offices will be able to attract from regional and international organizations. With respect to this second point and your first question, I would like to share with you my personal perspective as food security monitoring expert. I am working in the Food Security and Nutrition Statistics team, part of the Statistics Division of the Food and Agriculture Organization (FAO). The objective of the team is to provide technical assistance to countries in monitoring food security and nutrition, including two SDG indicators for which this team is the focal unit, namely the “Prevalence of Undernourishment (SDG 2.1.1)” and the “Prevalence of moderate or severe food insecurity in the population, based on the Food Insecurity Experience Scale (SDG 2.1.2)”. There are many considerations to be made when choosing which indicator to use for monitoring food insecurity, thus what kind of data to collect to derive the indicator. An indicator should be easy to use, provide timeliness information, be informed by data that are easy to collect (cost effective). But not only. As you rightly mentioned, it should provide valid and reliable information. These characteristics are very difficult to find in indicators aiming to “measure” food insecurity. This is the reason why the Food Security and Nutrition Statistics team of FAO in 2013 launched the Voices of the Hungry project (http://www.fao.org/in-action/voices-of-the-hungry/en/#.XRXngegzbcs) that brought to the development of a new tool called the Food Insecurity Experience Scale (FIES) (now SDG indicator 2.1.2). This new tool is according to me the best answer to your question. It is direct, easy to use, low cost, and statistically sound. It also allows distinguishing between severity levels, to disaggregate the results and compare results among countries and in time. But what if the indicator is informed by bad quality data? This is the main advantage of this tool, it is statistically sound! It allows performing a statistical validation of the collected data and – only if the information passes the validation – it can be used to derive estimates of prevalence of food insecurity in the population. This indicator is relatively new, but in less than two years the FIES module, 8 questions on access to food informing the tool, has been already included in more than 50 nationally representative surveys worldwide and 60 more countries have already plans in place to include it in their national surveys. This is according to me the demonstration of the power of this tool and an example on how we can monitor food security in SDG era producing valid, reliable, timeliness and cost effective information. Filippo

I think this is an excellent example of the consequences of over extending the financial resources of a host country. As I look at the overall economic conditions that are common to and effectively define developing countries, it is they represent financially suppressed economies. By this I mean economies that serve a generally impoverished population in which the bulk of the populations spend up to 80% of income or farm production on basic subsistence needs such as marginal food, clothing and shelter. While this can put tremendous downward pressure on consumer prices, there is also no discretionary funds to form a tax base that will fund civil services such as statistical analysis. Sorry, but no taxes; no services. Thus, governments are for all practically purposes financially stalled, barely meeting personnel financial obligations of salary plus benefits with virtually no operating funds for implementing programs, such as statistical surveys. This results in surveys being filled in by the enumerators in accordance with their biases or expected survey biases.  However, the results are accepted as accurate by the government with policy decisions being based on the results. Governments then heavily rely on external donor funded NGO to do much of the development activities. Such NGO often have their own objectives or are contracted to impose donor motivated solutions that may or may not be consistent with the needs of the beneficiaries such as smallholder farmers.  Please review the following webpages and links: https://smallholderagriculture.agsci.colostate.edu/financially-suppressed-economy-2/ and https://smallholderagriculture.agsci.colostate.edu/financially-stalled-governments/

Looking at the specific Benin survey case, I sense that being somewhat aware of the budget limits the organizers tried to compensate by making the survey more complex and more demanding on the enumerators. As an agronomist my discipline does not normally rely on survey information and are not fully familiar with survey technology. However, I am somewhat unusual for an agronomist and have often relied on survey information to get a fuller understanding of farming systems. Thus, I am familiar with some of the guidelines for conducting a quality survey. My normal guidelines are that a survey should take no more than an hour, an enumerator should conduct only 4 or at most 5 interviews per day. Thus, expecting them to do a 15-page survey would be very difficult in an hour and 10 per day virtually impossible. Thus, the enumerators will have to resort to group interviews that I tend to dismiss as having individual biases of outspoken minorities in them, or simply fill out the surveys on behalf of the interviewees. If you are expecting results from remote areas and not able to provide transportation, is there an alternative to having the enumerators fill out the surveys and still meet the limited deadlines for completing the survey? I don’t think so.

The result is the data is full of bias and unreliable, but consistent with the financial state of the country. It also means that the government is substantially out of touch with most of the populations. I recall hearing a few years ago that about 90% of smallholder farmers have never had any contact with agriculture officials and they are perhaps better off because of that.

A more specific case that I am personally knowledgeable of would be the Ruaha river basin in Southern Tanzania.  Here there were many complaints that the river was drying up earlier each year and endangering the wild life in the Ruaha National Park, which was blamed on climate change. According to the government there were 4 rice schemes above the National Park totally some 11,000 ha. However, when they got an aerial or satellite view the total rice acreage was some 40,000 ha. The difference was many small illegal informal schemes build by the farmers even including some permanent concrete abutments to divert the water.  Now what civil officers is going to wonder through the area on a rough road that will take at least 6 hours to fully transvers the 200 km and tell the poor smallholder farmers that must stop growing rice so foreign visitors can enjoy the wild life in the National Park. More likely they will visit the area, inform the farmers the rice cultivation is illegal, but with a modest gratuity allow them to continue until they return in 3 or 4 years. What government official will acknowledge this? Easier to simply continue to blame the river drying on climate change.

So, what can be done to get more reliable survey information? I don’t really know. I would suggest extrapolating from donor assisted NGO administered projects where they do have enough budget to manage a reliable survey. This would make for a patchwork of quality information across the country, but I would expect it to be fairly representative. Until the economy can expand to provide a reasonable tax base to finance civil services I fear this is the best means forward.

I hope this is helpful.

Thank you.

Dick Tinsley

La gestion de la qualité des données et des enquêtes 

La Politique générale en matière de gestion de la qualité des données en matière de l’agriculture prévoit l’élaboration d’un Recueil de bonnes pratiques dans les enquêtes permettant l’harmonisation des façons de faire lors des phases de conception et de réalisation. On entend ici par « enquête » toute activité visant à recueillir ou à acquérir des données à des fins statistiques. Cela inclut les recensements, les enquêtes par sondage et la production de statistiques à partir de données provenant de dossiers administratifs produit par les agents. La constitution et le maintien d’un fichier administratif à des fins statistiques n’entrent pas dans cette catégorie, seule l’exploitation d’un tel fichier à des fins statistiques appartient au domaine des enquêtes.

Le Recueil de bonnes pratiques de collecte et d’analyse des données constituent un des mécanismes internes concourant à la qualité des processus, condition indispensable à la qualité des produits et services. Diffusé au personnel des guides pratiques tout en orientant la responsabilité depuis 2016 la GIZ a initié un mécanisme institutionnel de collecte des données et d’un autre côté une mesure d’Audit (contre vérification) de la qualité des données collectées par l’équipe de travail responsable de la production des données.

Cependant, le désir de la GIZ de s’adresser à un plus large public formé de clients, utilisateurs et partenaires a mené à la conception du Document de principes sur la qualité dans les enquêtes. Celui-ci est constitué des sections du Recueil de bonnes pratiques dans les enquêtes dont le contenu est moins spécialisé. 

La définition de la qualité 

Comme de nombreux organismes statistiques, l’Institut définit la qualité d’un produit par l’ensemble des caractéristiques qui influent sur sa capacité à satisfaire un besoin donné, à permettre un usage prévu. La GIZ retient six dimensions comme critères de la qualité : la pertinence, la fiabilité et l’objectivité, la comparabilité, l’actualité, l’intelligibilité et l’accessibilité. Le tableau 1 résume la définition et les orientations générales d’assurance de la qualité pour chacune de ces dimensions.

Pendant la réalisation d’un projet d’enquête, les dimensions constituent des cibles à atteindre pour assurer la qualité de l’information statistique issue de cette enquête. Après sa réalisation, elles représentent les critères qui permettent d’évaluer la qualité de l’information statistique produite. En cours de réalisation, les décisions prises en ce qui a trait aux façons de faire et à leur mise en œuvre doivent tenir compte de toutes les dimensions de la qualité. L’assurance de la qualité d’un produit d’enquête dépend donc du travail effectué par l’équipe de réalisation et par l’ensemble du personnel impliqué, et ce, à toutes les étapes de réalisation de l’enquête.

 Hi Emile,

How to measure food and nutrition security is a critical one which must be effectively addressed if we must adequately determine the effectiveness of the huge investments in this area and track the global efforts in achieving the respective SDG indicators.

As for us at the Nema Program in The Gambia we use three mechanisms, namely: 1: Number of hungry months in a year per household, 2: Amount of rural income and 3: Food and nutrition security scores provided by the National Nutrition Agency which publishes these data annually.

The number of hungry months is counted as the number of months in a year when farming households run out of stock of food they produced by themselves and have to resort to other means to acquire food for the family. In other words it is the number of months in the year when rural farming households have to buy rice from a shop. In The Gambia in a baseline study of 2013/14 revealed that this was at 5 months on average and the project target was to reduce this to two months by the project end.

Rural incomes are measured by the Bureau of Statistics and Planning and it compares the average annual incomes generated by rural households as compared to urban dwellers. Let me however state this one is particularly very difficult to define and as such we have tried as much as possible to avoid using this method.

The annual data produced by the National Nutrition Agency is promising one except that the agency needs to be well funded to conduct the survey in regular frequency. This has been a challenge over the years hence it’s not been entirely depended upon by projects.

In terms of general challenges food and nutrition data collection have cultural implications as Gambia rural dwellers hold this information dear to their heart, it relates directly to their prestige and so do not willingly give it out to. Aside from that other methods of measuring yield and productivity data are costly as it requires field presence of extension service personnel to ensure local secretaries record yield accurately. Extension coverage is very low in Gambia hence the data recorded by local community based secretaries can be subject to quality opinions. Then there is also the disharmony in food and nutrition data reporting requirements by the various players and donors. This does not help the situation either. And finally the capacity of evaluation staff to collect and report on food and nutrition security is not up to scratch in many developing countries, Gambia being no exception.

Looking forward to hearing your stories too!

Dear Emile

Answering to your questions is highly complex in terms of collecting data and collecting quality data.

See below my answers, which I hope will somewhat help you or that we can come together to improve the data collection.

• How can we monitor and evaluate efficiently progress towards the SDG2 – End Hunger if we cannot count on reliable data and consequent statistics and indicators?

All SDGs are complex, unable to be determined through direct data and most data vary from community to community.

• Do you think there are also weakness and  challenges in data collection in your country?

Yes we do have same issues. Departments of censers and statistics are not able to identify the data collection methodologies for SDGs. Practically it has been a nightmare to them.

Another issue is that we could be able to gather many quality data could through CSO's. But those data are not being sharing at national level and within the organization. Therefore, we are losing quality data and useful data.

Therefore, how do you address this issues?

Then it comes to "Big Data". How accurate and viable when you know that we are missing some data?

Dear Dr Emile N. Houngbo,

I am glad that you raised this important issue. The authenticity, credibility, reliability, accuracy, relevancy and right interpretation of data all are crucial not only to improve the quality of food security but also to address other development cooperation areas effectively as it feeds into decision-making processes. Unfortunately, in most of the underdeveloped and developing countries, poor quality of data poses a real problem and the projections made on the basis of these data are far from the reality.

Here, in India , data collection is a big challenge as most of our enumerators, surveyors and field agents not only work in poor working conditions, they live in adverse conditions, are not oriented to the issue appropriately and do not have access to any kind of training in tools, techniques, methodologies, approaches and processes involved in data collection. Most of our field staff are either NGO Workers already over burdened with other responsibilities in voluntary sector or recent graduates with no knowledge of the subject, no time to spend on the work and the low interest in data collection activity. In both the cases, they are lowly paid workers in the non-profit sector with no means of transportation to reach out to the remote areas and difficult terrains and badly lack basic training in application of the right tools and methodologies. Therefore, the Monitoring & Evaluation Frameworks, Key Performance Indicators, Assumptions, Project Findings, Recommendations and the End Product all may not be the representative of the real picture.

It is highly recommended that the research or evaluation agency invest in proportion in the high-end surveyors and the field staff for quality data collection as the agency invests in the desk job data analysts, data interpreters and the evaluators. Depending upon the complexity of data, the field staff should also undergo 1 to 2 days training before they embark upon the journey of data collection in the field.

Thanks,

Archana Sharma
Director                                                                                                

https://www.youtube.com/watch?v=yARyvU9ijAQ

Hi Lal,

I entirely agree with you!!! In fact Food Security - using who's lens or measurement??? This can really be contestable. It is high time we pay heed to large extent to the context of our local communities or indigenous people and their perceptions and understandings related to food security issues. We have carried away by scientific beliefs for too long now and we have not made much headway. It is high time we include local communities/indigenous people in our discourses on food security and poverty issues to inform policy since they are the most affected when it comes to such issues. They have been living and struggling for centuries with food security and poverty issues and thus are in much a better position to deep dive into these issues with more realistic information and confidence.

Just food for thought!!!

The debate continues!!!

Greetings!

I think an answer to Dr. Houngbo that reflects reality as it is, and not what we would like it to be, would sound very discouraging. I  believe that contrary to the common belief, it would be unwise to trust implicitely data on nutrition even from the affluent and technically advanced countries.

Perhaps, this question may underline with sufficient force the limitations one would have to face in policy formulation in general and that with respect to nutrition in particular. Consider the general methods in use to ascertain the adequacy of nutrition even in a small area.

1. Bio-metrics with reference to age, sex, etc.

1. Does one has a ‘valid’ baseline for comparison?
2. How long should one monitor to arrive at such a baseline?
3. What guarantee does one have that food intake of the participants could or would remain constant during the monitoring period?
4. In the absence of prior group specific values, how does one determine what would be an adequate diet for each test category during the monitoring period?

Well, I could describe some more difficulties, and this is only on establishing a baseline for comparison.

Then of course, we have the usual difficulties regarding transport, monitor competence, inadequate numbers, not to forget people’s willingness to participate.

Some have proposed a work-around or an indirect method, which would have been amusing had it not been put forth as a solution to an important problem. It is to monitor the consumption of various food items in an area. The untenability of this method is too obvious to be elaborated.

I know that I sound most discouraging. But, haven’t we perhaps placed too much trust in numbers because of their ‘seeming’ objectivity? After all, numbers are no more objective than any man-made symbol. Do we think something is automatically the way forward, because it looks like being scientific?

Our notion of science can totally mislead us just as any other belief system could. As soon as we say, X is f and that is an absolute fact with the firmness of any fanatic, out flies the science.

So, can we think of some other approach? It could use statistics for what it is worth as an adjunct, but let us base our policies on agriculture and food distribution on common sense and the applicable norms of nutrition. By applicable, I mean food needs of the local people in line with their food culture as much as possible. Then of course, the powers that be should do all they can to ensure that the facilities are available to local people to produce enough food, a fair distribution and most important making it available at affordable prices. A well-planned and sustained cooperative endeavor free from monopolies seems to be the only way forward, if none is to be left behind starving.

Value of food stems from it being essential to life, not because  it forms ‘value chains’ that enrich a host of intermediaries.

Best wishes!

Lal.