Global development indicators are not just numbers

The theme of this year's report is "Transforming food systems for affordable healthy diets". The SOFI 2020 report examines the cost of healthy diets around the world, by region and in different development contexts. Food quality is an important factor in food security.

The report uses a number of indicators, and I would like to take this opportunity to talk about the political and cultural dimensions of development indicators, by briefly analyzing two indicators related to the SDG target 2.1 which aims to eradicate hunger, and which (among others) are used in the SOFI report.

The idea that hunger is a political problem is not recent: Josué de Castro (Brazil) already pointed this out in Geopolitics of Hunger (1951), and Amartya Sen in Poverty and Famines: An Essay on Entitlement and Deprivation (1981). In the era of the Sustainable Development Goals (SDGs), global development indicators have become very important and visible. As a result, they have become much more than mere socio-economic variables, neutral and disembodied. They often demonstrate political will, while offering new ways of apprehending and understanding development problems.

The first indicator for SDG target 2.1 is the prevalence of undernourishment (PoU, indicator 2.1.1). It is an estimate of the number of people in a given country suffering from an inadequate caloric intake. This indicator dates back to the 1970s, and has been used to monitor progress towards MDG1 - "eradicate hunger and extreme poverty". FAO and its member countries calculate it annually for each country by estimating the amount of food available at the national level. This estimate of available food is then socially distributed, according to social inequality ratios such as the GINI, to determine the proportion of the population that may not have access to an adequate caloric intake.

The viewpoint conveyed by the PoU indicator is therefore that of a macro-economist, who considers that a nation must produce and purchase cereals in sufficient quantities to feed its population. This is roughly how the problem of hunger was understood in the 1970s: in terms of the amount of wheat, rice or maize available to the population.

The PoU has been the subject of much criticism, which has pointed out that the empirical basis for some of the variables in the model is fragile. The shortcomings of the PoU indicator are recognized by FAO, which has regularly improved both the calculations and the data on which they are based.

The second indicator of the same target is much more recent. It is indicator 2.1.2 - Prevalence of moderate or severe food insecurity, based on the Food Insecurity Experience Scale (FIES). The FIES is an eight-question scale that can be easily inserted into pre-existing surveys such as demographic and health surveys or World Bank living standards studies. These questions focus on the ability to access healthy and varied food. Two indicators are derived from them: one on severe food insecurity and the other on moderate food insecurity.

FAO is the "custodian" of the methodology for these indicators, i.e. it is responsible for developing the methodology and training the national teams that collect and analyze the data within each country. Indeed, in the era of SDGs, the relevant national authorities (such as National Statistics Offices) are responsible for the collection and analysis of development data.

The FIES methodology is very different from that of the PoU. Instead of directly measuring the amount of food or income people have, they are asked to talk about their own experiences through a survey, with questions such as: "Have you had to reduce your food consumption or settle for cheap food? Have you had to skip meals?" etc. This experience-based indicator therefore conveys a different concern, which can be summarized as follows: "It is important to consult the population of a country on its development problems."

The FIES had a difficult start due to:

• A criticized introduction process. In order to test the FIES’s robustness in different cultures, languages and economic conditions, and to ensure the availability of data until national statistical offices are able to collect them, FAO commissioned a private company (Gallup World Poll) to apply the FIES survey module to a globally representative sample of 150 countries, during a period from 2014 to 2016. This allowed the validation of the methodology and the inclusion of the two FIES indicators on the SDG indicators "menu" in 2017, but this approach was subsequently criticized because in the spirit of the SDGs, it is the member countries themselves that have to collect the data. Some countries requested that the data collected by FAO and Gallup should not be disseminated, and that they should be removed from the official UN website on SDG indicators.
• Local adaptation of the scale of questions. Some countries such as Brazil or the United States already have their own national scale with a larger number of questions, while other countries, such as China, have preferred to use a shorter question scale.
• The translation of the FIES questions into more than 100 national languages. The nuances introduced almost necessarily by any translation could generate significant biases in the data, especially if the translation is done quickly.
• Last but not least, the FIES gives a voice to society. It treats respondents as subjects whose experiences are important, rather than as mere objects of study. This positive feature can also have a drawback, by introducing some subjectivity into the collected data. Because it is based on experience, the FIES may inherently be more subjective than direct anthropometric measurements, as implied in stunting indicators or estimates of national staple food availability (PoU).

The FIES and the PoU are closely correlated but some anomalies have emerged (SOFI 2017). Some countries in Eastern and Southern Africa had very high FIES values when compared to their PoU. This could indicate the FIES’s improved ability to reflect in real time the impact of three years of drought related to the El Nino effect. Other countries displayed the opposite gap: a low prevalence of food insecurity as measured by the FIES combined with a high prevalence of undernourishment as measured by the PoU. According to the authors of the SOFI 2017, this could be explained by a trend of food difficulties under-reporting in some countries.

In particular, the FIES’s sampling approach does not necessarily work in countries where freedom of expression is not guaranteed.

Some observers question whether an experience-based indicator such as the FIES qualifies as a global indicator, or whether such an indicator is loaded with cultural subjectivity and local biases that data cannot be accurately compared across countries.

In 2018, an evaluation confirmed that the FIES is a robust tool for measuring the access to food, and that it has some methodological advantages, as well as disadvantages, compared to other indicators. The ease of collection represents a great advantage within the SDG monitoring framework, which is complex and can be a significant financial burden for developing countries. The FIES data is also easy to disaggregate by locality, gender or social group, whereas the PoU has no capacity to disaggregate at national level. This ability to disaggregate is an important advantage of the FIES in identifying the geographic areas and social groups most at risk, and thus "leaving no one behind", one of the fundamental principles of the SDGs. Beyond the FIES, the evaluation underlines that the SDG monitoring framework includes many other indicators collected through opinion surveys.

For all these reasons, FAO has continued to develop and use the indicators that derive from the FIES, despite the opposition of some member countries of the organization. I think that this is its role - to circulate ideas and tools across borders, without neglecting local particularities and refraining from any tendency towards cultural hegemony.

In conclusion, no food security indicator will ever be perfect, hence the value of a variety of indicators that complement each other when carefully triangulated and interpreted. If, in the case of the FIES, the difficulties of introduction seem to be largely behind us today, there might always be unexpected twists and turns, especially if the figures worsen, which seems to be the case today…