Proving the value of agroecology for farmers and food systems: what methods and evidence do we have?

Dear all

I am finding Pablo Tittonell's new book, A Systems Approach to Agroecology (July 2023), enormously helpful and insightful on many fronts.

best, John

Dear participants, 

Thank you all for your contributions. Here are my responses and I look forward to hearing more from you and members of the EvalForward community insights and experiences. 

Jillian, thanks for sharing this highly relevant paper as it analyses and summarises the development in the field of measuring agroecological transitions at farm /household and landscape /food systems level. The paper is a ‘must’ read for those working on the intersection of implementing, researching and measuring agroecology impacts. And as the paper says, “there will never be a perfect tool or framework for assessing agroecology that can meet every objective in all possible contexts”, therefore we need to discuss and debate differing perspectives and experiences around the key questions of ongoing methodological experiments /innovations in different context (including measuring agroecology at landscape /food system level which the paper found to be less prevalent) but also any demonstrative empirical evidence that prove /disprove the worth of agroecology. Will be great to hear such perspectives /experiences from EvalForward community. 

Dushyant, thanks for providing this idea on possibilities of using satellite data at village /farm level to track change in agroecological transitions at farm level. I am intrigued and would be great if you can share an example where this was done at this scale (farm /village). This would be hugely beneficial for programmers /researchers /M&E professionals to understand where this has been achieved and how this approach can be applied to track agroecological transitions. 

Dario, thanks for summarising utility of TAPE in understanding agroecology transitions and in creating demonstrative evidence on agroecology contribution to poverty, human health and environment. In Nutrition Research Facility project, we have taken considerable inspirations from TAPE in developing our methodology for assessment of agroecology interventions in the context of an EU programme in Madagascar. This is a quasi-experimental (difference in difference approach) research for which baseline was conducted in 2022 and we intend to carry out the endline research in 2024/5 to see the effects of agroecology interventions. Apart from a household survey (n=1695), we have deployed a qualitative approach to understand all the factors that hamper or promote agroecology at farm or food systems level. These factors in the Malagasy context are – insecure land tenure status, land fragmentation and conflicts, shifting pattern of agriculture production, low quality and high costs of agriculture inputs (seeds, agrochemicals), insecurity and theft of crop and livestock, limited collectivisation and negotiating power of producers, limited storage solutions, scarcity of manure, limited financial linkages and indebtedness of producers, low women empowerment status (agency, opportunities, and outcomes). These and many other challenges encountered by the producers limit their ability to apply agroecology principles and practices. One of our key insights from this research – for agroecology to achieve poverty, human health and environmental outcomes, constraints to its adoption would have to be resolved. This would require understanding of and finding solutions to context-specific challenges. The question is whether the agroecology programmes are designed in flexible and holistic ways to understand and address these context-specific challenges?   

Ram, thanks for very useful inputs to this ongoing debate /discussions. Community scorecards is an excellent idea, which the FAO’s TAPE methodology also incorporate. In our research in Madagascar, we have used community score card methodology in focus group discussions on several elements of agroecology such as resilience, synergy, farm workers welfare and rights etc. In the household survey also, some kind of a score card is used as a 5-point scale is used to assess different aspect of agroecology. This has been useful in quantifying the status of agroecological transitions. We are expected to use community score card methodology again in 2024/5 when conducting endline research in Madagascar and therefore will be able to assess how far these agroecological transitions are taking place and more importantly, how these transitions (if underway) are contributing to poverty reduction, human health and to the enviornment. Will come back on this forum to share the results of this research. 

Regarding your other points, it will be interesting to hear more from you in terms of how (and where) you have used these indicators and what are the results indicating in terms of the value or worth of agroecology related interventions as this is also one of the point under discussion.     

Many thanks Expedit for sharing your experience and Abdoulaye for reinforcing the message about  the value of TAPE. Interesting to know that you have used TAPE in several studies in Benin. It will be truly great if you are able to share further on these experiences, in terms of what adaptations you have carried out in TAPE to address context specificities and what empirical evidence you are getting in terms of proving /disproving worth of agroecology related interventions. These insights would provide useful lessons to this community to understand and design better methodologies to measure agroecology. 

Thank you for this information 
Indeed TAPE is still a very interesting tool, but it needs to be adapted to your needs.

[translated from French]

For me, TAPE is a very good and effective tool for assessing the ecological and economic sustainability of our farming systems. And I've already used it in several studies I did in Benin. However, I think that the implementation of the three phases of TAPE seems too complex and certain themes or concepts have no equivalent in developing countries. I suggest that researchers try to adapt it to the local realities and concepts of their country.

[Translated from French]

Dear Ravi ji,

Thanks for raising this important point. I have my two cents' input based on my limited experience in this area. 

1. Agro-ecology is a multi-dimensional concept at different levels or scales so capturing the multifunctionality of agro-ecology for assessing the performance by developing a common framework may be challenging. Given its externalities/known–unknown/unknown-unknown interactions and functions, performance assessment by using quantitative techniques at the household or farm level will not be adequate. Participatory tools such as community scorecards, by using people’s observations/satisfaction can be useful. 
2. Environmental, economic and social/institutional aspects are important to see the performance but also its likelihood of continuation of the results. The performance indicators may vary depending on the local context but I used some indicators such as the presence of pollinators, level of pest attack, dilatory diversity & food security, use of traditional/indigenous crops, diversity of plants/crops used for food & medicinal purpose, contribution in farm income, level of stress (such as climate risks) tolerance and so on.   

Best regards,

Ram Chandra Khanal 

Dear Ravi, 

With reference to the Tool for Agroecological Performance Evaluation (TAPE), please find below my answers to the points you have raised in the discussion topic: 

1. Measuring Performance of Agroecological Transitions:

TAPE adopts a comprehensive approach to i) characterize the level of agroecological transitions of any kind of production system in agriculture, and to 2) assess performance across environmental, social, economic, and cultural dimensions. For poverty alleviation, it considers economic indicators, such as productivity, value added and income. For human health it measures dietary diversity, food security, and exposure to pesticides. For environment, is measures, among others, agrobiodiversity and soil health.

The tool's versatility allows users to customize assessments to fit specific farm or food system contexts, providing a nuanced understanding of performance.

2. Innovative Methodological Approaches to Measuring Agroecological Transitions:

TAPE employs a participatory approach. The tool is context-specific and adaptable. The methodology encourages the establishment of baseline data, allowing for the tracking of changes over time and the assessment of the effectiveness of interventions.

TAPE's flexibility makes it replicable in various contexts, enabling its use across different scales and geographic locations. The tool has reached already more than 10.000 production systems in more than 50 countries across all regions of the world. 

3. Demonstrative Empirical Evidence on Agroecology:

TAPE has been created, among other things, for producing evidence on the performance of agroecology. 

Numerous case studies and projects employing TAPE have showcased positive outcomes, demonstrating the value of agroecology in fostering sustainable and resilient food systems (results from Mali, Argentina, Lesotho have been already published).

I hope this helps.

Best regards,

Dario

Season's Greetings Ravinder,

Your work and approach sound interesting, although most of it is not in my direct area of work. I can think of possible ways involving satellite data that can provide some insights at moderate/low resolution (village or farm field level). It should be possible to assess the impact of different practices on the vegetation / food system productivity, and consequently on SDG goals. 

 Please feel free to connect in case you would like to know more.

Best regards,
Dushyant Mahadik
Asst Professor
School of Management, 
National Institute of Technology Rourkela 769 008
Ph: +91-661-246-2809

I would like to contribute the following paper to the discussion. It was recently published by Outlook on Agriculture in a Special Issue on Agroecology:

“Measuring agroecology and its performance: An overview and critical discussion of existing tools and approaches”

Matthias S Geck, Mary Crossland and Christine Lamanna 

Outlook on Agriculture 52:349-359

https://journals.sagepub.com/doi/full/10.1177/00307270231196309

Abstract

Agricultural and food systems (AFSs) are inherently multifunctional, representing a major driver for global crises but at the same time representing a huge potential for addressing multiple challenges simultaneously and contributing systemically to the achievement of sustainable development goals. Current performance metrics for AFS often fail to take this multifunc-tionality into account, focusing disproportionately on productivity and profitability, thereby excluding “externalities,” that is, key environmental and social values created by AFS. Agroecology is increasingly being recognized as a promising approach for AFS sustainability, due to its holistic and transformative nature. This growing interest in and commitment to agroecology by diverse actors implies a need for harmonized approaches to determine when a practice, project, investment, or policy can be considered agroecological, as well as approaches that ensure the multiple economic, environmental, and social values created by AFS are appropriately captured, hence creating a level playing field for comparing agroecology to alternatives. In this contribution to the special issue on agroecology, we present an overview of existing tools and frameworks for defining and measuring agroecology and its performance and critically discuss their limitations. We identify several deficiencies, including a shortage of approaches that allow for measuring agroecology and its performance on landscape and food system scale, and the use of standardized indicators for measuring agroecology integration, despite its context-specificity. These insights highlight the need for assessments focused on these overlooked scales and research on how best to reconcile the need for globally comparable approaches with assessing agroecology in a locally relevant manner. Lastly, we outline ongoing initiatives on behalf of the Agroecology Transformative Partnership that aim to overcome these shortcomings and offer a promising avenue for working toward harmonization of approaches. All readers are invited to contribute to these collaborative efforts in line with the agroecology principle of participation and co-creation of knowledge.