Making data collection meaningful and useful to farmers: what is your experience?

©FAO/Mutasim Billah

Making data collection meaningful and useful to farmers: what is your experience?

I am part of a team at the Food and Agriculture Organization of the United Nations (FAO) that is currently developing a survey-based tool to support the monitoring and evaluation of farm production sustainability, in alignment with the principles of Climate Smart Agriculture (CSA) and the SDGs, particularly SDG 2.4.1 on sustainable agriculture.

The tool is geared towards agriculture extension agents, project developers and implementers, and M&E practitioners. Its goal is to assess the sustainability of farm production in smallholder settings, help identify areas for improvement, and monitor change over time.

In view of the finalization of the tool, I would like to invite members, and particularly those of you involved in the collection, analysis and reporting of data related to agriculture, rural development and food security, to share comments and experience on the following:

1.Striking a balance between depth and length of assessment: monitoring and assessment exercises based on interviews and farm surveys can put significant burden on respondents, for example diverting time that would be otherwise allocated to other activities. Respondent fatigue due to lengthy interviews/surveys can also result in lower quality of data collected, and therefore in lower reliability of results. On the other hand, a shorter assessment may result in a level of depth that is insufficient to design effective interventions.

  • How can the burden on smallholder farmers be reduced during M&E assessments?
  • What are the best ways to incentivize farmers to take part in the survey (e.g. non-monetary incentives, participation in survey tailoring, in presentation of results)?

2. Making findings from M&E assessments useful to farmers: considering the burden on farmers resulting from M&E exercises, it is key to ensure results are meaningful and accessible to them. This is in fact an explicit objective of the M&E tool we are developing. The assessment seeks to provide an indication of sustainability strengths and weaknesses that can be used by e.g. extension agents to help farmers identify targeted practices that can increase overall sustainability of production.

  • Based on your experience, what could be the most effective ways to communicate results from the sustainability assessment to farmers (e.g. field visits and peer learning, technical information workshop)? What kind of communication materials (e.g. briefs, leaflets, others) are most appropriate to support knowledge sharing events?
  • Do you have experience in comparing results among farmers in a participatory way? What method have you used to do this? Was it effective?
  • How can the results be used for non-formal education of farmers (e.g. to raise awareness and/or build capacity on ways to increase farm sustainability)?

Given the specific expertise of the EvalForward community in applied M&E processes, we welcome members that are interested in taking part to the review of the tool to reach out with a brief message outlining their current affiliation and area of expertise to: Reuben.Sessa@fao.org and David.Colozza@fao.org

 

Dear David,

Thanks for initiating this useful discussion. I want to share experiences from our organisation on how we have navigated some of the points you raised.

To minimise the time burden on respondents, we try to be strict during questionnaire development. This generates some payback that all the questions we ask relate to some indicator that we will analyse. It also helps us remove some useful and desirable questions to only relevant questions without which our assessment would not be complete.

We have found engaging farmers to tell their stories as we present our findings very useful. It also helps ground-truth our findings. in addition, they can enumerate lessons learnt during the assessment.

For us at Tegemeo Institute, we try and have forums with farmers where we discuss our findings and how they can use the findings for their benefit. in addition, we have found the use of infographics handy with farmers as utilising local information networks to disseminate information. Furthermore, when we have compared farmers, we have found their approaches to make comparisons and deductions quite informative. I definitely recommend participatory approaches.

To reduce the burden on small-scale farmers during monitoring and evaluation assessments, the collection tools should be adapted to the different social realities of these farmers, and farmers should be involved in the design of these tools by involving them directly in the preparatory work.

As for the best ways to encourage farmers to participate in the survey, it will be necessary to propose compensatory measures that would correspond to the time they would have lost in the monitoring-evaluation exercise, for example by offering some refreshment when they participate in the activity. Also, it will be necessary to adapt the collection tools to what they would have proposed (make proposals for the choice of tools if possible).

[Contribution originally posted in French]

The main strategy to make data collection meaningful and useful to farmers is to involve them in the process of investigation, from the beginning to the end. The best method for that is the IAR4D Approach (Integrated Agricultural Research for Development), developed by the Forum for Agricultural Research in Africa (FARA) in 1983. Then, the raison and the process of the data collection are explained to the farmers in a participatory way, and in their local language. Then, evaluation is considered as a system that is made of many sub-systems that must work together to foster development. All actors that are involved in the process, including farmers, interact and jointly foster their capacities. Thus, the IAR4D Approach simultaneously addresses research and development as a fused continuum for generation of innovation. Generally, in the process, the farmers’ analysis of the evaluation steps and their analysis of the findings are different from that of the researchers/evaluators, and then improve the quality of the evaluation. With this approach, the farmers constitute at the same time the channel for the process explanation and the results dissemination.

For more information, see https://faraafrica.org/iar4d/

Dear members,

I would like to share my experience in data and information collection among farmers with the Indonesian Farmers Association. 

Usually we contact farmers though the chief of village or farmers leader and invite them in a certain day/time and place. We have a preliminary discussion with the chiefs and send the questions in advance so that the leaders can respond generally to them to get an idea and have prior information. At the meeting we go deeper with farmers also one-by-one and get fresh information and do crosschecks when needed.

This was in the pre-pandemic situation, as currently gatherings are stopped and we only call on individuals.

It is important to note that in our rural areas there is a culture and tradition by which if we invite farmers we need to set up a type of ceremony: we have prayers, delivery speeches by chief of village or farmers leaders, open discussion and we prepare food. This is also the incentive. Meetings can be one day long and in some cases, since farmers dedicate a long time, we pay back the lost workday.

In addition to the chief, we also invite the extension workers: these are the ones who maintain the relations with the farmers and the leaders and already have a lot of data and information.

In the case of international donor-funded initiatives and the related Monitoring and Evaluation missions, in order to avoid farmers feeling this as a verification or audit, we prefer not to use the terms M&E and call them SIS - Supervision Implementation Support missions.

On communication: we communicate back results from the programme evaluations. Also in this case we have gatherings and we send in advance the highlights (summary) of the evaluation reports so that the farmers can react and disagree / respond during the meetings.

I hope this is useful information when approaching farmers for data collection and communication.

See my video here on the AFOSP-MTCP2 Indonesia Report:  https://youtu.be/kICPu8tb7jc?list=PLtXDxoTN3R8ajQmREnrdJUEvXlwCCwaB6

Agusdin Pulungan

Hello everyone,

I have tried to answer each question and my answers are below. They are based on some of my experience with small farmers.

1. Striking a balance between depth and length of assessment: 

Small farmers are very busy because they have to find alternative / complementary sources of income. In addition, social time is important (marriages, tea time, football for the young, carpet making for women...).

Thus, assessment time should fit within their schedule. I suggest short questionnaires that are meaningful to them, which brings in that the programme should take into account their actual needs and not 100% according to organizational needs.

  • How can the burden on smallholder farmers be reduced during M&E assessments?
  • What are the best ways to incentivize farmers to take part in the survey (e.g. non-monetary incentives, participation in survey tailoring, in presentation of results)?
  1. Make it a social time and talk about what is meaningful  to them (ex cereals in the mountainous areas). The usually preferred time is the afternoon. ex. plan assessment time during tea time and work with focus group. If the questionnaire is preferred then it will take more  time for the evaluator because she / he will have to adjust to each farmer;
  2. Allow women to bring in toddlers or small infants ( up to 5 years);
  3. Give away written information on the programme. They will keep it and show it to their schooled children;
  4. Plan on lunch or afternoon tea with snacks.

2. Making findings from M&E assessments useful to farmers: 

- Like for the assessment, plan on information workshops in "a between seasons" time to avoid getting on the way for "actual" work;

- Provide leaflets, audios, videos, pictures;

- Allow for Q&A sessions.

  • Do you have experience in comparing results among farmers in a participatory way? What method have you used to do this? Was it effective?
  • How can the results be used for non-formal education of farmers (e.g. to raise awareness and/or build capacity on ways to increase farm sustainability)?

1. Comparing results among farmers is effective in showing results and making farmers adopt new techniques. I used a treatment / non treatment method . The non treatment was actually from farmers not adhering to the programme. Once results were obvious, they asked to be included.

2. Results could be used in  non-formal education of farmers through exchange visits among peers, audios and videos distributed through instant messaging, result presentations on field visits of extension workers. 

Malika Bounfour

 

Dear David,

Thanks for bringing these important issues to this community of Practice. Based on my experience working in the developing countries, I have the following input for your reference.

With best regards, 

1. Striking a balance between depth and length of assessment: monitoring and assessment exercises based on interviews and farm surveys can put significant burden on respondents, for example diverting time that would be otherwise allocated to other activities. Respondent fatigue due to lengthy interviews/surveys can also result in lower quality of data collected, and therefore in lower reliability of results. On the other hand, a shorter assessment may result in a level of depth that is insufficient to design effective interventions.

How can the burden on smallholder farmers be reduced during M&E assessments?

I prefer to do/ am doing: 

  • Making objective oriented short questionnaires
  • Mostly close ended but also provision of sharing their views and perspectives
  • Interview in their own setting and preferred time
  • Making them feel they are also benefitted from this exercise
  • Create strong rapport (interpersonal skills) – (not mechanical but also speak on their personal issues)
  • Provide some present (this can be to their children)

What are the best ways to incentivize farmers to take part in the survey (e.g. non-monetary incentives, participation in survey tailoring, in presentation of results)?

I prefer to do / am doing: 

  • When I was programme/ project manager I used to provide farmers some financial compensation (I strongly feel we need to pay the information provider as we information collectors are making a good sum of money for similar kind of functions)
  • Compensate their time with good snacks / refreshment
  • Provide them a present as a ‘token of love’
  • Acknowledging their support

2. Making findings from M&E assessments useful to farmers: considering the burden on farmers resulting from M&E exercises, it is key to ensure results are meaningful and accessible to them. This is in fact an explicit objective of the M&E tool we are developing. The assessment seeks to provide an indication of sustainability strengths and weaknesses that can be used by e.g. extension agents to help farmers identify targeted practices that can increase overall sustainability of production.

Based on your experience, what could be the most effective ways to communicate results from the sustainability assessment to farmers (e.g. field visits and peer learning, technical information workshop)? What kind of communication materials (e.g. briefs, leaflets, others) are most appropriate to support knowledge sharing events?

  • Clarify the objectives – how sustainability assessment are important to farmers and their groups/
  • Organize sharing meeting and get their feedback on the result
  • Take full use of local resources persons/ local groups or trusted partners while sharing the results
  • Use illustration/visual aids/local language  
  • Less of use of technical words and complex terms

Do you have experience in comparing results among farmers in a participatory way? What method have you used to do this? Was it effective?

When possible:

  • Making sharing group according to the interest groups (such as women farmers, youth farmers, farmers groups based on their production or participation in different value chain)
  • Use illustration / examples / visual aids/ simple demonstration (such as big maize cub vs small cub) relevant to local context  (for example if you say 50%, in many cases – they do not understand, if you give example such as 100 unit and 150 unit (50% additional)- may be will be in position to understand

How can the results be used for non-formal education of farmers (e.g. to raise awareness and/or build capacity on ways to increase farm sustainability)?

  • Develop participatory based farmer centred training module considering the need of the farmers
  • Provide sharing opportunity by farmers (farmer to farmer approach)
  • Use of Use illustration / examples / visual aids/ simple demonstration

-----------------------------------------

Ram Chandra Khanal (PhD)

Evaluator and programme manager: Climate change/NRM/Agriculture

Dear Colleagues,

I would like to share with you our experience in the implementation of the GAFSP MMI project in Senegal.

We have set up a participatory M&E system in which the communities are the central elements. For each PO (producer organisation), we selected a team of supervisors and facilitators from the communities. We trained these people in digital data collection via kobo and also in the facilitation of a qualitative survey in order to empower them in data collection. The training was difficult, given the level of some facilitators, but it must be acknowledged that they are very much empowered and accepted by the communities for the provision of reliable data. We are continuing this process to make monitoring booklets available to the grassroots producer organisers, which are filled in every year so that the facilitators, via the tablet, will go on missions to collect data in the monitoring booklets of the value chain.

It is necessary to have an inclusive approach and to define an M&E guideline that makes producers responsible for the implementation of the monitoring-evaluation system.

[this contribution was originally posted in French]