Thank you very much for the interesting topic for the discussion. Below, I'd like to share some insights from my practice as evaluation consultant and researcher. I worked on a number of assignments from the local to global level, which involved surveys with farmers and other rural people in diverse geographic contexts. My most recent project has been focused on the capacities of evaluation stakeholders in multi-actor projects targeting agricultural innovation (https://liaison2020.eu). I've a strong background in sociology and psychology, which also affect my approaches to surveying.
Striking a balance between depth and length of assessment:
How can the burden on smallholder farmers be reduced during M&E assessments
This usually depends on the context. For instance, I interviewed farmers who were very interested in chatting with me, both about the survey questions and non-related topics. It's important to recognize their needs and issues they face, which may be often different from what we expect as evaluators. Some people are more, some less busy, introvert or extrovert and it can also affect their eagerness to engage into the task. I normally strive towards a balance between their needs and mine. At times, one may need to compromise skipping some questions in the survey. This could be reflected at an earlier stage - the evaluation design, when decisions are to be made on the direct and proxy indicators.
What are the best ways to incentivize farmers to take part in the survey (e.g. non-monetary incentives, participation in survey tailoring, in presentation of results)?
It can be helpful to ask what are their evaluation needs: a problem they want to solve, in which evaluation and data could help. They may be quite different from what the evaluators intend, so one should try to negotiate and seek an optimization in the evaluation design. It’s helpful to engage farmers into defining the scope of evaluation, relevant questions and indicators. For instance, I once run a workshop where participants were presented with a list of possible indicators and could rate those, which were most relevant in their opinion. The result was quite different from what the evaluators anticipated. Non-monetary incentives are also helpful. I remember bringing a box of fine chocolate from my home city to farmers, with whom I stayed during the survey work. They were helping me to identify other survey participants (snowballing) and at the end gave me also eggs from their farm to bring home. Concerning monetary incentives, I always fear the Hawthorne’s effect, i.e. an increased performance of respondents under the pressure of being studied and rewarded.
Making findings from M&E assessments useful to farmers:
Based on your experience, what could be the most effective ways to communicate results from the sustainability assessment to farmers (e.g. field visits and peer learning, technical information workshop)? What kind of communication materials (e.g. briefs, leaflets, others) are most appropriate to support knowledge sharing events?
Definitely, P2P learning is very useful. This way people can exchange with each other using the same language. As evaluators we often tend to communicate in a different way than farmers, hence a skilled facilitation is usually a better option than and top-down way of presenting the results. It’s good to have it as a facilitated discussion, field trip and some informal get together. In addition, various dissemination channels can be helpful, such as radio, videos or leaflets. Using visual communication is quite effective, in my experience. I remember evaluating a project where farmers had issues with recognizing grapevine diseases, which already existed in their area. They did not know actual names of those, but pictures helped to recognize them.
Do you have experience in comparing results among farmers in a participatory way? What method have you used to do this? Was it effective?
I remember a visioning exercise where evaluation results were presented and further elaborated. It was a mid-term project evaluation, where people who were earlier interviewed (farmers and other rural community members) participated in the event, also some contributed with their stories. Based on this, a visioning exercise was run by the external facilitators, which was intended to help in improving the project and planning other activities for the community’ future. Various methods were used, including the facilitator’s toolbox with sticky notes, flipchart and others.
How can the results be used for non-formal education of farmers (e.g. to raise awareness and/or build capacity on ways to increase farm sustainability)?
In principle, the evaluation results need to be translated into the farmers’ language. With these, they can be used in many ways through capacity building activities. Forms of P2P and experiential learning are in my experience most effective to maximize the uptake of the evaluation results at the farm level. Sometimes, the broader enabling environment of evaluation need to be also considered, for instance farmers may lack some incentives to change their practice, despite an increased awareness on the issue. It's important to choose the right means of communication, which can be also different in various countries, regions and depend on the literacy of the farmers and their community leaders.
RE: Making data collection meaningful and useful to farmers: what is your experience?
Dear David and Colleagues,
Thank you very much for the interesting topic for the discussion. Below, I'd like to share some insights from my practice as evaluation consultant and researcher. I worked on a number of assignments from the local to global level, which involved surveys with farmers and other rural people in diverse geographic contexts. My most recent project has been focused on the capacities of evaluation stakeholders in multi-actor projects targeting agricultural innovation (https://liaison2020.eu). I've a strong background in sociology and psychology, which also affect my approaches to surveying.
This usually depends on the context. For instance, I interviewed farmers who were very interested in chatting with me, both about the survey questions and non-related topics. It's important to recognize their needs and issues they face, which may be often different from what we expect as evaluators. Some people are more, some less busy, introvert or extrovert and it can also affect their eagerness to engage into the task. I normally strive towards a balance between their needs and mine. At times, one may need to compromise skipping some questions in the survey. This could be reflected at an earlier stage - the evaluation design, when decisions are to be made on the direct and proxy indicators.
It can be helpful to ask what are their evaluation needs: a problem they want to solve, in which evaluation and data could help. They may be quite different from what the evaluators intend, so one should try to negotiate and seek an optimization in the evaluation design. It’s helpful to engage farmers into defining the scope of evaluation, relevant questions and indicators. For instance, I once run a workshop where participants were presented with a list of possible indicators and could rate those, which were most relevant in their opinion. The result was quite different from what the evaluators anticipated. Non-monetary incentives are also helpful. I remember bringing a box of fine chocolate from my home city to farmers, with whom I stayed during the survey work. They were helping me to identify other survey participants (snowballing) and at the end gave me also eggs from their farm to bring home. Concerning monetary incentives, I always fear the Hawthorne’s effect, i.e. an increased performance of respondents under the pressure of being studied and rewarded.
Definitely, P2P learning is very useful. This way people can exchange with each other using the same language. As evaluators we often tend to communicate in a different way than farmers, hence a skilled facilitation is usually a better option than and top-down way of presenting the results. It’s good to have it as a facilitated discussion, field trip and some informal get together. In addition, various dissemination channels can be helpful, such as radio, videos or leaflets. Using visual communication is quite effective, in my experience. I remember evaluating a project where farmers had issues with recognizing grapevine diseases, which already existed in their area. They did not know actual names of those, but pictures helped to recognize them.
I remember a visioning exercise where evaluation results were presented and further elaborated. It was a mid-term project evaluation, where people who were earlier interviewed (farmers and other rural community members) participated in the event, also some contributed with their stories. Based on this, a visioning exercise was run by the external facilitators, which was intended to help in improving the project and planning other activities for the community’ future. Various methods were used, including the facilitator’s toolbox with sticky notes, flipchart and others.
In principle, the evaluation results need to be translated into the farmers’ language. With these, they can be used in many ways through capacity building activities. Forms of P2P and experiential learning are in my experience most effective to maximize the uptake of the evaluation results at the farm level. Sometimes, the broader enabling environment of evaluation need to be also considered, for instance farmers may lack some incentives to change their practice, despite an increased awareness on the issue. It's important to choose the right means of communication, which can be also different in various countries, regions and depend on the literacy of the farmers and their community leaders.
With best wishes from Budapest,
Anna Maria Augustyn
https://www.linkedin.com/in/aniaaugustyn/
LIAISON2020 | Optimising interactive innovation