I agree with most of what has been contributed below, especially Anna Maria's comments on involving farmers or other local stakeholders in developing the evaluation design and relevant indicators. Using a participatory approach will help uncover assumptions in program design or other local incentives that the research team did not foresee, and which therefore would not be covered in the questionnaire. Involving local research team members is helpful, but the closer you can get to bringing the individuals who would be program participants themselves into the design, the more relevant and targeted the survey will be.
Just a few thoughts in addition to what has already been said:
1.Striking a balance between depth and length of assessment:
Think critically about how the data will actually be used to design/adapt programming or inform decisions, and then eliminate the assessment questions that collect unnecessary data points. One way to do this is to conduct a simulation of use: the evaluation team brainstorms potential evaluation findings, and then has a facilitated discussion with the intended evaluation users to discuss how those findings would influence program design or funding decisions. This can help narrow down which data points you really need. For example, you might have questions about types of employment for various household members, but recognize that these data aren't likely to impact your program design - so you can eliminate them from the questionnaire.
I'd also recommend bringing in more participant-led data collection methods (e.g. Most Significant Change or other storytelling formats, participant-led stakeholder mapping exercises in which they visually map out how the local system presents challenges or changes to information flow, resource access, etc), as this will add depth to your data.
I agree with the comments made by others about involving local stakeholders in data collection (to the extent possible in light of methodological limitations in an experimental setting). This example of community-led enumeration in Ghana is good inspiration!
2. Making findings from M&E assessments useful to farmers:
The data placemat approach, in which the facilitator presents data visually, then guides stakeholders to explore and explain the data in their own words, is a good way to get people interacting with the data. For an audience of smallholder farmers, you'd need to rely more heavily on visuals that farmers relate to (rather than bar charts!) but I think there's a way it could be done!
After presentation of findings, you could have farmers generate short skits that represent their vision of a future in which the lessons about farm sustainability are implemented. This helps generate enthusiasm for action, as motivated by evaluation findings.
Amanda Satterwhite
Monitoring, Evaluation and Learning Specialist USAIDHello all,
I agree with most of what has been contributed below, especially Anna Maria's comments on involving farmers or other local stakeholders in developing the evaluation design and relevant indicators. Using a participatory approach will help uncover assumptions in program design or other local incentives that the research team did not foresee, and which therefore would not be covered in the questionnaire. Involving local research team members is helpful, but the closer you can get to bringing the individuals who would be program participants themselves into the design, the more relevant and targeted the survey will be.
Just a few thoughts in addition to what has already been said:
1.Striking a balance between depth and length of assessment:
Think critically about how the data will actually be used to design/adapt programming or inform decisions, and then eliminate the assessment questions that collect unnecessary data points. One way to do this is to conduct a simulation of use: the evaluation team brainstorms potential evaluation findings, and then has a facilitated discussion with the intended evaluation users to discuss how those findings would influence program design or funding decisions. This can help narrow down which data points you really need. For example, you might have questions about types of employment for various household members, but recognize that these data aren't likely to impact your program design - so you can eliminate them from the questionnaire.
I'd also recommend bringing in more participant-led data collection methods (e.g. Most Significant Change or other storytelling formats, participant-led stakeholder mapping exercises in which they visually map out how the local system presents challenges or changes to information flow, resource access, etc), as this will add depth to your data.
I agree with the comments made by others about involving local stakeholders in data collection (to the extent possible in light of methodological limitations in an experimental setting). This example of community-led enumeration in Ghana is good inspiration!
2. Making findings from M&E assessments useful to farmers:
The data placemat approach, in which the facilitator presents data visually, then guides stakeholders to explore and explain the data in their own words, is a good way to get people interacting with the data. For an audience of smallholder farmers, you'd need to rely more heavily on visuals that farmers relate to (rather than bar charts!) but I think there's a way it could be done!
After presentation of findings, you could have farmers generate short skits that represent their vision of a future in which the lessons about farm sustainability are implemented. This helps generate enthusiasm for action, as motivated by evaluation findings.
Best of luck!
Amanda