From my interest the most important impact of evaluations is the guidance they provide for future projects in guiding them in evolving to better serve the intended beneficiaries. Unfortunately, at least for smallholder agriculture development projects, the evaluations have become more a mechanism for propagandizing projects regardless of how well they are relied upon or avoided by the beneficiaries.
To some degree this is necessary as the up-front cost in getting projects from conception to implementing contract and finally an opportunity for detailed discussions with beneficiaries can be a couple million dollars and over two years of extensive time and effort by a multitude of people. With that much up-front costs before the implementer have a chance for detailed interviews with beneficiaries to fully ascertain their needs, no one wants to learn that the beneficiaries were not really interested in the activity and mostly avoiding the project for various valid reasons.
However, when used more for propaganda then guidance, there may be some benefits to the implementer in terms of project extensions and future projects, but no real benefit for the beneficiaries. Instead, projects, that by most standards would be considered a failure, become more deeply entrenched in the future development projects, reducing any effort to adjust projects, and squandering massive amounts of development investments that could better be used in an alternative approach.
Preventing an evaluation effort become more a propaganda tool instead of guidance for future projects may be more how data is analysis than what is collected. This could be as easy as avoiding aggregate analysis in favor of more percent analysis. In dealing with smallholder farmers, each managing one to two hectares, one can quickly count some impressive numbers of individual participating and get a good propaganda appraisal, but it will say nothing about what would be possible. However, if expressed in terms of percent of potential beneficiaries, the impressive aggregate number could become highly questionable. The same could be said about the amount of produce marketed through the project. The total volume can sound impressive, but if prorated to individual members could be only a small percent of their production, perhaps representing only in-kind loan repayments, with the bulk of their business going to the very people the project is attempting to replace to provide the beneficiaries a better business deal. If converted to a community basis, which is what development projects are based on, it would be nearly a trivial amount.
The real need for an evaluation is, before beginning the evaluation to develop a set of targets for what would separate success from failure and do this as percent rather than an aggregate number. This would be the percent of the farmers activity participating in program, or the percent of farmers produce marketed through the project, as well as the percent of the communities produced marketed through the project. For example: 70% of the farmers activity participating instead of 150 famers involved, or 80% maize production marketed through the project with only 20% side-sold. This could easily give a very different perspective of how well the project was received and relied upon by the smallholder beneficiaries and result in make some major adjustment to future projects that would better serve the beneficiaries.
RE: Reporting and supporting evaluation use and influence
From my interest the most important impact of evaluations is the guidance they provide for future projects in guiding them in evolving to better serve the intended beneficiaries. Unfortunately, at least for smallholder agriculture development projects, the evaluations have become more a mechanism for propagandizing projects regardless of how well they are relied upon or avoided by the beneficiaries.
To some degree this is necessary as the up-front cost in getting projects from conception to implementing contract and finally an opportunity for detailed discussions with beneficiaries can be a couple million dollars and over two years of extensive time and effort by a multitude of people. With that much up-front costs before the implementer have a chance for detailed interviews with beneficiaries to fully ascertain their needs, no one wants to learn that the beneficiaries were not really interested in the activity and mostly avoiding the project for various valid reasons.
However, when used more for propaganda then guidance, there may be some benefits to the implementer in terms of project extensions and future projects, but no real benefit for the beneficiaries. Instead, projects, that by most standards would be considered a failure, become more deeply entrenched in the future development projects, reducing any effort to adjust projects, and squandering massive amounts of development investments that could better be used in an alternative approach.
Please review the following webpage:
https://agsci.colostate.edu/smallholderagriculture/appeasement-reporting-in-development-projects-satisfying-donors-at-the-expense-of-beneficiaries/
Preventing an evaluation effort become more a propaganda tool instead of guidance for future projects may be more how data is analysis than what is collected. This could be as easy as avoiding aggregate analysis in favor of more percent analysis. In dealing with smallholder farmers, each managing one to two hectares, one can quickly count some impressive numbers of individual participating and get a good propaganda appraisal, but it will say nothing about what would be possible. However, if expressed in terms of percent of potential beneficiaries, the impressive aggregate number could become highly questionable. The same could be said about the amount of produce marketed through the project. The total volume can sound impressive, but if prorated to individual members could be only a small percent of their production, perhaps representing only in-kind loan repayments, with the bulk of their business going to the very people the project is attempting to replace to provide the beneficiaries a better business deal. If converted to a community basis, which is what development projects are based on, it would be nearly a trivial amount.
The real need for an evaluation is, before beginning the evaluation to develop a set of targets for what would separate success from failure and do this as percent rather than an aggregate number. This would be the percent of the farmers activity participating in program, or the percent of farmers produce marketed through the project, as well as the percent of the communities produced marketed through the project. For example: 70% of the farmers activity participating instead of 150 famers involved, or 80% maize production marketed through the project with only 20% side-sold. This could easily give a very different perspective of how well the project was received and relied upon by the smallholder beneficiaries and result in make some major adjustment to future projects that would better serve the beneficiaries.
Please review the following webpages:
https://agsci.colostate.edu/smallholderagriculture/mel-impressive-numbers-but-of-what-purpose-deceiving-the-tax-paying-public/
https://agsci.colostate.edu/smallholderagriculture/perpetuating-cooperatives-deceptivedishonest-spin-reporting/
https://agsci.colostate.edu/smallholderagriculture/request-for-information-basic-business-parameters/
Thank you