I hope everyone is healthy and take good care of yourselves and families, particularly our colleagues working in Rome. Are you all able to work from home and keep up with the development efforts? It does provide a good chance for us emeritus people to draft responses to various forum of interest.
Regarding the interest in M&E, either individually or jointly, my concern is that the value of the exercise is only as good as the questions being asked, the way the data is tabulated, and the finances available to implement the M&E program, particularly within a host governments with limited tax base to support any services including development activities. Associated with this is the stated vs. underlying objective of the M&E effort. As I understand M&E it is designed to be an independent effort on behalf of the underwriting taxpayers to assure the development money is well invested and not wasted as well as guide future projects to more effectively address the objective and allow projects to effectively evolve to better serve the intended beneficiaries. It should not be a propaganda tool to promote projects as successful when by all normal business standards, they are complete failures. As I have listened to many of the USAID MEL (Monitoring, Evaluation, & Learning) webinar and reviewed numerous project reports I am left with the distinct impression the MEL effort is primary to deceive the American public and the elected members of Congress into thinking they are making major contributions to rural poverty alleviation of smallholder farmers, while most of the intended beneficiaries are avoiding active participation like it were the plague, or perhaps the current coronavirus. This may effectively attract continued funding but does nothing for beneficiaries other than keep them deeply entrenched in poverty. More likely M&E activities diverted to project propagandizing will have a substantial negative impact as it reinforces failed programs into future programs preventing them for evolving to better serve the beneficiaries.
Allow me to illustrate use poverty alleviation for smallholder farmers as reference beneficiaries:
Missed question: One question that has been overlooked for most of the last 50 years is the timing of agronomic activities starting with crop establishment. This gets to the limitations of Agronomy research which does an excellent job of determining the physical potential of an area but says nothing about the operational requirements to extend the small plot research across the rest of the farm or community. The assumption is that labor or contract mechanization is readily available, and the farmers only need to be “taught” the value of early planting. A valid M&E effort 50 years ago seeking information including simple field observations on the timing of agronomic operations would have noted that under manual operations crop establishment was spread over 8 weeks or more with rapidly declining yields with the delay to the point it is impossible for manual agriculture to meet basic family food security. If the M&E programs did pick that up the whole poverty alleviation effort would have shifted from badgering smallholders on the importance of early planting and focus on providing access to the operational requirements such as access to contract tillage that would make timing of crop establishment discretionary. It would also note to the extent the current emphasis is on value chain development as a means to promote additional production with the underlying assumption there is surplus operating capacity, is premature until after the operational capacity is increased so farmers could get their crops planted in a more timely manner, produce enough to meet family food security and still have ample production to justify the improved value chain.
The failure to identify the operational limits of smallholder farmers has a major impact on the current emphasis for quality nutrition. Here the underlying question is what are the calories needed to optimize the economic opportunities which are largely associated with heavy manual agronomic field work? This requires at least 4000 kcal/day but with the manual agriculture most farmers are limited to 2000 to 2500 kcal/day barely meeting the 2000 kcal/day basic metabolism, with only enough work calories for a couple diligent hours of manual labor. That may go a long way to explaining the 8-week crop establishment time. But it does mean that it will be difficult to accept a diversified diet if it requires less calories which in turn will reduce their economic opportunity including crop production. However, the nutrition M&E emphasis the impressive number of “beneficiaries” informed without looking at their compromises in being able to utilize the information and the affordability of the improved nutrition vs. available income. The whole concept of dietary requirement to meet economic opportunities seems completely lost to the nutrition improvement effort but certainly not the “beneficiaries” being badgered with nutritional information they cannot use. How often are our proposed agronomic interventions more labor intensive and thus an attempt to obligate hungry exhausted smallholders to exert energy will in excess of their available calories, possibility as much as twice their available calories. When doing so have we met the definition of genocide or come very close to doing such?
Data Manipulation: Even if quality M&E data is collected it could be tabulated either as a propaganda tool to promote programs even if they are basically a failure or as a guide the evolution of future programs. There basically two ways of tabulating M&E data. If you are interested in using M&E for propaganda promotions of the project regardless of how successful it is you simply report to aggregate totals of the data. With a large multi-nation with multi-programs within a nation it can easily provide some highly impressive number, often in the hundreds of thousands. This will appease the public perhaps assure future funding but be meaningless as evaluation tool. A more effective evaluation tool for guiding future project would the expressing the same data as a percent of the potential. Using my pet concern of the overemphasis on farmer organizations to funnel assistance to smallholder farmers. It is possible and the USAID MEL program routinely does, claim they are assisting several hundred thousand smallholder farmers, but a more detailed analysis would indicate rarely do they have more than 10% of the potential farmers, within the smallholder communities they are claiming to assist, actively participating and even then they will divert most of their business to alternative service providers. Thus, they may be assisting a few hundred thousand but have a few million potential beneficiaries avoiding the project. Then since most of the active members will side sell all but what is needed for loan repayments, the total market share from the community will be a trivial less than 5% and virtually no impact on the overall communities’ economy. Not what would be considered a success by most business standards!! What is urgently needed here is an upfront statement of what will constitute a minimal successful project in terms of percent of potential beneficiaries actively participating in the project and the market share funneled through the project. All expressed in percent of the community.
This target for a success project needs to be consistent with what the underlying taxpayers are expecting. In the case of farmer organizations, the expectation is over 60% participation and over 50% market share. Can anyone come close to this level of success particularly when projects are openly attempting to compete with private service providers?? A good sincere M&E program, either individual or together, should have realized this some 30 years ago and if so the development effort would have carefully looked at alternatives, including accepting that the much vilified private service providers were effective efficient business models that actually provide the farmers the financially best and most convenient business services. Also, it would not take a lot of effort to appreciate the farmer organization model is administratively too cumbersome to compete with private service providers. The overhead costs of operating a farmer organization substantially exceeded the private service profit margins so relying on farmer organization would drive the smallholder members deeper into poverty despite the massive rhetoric about poverty alleviation. Thus, the farmers wisely avoid the farmers organization so that they require continuous external support and facilitation and fully collapse once external support ends.
Financial Limits: The concern with financial limits is that host government are working with a limited tax base to provide support services for which the administrative nature of a national M&E program just cannot get the priority needed for quality data collection. The overall economic environment in most host countries is what I refer to as financially suppressed. This is an economy with such a high level of poverty that most families spend 80% of wages or production just to obtain a meager diet for their family even when the consumer prices are only 1/3rd to 1/5th the USA or EU prices. With that percent of income devoted to survival there just is not enough “discretionary” income to make an effective tax base for the government to raise funds for civil services. Sorry but no taxes no services. The result is that for all practical purposes the government is financially stalled. I think this is why, in previous comments, most of the M&E was done through NGOs with access to external funds. The problem is when you try and compel a government to undertake an administrative task that cannot be fully funded including the operational costs for the field trips necessary to collect reliable information, the quality and reliability of the task become questionable. With no funds but pressure to provide the data, civil officers will often simply complete the information as best they can according to what they perceive is taking place. It is the best they can do but could be far from reality. It should be noted that virtually 90% of all smallholder farmers in most host countries have never interacted with a civil officer including village agricultural extension officers. It is really better not to ask for a M&E program than have one completed by perception instead of facts.
The bottom line of this comment is to be very careful and make certain any M&E program alone or combined is mechanism for representing the beneficiaries and advancing the development effort and does not become a means for embellishing programs that may be socially desirable but for which the beneficiaries are avoiding like the current virus. It will be very difficult to meet the SDG when the M&E activities are more promoting and instituting failed programs instead of guiding the evolution of programs to better serve the beneficiaries.
RE: Peut-on se contenter de l'évaluation seule pour s'assurer de l'atteinte des ODD?
I hope everyone is healthy and take good care of yourselves and families, particularly our colleagues working in Rome. Are you all able to work from home and keep up with the development efforts? It does provide a good chance for us emeritus people to draft responses to various forum of interest.
Regarding the interest in M&E, either individually or jointly, my concern is that the value of the exercise is only as good as the questions being asked, the way the data is tabulated, and the finances available to implement the M&E program, particularly within a host governments with limited tax base to support any services including development activities. Associated with this is the stated vs. underlying objective of the M&E effort. As I understand M&E it is designed to be an independent effort on behalf of the underwriting taxpayers to assure the development money is well invested and not wasted as well as guide future projects to more effectively address the objective and allow projects to effectively evolve to better serve the intended beneficiaries. It should not be a propaganda tool to promote projects as successful when by all normal business standards, they are complete failures. As I have listened to many of the USAID MEL (Monitoring, Evaluation, & Learning) webinar and reviewed numerous project reports I am left with the distinct impression the MEL effort is primary to deceive the American public and the elected members of Congress into thinking they are making major contributions to rural poverty alleviation of smallholder farmers, while most of the intended beneficiaries are avoiding active participation like it were the plague, or perhaps the current coronavirus. This may effectively attract continued funding but does nothing for beneficiaries other than keep them deeply entrenched in poverty. More likely M&E activities diverted to project propagandizing will have a substantial negative impact as it reinforces failed programs into future programs preventing them for evolving to better serve the beneficiaries.
Allow me to illustrate use poverty alleviation for smallholder farmers as reference beneficiaries:
Missed question: One question that has been overlooked for most of the last 50 years is the timing of agronomic activities starting with crop establishment. This gets to the limitations of Agronomy research which does an excellent job of determining the physical potential of an area but says nothing about the operational requirements to extend the small plot research across the rest of the farm or community. The assumption is that labor or contract mechanization is readily available, and the farmers only need to be “taught” the value of early planting. A valid M&E effort 50 years ago seeking information including simple field observations on the timing of agronomic operations would have noted that under manual operations crop establishment was spread over 8 weeks or more with rapidly declining yields with the delay to the point it is impossible for manual agriculture to meet basic family food security. If the M&E programs did pick that up the whole poverty alleviation effort would have shifted from badgering smallholders on the importance of early planting and focus on providing access to the operational requirements such as access to contract tillage that would make timing of crop establishment discretionary. It would also note to the extent the current emphasis is on value chain development as a means to promote additional production with the underlying assumption there is surplus operating capacity, is premature until after the operational capacity is increased so farmers could get their crops planted in a more timely manner, produce enough to meet family food security and still have ample production to justify the improved value chain.
Webpage References:
https://webdoc.agsci.colostate.edu/smallholderagriculture/OperationalFeasibility.pdf
https://smallholderagriculture.agsci.colostate.edu/integration-an-under-appreciated-component-of-technology-transfer/
https://smallholderagriculture.agsci.colostate.edu/most-effective-project-enhancing-access-to-contract-mechanization-via-reconditioned-used-tractors/
https://webdoc.agsci.colostate.edu/smallholderagriculture/BrinksDrudgery.pdf
The failure to identify the operational limits of smallholder farmers has a major impact on the current emphasis for quality nutrition. Here the underlying question is what are the calories needed to optimize the economic opportunities which are largely associated with heavy manual agronomic field work? This requires at least 4000 kcal/day but with the manual agriculture most farmers are limited to 2000 to 2500 kcal/day barely meeting the 2000 kcal/day basic metabolism, with only enough work calories for a couple diligent hours of manual labor. That may go a long way to explaining the 8-week crop establishment time. But it does mean that it will be difficult to accept a diversified diet if it requires less calories which in turn will reduce their economic opportunity including crop production. However, the nutrition M&E emphasis the impressive number of “beneficiaries” informed without looking at their compromises in being able to utilize the information and the affordability of the improved nutrition vs. available income. The whole concept of dietary requirement to meet economic opportunities seems completely lost to the nutrition improvement effort but certainly not the “beneficiaries” being badgered with nutritional information they cannot use. How often are our proposed agronomic interventions more labor intensive and thus an attempt to obligate hungry exhausted smallholders to exert energy will in excess of their available calories, possibility as much as twice their available calories. When doing so have we met the definition of genocide or come very close to doing such?
Webpage References:
https://webdoc.agsci.colostate.edu/smallholderagriculture/ECHO-Diet.pdf
https://smallholderagriculture.agsci.colostate.edu/ethiopia-diet-analysis/
https://webdoc.agsci.colostate.edu/smallholderagriculture/DietPoster.pdf
https://smallholderagriculture.agsci.colostate.edu/1028-2/
https://smallholderagriculture.agsci.colostate.edu/affordability-of-improved-nutrition-while-optimizing-economic-opportunities/
Data Manipulation: Even if quality M&E data is collected it could be tabulated either as a propaganda tool to promote programs even if they are basically a failure or as a guide the evolution of future programs. There basically two ways of tabulating M&E data. If you are interested in using M&E for propaganda promotions of the project regardless of how successful it is you simply report to aggregate totals of the data. With a large multi-nation with multi-programs within a nation it can easily provide some highly impressive number, often in the hundreds of thousands. This will appease the public perhaps assure future funding but be meaningless as evaluation tool. A more effective evaluation tool for guiding future project would the expressing the same data as a percent of the potential. Using my pet concern of the overemphasis on farmer organizations to funnel assistance to smallholder farmers. It is possible and the USAID MEL program routinely does, claim they are assisting several hundred thousand smallholder farmers, but a more detailed analysis would indicate rarely do they have more than 10% of the potential farmers, within the smallholder communities they are claiming to assist, actively participating and even then they will divert most of their business to alternative service providers. Thus, they may be assisting a few hundred thousand but have a few million potential beneficiaries avoiding the project. Then since most of the active members will side sell all but what is needed for loan repayments, the total market share from the community will be a trivial less than 5% and virtually no impact on the overall communities’ economy. Not what would be considered a success by most business standards!! What is urgently needed here is an upfront statement of what will constitute a minimal successful project in terms of percent of potential beneficiaries actively participating in the project and the market share funneled through the project. All expressed in percent of the community.
This target for a success project needs to be consistent with what the underlying taxpayers are expecting. In the case of farmer organizations, the expectation is over 60% participation and over 50% market share. Can anyone come close to this level of success particularly when projects are openly attempting to compete with private service providers?? A good sincere M&E program, either individual or together, should have realized this some 30 years ago and if so the development effort would have carefully looked at alternatives, including accepting that the much vilified private service providers were effective efficient business models that actually provide the farmers the financially best and most convenient business services. Also, it would not take a lot of effort to appreciate the farmer organization model is administratively too cumbersome to compete with private service providers. The overhead costs of operating a farmer organization substantially exceeded the private service profit margins so relying on farmer organization would drive the smallholder members deeper into poverty despite the massive rhetoric about poverty alleviation. Thus, the farmers wisely avoid the farmers organization so that they require continuous external support and facilitation and fully collapse once external support ends.
Webpage references:
https://smallholderagriculture.agsci.colostate.edu/perpetuating-cooperatives-deceptivedishonest-spin-reporting/
https://smallholderagriculture.agsci.colostate.edu/appeasement-reporting-in-development-projects-satisfying-donors-at-the-expense-of-beneficiaries/
https://smallholderagriculture.agsci.colostate.edu/relying-on-cooperatives-taxpayers-expectations/
https://smallholderagriculture.agsci.colostate.edu/loss-of-competitive-advantage-areas-of-concern/
https://smallholderagriculture.agsci.colostate.edu/farmers-organizations-and-cooperatives-is-there-a-competitive-adantage/
Financial Limits: The concern with financial limits is that host government are working with a limited tax base to provide support services for which the administrative nature of a national M&E program just cannot get the priority needed for quality data collection. The overall economic environment in most host countries is what I refer to as financially suppressed. This is an economy with such a high level of poverty that most families spend 80% of wages or production just to obtain a meager diet for their family even when the consumer prices are only 1/3rd to 1/5th the USA or EU prices. With that percent of income devoted to survival there just is not enough “discretionary” income to make an effective tax base for the government to raise funds for civil services. Sorry but no taxes no services. The result is that for all practical purposes the government is financially stalled. I think this is why, in previous comments, most of the M&E was done through NGOs with access to external funds. The problem is when you try and compel a government to undertake an administrative task that cannot be fully funded including the operational costs for the field trips necessary to collect reliable information, the quality and reliability of the task become questionable. With no funds but pressure to provide the data, civil officers will often simply complete the information as best they can according to what they perceive is taking place. It is the best they can do but could be far from reality. It should be noted that virtually 90% of all smallholder farmers in most host countries have never interacted with a civil officer including village agricultural extension officers. It is really better not to ask for a M&E program than have one completed by perception instead of facts.
Webpage References:
https://smallholderagriculture.agsci.colostate.edu/financially-suppressed-economy-2/
https://smallholderagriculture.agsci.colostate.edu/financially-stalled-governments/
https://smallholderagriculture.agsci.colostate.edu/impact-of-financially-stalled-government-limited-variety-improvement-seed-certification/
The bottom line of this comment is to be very careful and make certain any M&E program alone or combined is mechanism for representing the beneficiaries and advancing the development effort and does not become a means for embellishing programs that may be socially desirable but for which the beneficiaries are avoiding like the current virus. It will be very difficult to meet the SDG when the M&E activities are more promoting and instituting failed programs instead of guiding the evolution of programs to better serve the beneficiaries.
Webpage Reference:
https://smallholderagriculture.agsci.colostate.edu/monitoring-evaluation-the-voice-of-the-beneficiaries/