Another interesting and important discussion topic. As I look at the comments to date, they remain mostly confined to the academic/research community and do not get to the final beneficiary of the scientific results. For the CIGAR this can be critical as the CIGAR is considerable isolated from the intended final beneficiaries. That is the primary clients for the CIGAR are the Host Country National Agriculture Research Systems (NARS), while the final beneficiaries are multitude of normally unnamed smallholder farmers, who are still one step removed from the CGIAR’s NARS clients. That would be the national agricultural extension program. These final beneficiaries usually have no direct access to the research results and cannot afford the referred technical journals mentioned as the primary product for the research.
Thus, while I think the CGIAR does an excellent job of basic research particularly regarding varietal improvement for many host countries, I do question how effectively the results can be utilized by the smallholder final beneficiaries. I think most of the varietal improvement in developing countries such as those in Sub Sahara Africa are facilitated by “collaborative” programs between the appropriate CGIAR center and the host NARS. However, the collaboration is supported by external funding to the CGIAR team to cover the operating costs. Thus, it becomes more a CGIAR effort than a fully collaborative effort. From a smallholder operational perspective, the varietal improvement program is the CGIAR most effective research intervention. The reason being it is a simple substitution for what smallholders are already doing, with limited, if any, additional labor required. There may be some substantial logistical requirements in getting newly released variety seed available to smallholder, and logistic can be a major hindrance in many host countries.
The problem comes when you get away from the varietal improvement effort and work with innovations with a labor or other higher operational requirement. Then the limits of small plot research, the basis of most agronomic analysis, becomes a problem. While small plot research does a tremendous job of determining the physical potential of a research innovations in the region it is undertaken and can produce high quality referred journal papers well appreciated by the academic/research community, it does not address the operational requirements, such as labor, necessary to extend the results across a smallholder community. It just assumes it is not a problem. However, labor can be highly limited in most smallholder communities as well as the dietary energy to fuel the labor. How often are the CGIAR’s non varietal improvement research innovations more labor intensive than what the smallholders are currently doing? The question is who within the CGIAR collaborative effort to assist smallholder farmers is responsible to:
Does this fall into an administrative void between the agronomists or other applied bio-scientists and the social scientists assisting small holder communities? Until this is recognized and addressed will the high quality CGIAR center research receive limited acceptance by the smallholder farmers? It is very interesting that the Baker/Hurd Yield Gap analysis started at IRRI some 40 years ago has never address labor as major contributor to the yield gap analysis. I think it would make a major difference and explain much of the yield gap and low level of research acceptance for the CGIAR quality research results.
The problem is exasperated by limited dietary energy available to most smallholder farmers resulting in research & extension attempting to compel smallholder farmers to exert up to twice their available dietary calories. It is interesting that we acknowledge that smallholder farmers are poor and hungry but never factor that as a major hindrance to their ability to take advantage to the quality research being done for their benefit. It is equally interesting that there is very little hard data on the calories available to poor hungry smallholders, let alone how that compares to the 4000 kcal/day needed for a full day of agronomic fieldwork. What little is available typically shows smallholder farmers have access to only about 2000 – 2500 kcal/day, barely providing the basic metabolism requirements, leaving little energy for field work such as the 300 kcal/hr. needed for basic manual land preparation. The result is to prolong periods of crop establishment against the declining yield potential associated with delayed crop establishment until it is no longer possible to meet family food security needs. This will again severely limit the usefulness of the high-quality research coming from the CGIAR centers and the collaborating NARSs, for much of agronomic research the effectiveness is time sensitive.
Mention is made of the MEL evaluation process. This needs to be review with care to make certain it is true evaluation process that guides future projects to better serve the beneficiaries with more effectively programs and not a propaganda tool to cover-up and promote failed programs as often appear to be the case. The concern is in both the criteria being included or excluded in an MEL evaluation and use of aggregate analysis vs percentage analysis. For example, how often do MEL analysis for agronomic programs include timing of field operations that can be highly visible and would pick up the labor constrains mentioned above, and then guide programs to facilitate smallholder access to contract mechanization that would expedite crop establishment, improve timing, compliance with research recommendations, yields, and enhance family food security? I have never seen it included. Looking at USAID MEL effort on reliance on producer organizations to assist smallholder, when you do an aggregate analysis, you come up with some very impressive numbers that only measures the massiveness of the total program, while saying little or nothing about effectiveness or appreciation of the effort to the beneficiaries. However, if shift the same data to percent such as:
the impact on the individuals and communities can be trivial. In this case the MEL will represent a little bit of monitoring, but no real evaluation, and the only learning is “how to deceive the underwriting taxpayers”. However, such an MEL analysis assures the continuation and entrenchment of programs the beneficiaries are avoiding like the black plague or perhaps in today’s context COVID-19. This would really be a major disservice the beneficiaries, while assuring the implementer future opportunities.
Please allow me to support the above concerns with some webpages from the Smallholder Agriculture website I manage.
For operational limits and dietary energy balance:
Soil & Crops Sciences
Colorado State University
As I review the various contributions to this important discussion, I note how often the problem centers around financial resources. I think this is often critical but needs to be considered in the overall economic context of most host countries. As I look at the overall economic environment of host countries, I defined them as economically suppressed. That is, they are serving a mostly impoverished population that spend up to 80% of income or farm production for family essential food needs. This leave very little discretionary income to purchase other necessary goods and form a tax base to provide the government revenue to support government services. No Taxes/no services. Thus, most host country civil services are barely able to provide personnel benefits in term of salaries, retirements, health care and possible housing to the civil officers. This leaves little or no funds for operational cost such as travel to field locations for conducting a M&E analysis of projects. Thus, they have little choice but to rely on references experiences on the effectiveness of various innovation, which may or may not be accurate, or even more propaganda than analytical. Also, under these financial restrictions, perhaps it is better to assume effectiveness and use scarce financial resources to promote other innovations. Please review the following webpages:
Again, we have another interesting, critical, and challenging topic to comment on. As I have previously stated, the most important contribution of any project evaluation is the guidance it provides to future project so they can better serve the beneficiaries. Unfortunately, too often evaluations become propaganda tools necessary to appease donors and assure future projects, while doing little if anything for the intended beneficiaries.
Thus, while the commentary shows a good consensus on the importance of Neutrality-impartiality-independence, the commentary also shows that obtaining such can be very challenging as rarely do projects provide funds for external reviews by independent evaluators, but if when funds are available, vested interest in potential future assignments, will often bias the objectively needed to be sufficiently critical of results to guide future projects.
One way to minimize the bias might be to have some clear well-defined targets that will separate project success from failure. Has anyone ever seen a set of evaluation criteria that would do this? I have not! This set of criteria needs to be established at the beginning of a project. They also need to be close to what interested underwriting taxpayers are expecting and reflected in the project reporting. They may also need to be expressed in percentages of potential, instead of aggregate number. As when your emphasis aggregate numbers you can generate some very impressive but meaningless value that only reflect the massive size of projects and investments, while still having a trivial total impact. This is basically what the USAID MEL evaluation reflects. Instead, emphasis on percentage of the potential will give a better evaluation of project effectiveness. Also, there is a need to remember that most project are defined by the community they serve, and thus the evaluation needs to reflect community impact more than individual impact.
Allow me to illustrate with my favorite development concern. That is the nearly 40 years of over reliance on producer organizations to funnel assistance to smallholder farmers. While I agree producer organizations may be socially desirable, but they are also administratively cumbersome which quickly converts to overhead costs, that can more than consume the overall financial benefits so relying on the producer organizations will force smallholder farmers deeper into poverty. This saying nothing about the super inconvenience of consignment marketing in a highly cash-oriented society. Thus, the famers widely and wisely avoid them, so that they require continuous external facilitation and subsidies to exist, then collapse once external support end. Yet the smallholder agriculture development effort is totally committed to imposing them on the communities, and claiming they are the essential salvation for smallholders, as shown by the rhetoric accompanying any presentation that would lead one to believe anyone was foolish not to participate.
My approach is prior to my international agriculture class discussion on producer organizations, I always asked the students, many of whom were interested in international development with plans to join the Peace Corps, to list what they thought would be the minimum values for various business parameters associated with Producer Organizations and compared them to the best estimate I could obtain scrutinizing various reports with some simple computation. The results are in the webpage: https://agsci.colostate.edu/smallholderagriculture/request-for-information-basic-business-parameters/.
The expectation is for producer organizations to enroll well over 50% of the potential beneficiaries and they market through the organization 70% of their produce while side selling less that 10%. The actual result is only about 10 - 15% of the farmers participated, and even those will side-sell the bulk of their produce when possible, so the total impact on the community is a trivial >5%. Not what you would really want as a successful project. Please take a few minutes and review the webpage and some of the linked pages and provide comment back to this forum. Are the:
I feel the persistence in relying on producer organization represents a lack of or compromised independent evaluations. The limited acceptance by smallholders should have been identified decades ago, and the development effort moved on to identify and implement more effective options. This then represents donors with no sincere interest in assisting the plight of smallholder producers, but fully committed to imposing this horrible business model on them, that fortunately they are not gullible enough to fall for. The best you can say for the practice is it shows an easily publicized good intention without accomplishing anything. My bottom line on producer organizations is a real multi-decade, multi-billion-dollar scandal, with some possible serious liabilities for the cover-up. Am I wrong? If there had been clear target as what constituted success vs. failure could this have been avoided?
Some additional webpages that go into more detail are:
Silva, I can only endorse your concern that an evaluation as to include evaluating the TOR. I would say that is the most important part of an evaluation as only when you evaluate the TOR can you provide guidance for future projects to better serve the beneficiaries. Too often evaluations, particularly when internally, become propaganda tool to promote the TOR and the project. This can then become a major disservice to the intended beneficiaries as it will reinforce TOR and projects that are basically failures and provide no benefits. I think this is the case with the 40 years of reliance on Producer organization to funnel assistance to smallholder farmers. A close look will show these are really a scandal, attracting only a very small percent of the potential participants, and even then the members divert most of their business to other traders. Please review the following webpages:
From my interest the most important impact of evaluations is the guidance they provide for future projects in guiding them in evolving to better serve the intended beneficiaries. Unfortunately, at least for smallholder agriculture development projects, the evaluations have become more a mechanism for propagandizing projects regardless of how well they are relied upon or avoided by the beneficiaries.
To some degree this is necessary as the up-front cost in getting projects from conception to implementing contract and finally an opportunity for detailed discussions with beneficiaries can be a couple million dollars and over two years of extensive time and effort by a multitude of people. With that much up-front costs before the implementer have a chance for detailed interviews with beneficiaries to fully ascertain their needs, no one wants to learn that the beneficiaries were not really interested in the activity and mostly avoiding the project for various valid reasons.
However, when used more for propaganda then guidance, there may be some benefits to the implementer in terms of project extensions and future projects, but no real benefit for the beneficiaries. Instead, projects, that by most standards would be considered a failure, become more deeply entrenched in the future development projects, reducing any effort to adjust projects, and squandering massive amounts of development investments that could better be used in an alternative approach.
Please review the following webpage:
Preventing an evaluation effort become more a propaganda tool instead of guidance for future projects may be more how data is analysis than what is collected. This could be as easy as avoiding aggregate analysis in favor of more percent analysis. In dealing with smallholder farmers, each managing one to two hectares, one can quickly count some impressive numbers of individual participating and get a good propaganda appraisal, but it will say nothing about what would be possible. However, if expressed in terms of percent of potential beneficiaries, the impressive aggregate number could become highly questionable. The same could be said about the amount of produce marketed through the project. The total volume can sound impressive, but if prorated to individual members could be only a small percent of their production, perhaps representing only in-kind loan repayments, with the bulk of their business going to the very people the project is attempting to replace to provide the beneficiaries a better business deal. If converted to a community basis, which is what development projects are based on, it would be nearly a trivial amount.
The real need for an evaluation is, before beginning the evaluation to develop a set of targets for what would separate success from failure and do this as percent rather than an aggregate number. This would be the percent of the farmers activity participating in program, or the percent of farmers produce marketed through the project, as well as the percent of the communities produced marketed through the project. For example: 70% of the farmers activity participating instead of 150 famers involved, or 80% maize production marketed through the project with only 20% side-sold. This could easily give a very different perspective of how well the project was received and relied upon by the smallholder beneficiaries and result in make some major adjustment to future projects that would better serve the beneficiaries.
Please review the following webpages:
I very much appreciate the interest in localized project design, but consider it a real logistic challenge to obtain. The problem is related to the lead time and cost associated with bringing a project on-line so you can extensively meet and discuss the needs of the local beneficiaries. Typically from conception to contract implementation takes at least 2 years with the up-front investment of over a million US$. During this extended period most of the critical decisions are made regarding the type of innovation to be considered and staff hired to accommodate this. Thus by the time you have the opportunity for detailed discussion on needs, there is little flexibility in approach. Also, once implemented the MEL information to evaluated the program is often tilted to appease the donor as needed to secure contract extensions and future funding.
Please review the following webpages:
With regards to your comment below:
"I think we need to factor as well that the cooperative members derive non-monetary benefits in terms of opportunities for group sales, bulking produce, storage, all of which offer increased chances to attract higher prices and avoid gluts in the market, thereby improving the welfare of members. Other latent benefits such as timely access to inputs on credit basis, legal protection through enabling legal and policy environment made possible by the engagement power of the cooperative with the policy decision-makers that be, also exist for members."
Please note that yes these benefits are true but at a cost and the question is how often to the cost to obtain these benefits exceed the value of the benefits so relying on the cooperative model will force smallholder farmers deeper into poverty. Have you ever seen the detailed accounting of the overhead costs to obtain the stated benefits? I have been carefully looking for them for over a decades and have never seen overhead cost factored into the promotion of the cooperative model.
Please allow me to continue my comment and provide some references on how Knowledge Management to appease donors is detrimental to smallholder beneficiaries. Please review the webpage:
Are these the business parameters that will determine if a program is effective in assisting smallholder or not, why is this information just not available and should it be mandatory in M&E evaluations, that is unless the objective is not to assist the smallholder beneficiaries our of poverty but more to impose on them the horrible cooperative business model even if it drives them deeper into poverty.
Also, review the following webpages and see how fast the cooperative model can lose its envisioned competitive advantage.
Knowledge Management can be interesting as you can take the same data (Knowledge) and manage it in two simple but different way with polar opposite results, one propaganda/promotional while stagnating programs claiming success, and the other evaluation recognizing limited success and leading to program evolution and refinement. The two management tools are aggregate analysis vs. percentage analysis.
With aggregate analysis, you simply sum up the number of people involved, the total goods marketed, the total value of the good, etc. With a large program spread over an entire country or region, you can generate some very impressive numbers. This appears like a great program with no need to make any adjustments and has stagnated program conceptualization for several decades. This is good for promotion and perhaps getting future funding from legislatures but for future guidance purposes the results are meaningless.
However, from an evaluation perspective the numbers are meaningless, as they provide no comparison as to what potential could have been. This can be accomplished by taking the same data and concentrating on percentage analysis such as percent of potential beneficiaries actively participating, the percent market share vs. side selling, percent income improvement, etc. This percentage analysis could be strengthened with clearly state targets that will separate success from failure and such targets are consistent with the expectation of the underwriting taxpayers, as noted in previous mentioned webpage: https://smallholderagriculture.agsci.colostate.edu/request-for-information-basic-business-parameters/ Thus, the impressive aggregate numbers when expressed as a percent of potential could represent a trivial impact and need for major program adjustments, that is if there is a sincere interest in assisting smallholder beneficiaries out of poverty, and not imposing on them some horrible business model that more likely will force them deeper into poverty, and for which they very wisely and astutely avoid. Please review the Ethiopian coffee example in the webpage: https://smallholderagriculture.agsci.colostate.edu/perpetuating-cooperatives-deceptivedishonest-spin-reporting/
Unfortunately, when I see an impressive aggerate analysis I think it represents a cover up and program failure. As an American I have to careful take note of the USAID MEL process. This stands from Monitoring, Evaluation and Learning, but is really just great aggerate analysis that may have some monitoring component, but no real evaluation, and the only learn is how to deceive the American public. To me this is a deliberate effort to deceive the American Public into thinking their international assistance program is making major accomplishments, while most potential beneficiaries are avoiding the programs like the plague or the COVID-19 pandemic. I am not certain that some of this should be reviewed through the court system. https://smallholderagriculture.agsci.colostate.edu/vulnerability-for-class-action-litigation-a-whistleblowers-brief/
I get concerned when knowledge is managed to promote innovations that may not be well received by the beneficiaries. Here instead of objectively addressing the needs of the beneficiaries the objective is to appease the donors and this my be essential to secure project extensions and future projects. The case in point would be continued high reliance on farmer organizations, when an objective knowledge management effort would have shown they were to administratively cumbersome to be competitive, and the development effort would have steered clear of them a couple decades ago. Please review the webpage and linked other pages: https://smallholderagriculture.agsci.colostate.edu/appeasement-reportin…
This is a real knowledge management scandal wasting bullions of development dollars, euros, etc. each year.
I hope everyone is healthy and take good care of yourselves and families, particularly our colleagues working in Rome. Are you all able to work from home and keep up with the development efforts? It does provide a good chance for us emeritus people to draft responses to various forum of interest.
Regarding the interest in M&E, either individually or jointly, my concern is that the value of the exercise is only as good as the questions being asked, the way the data is tabulated, and the finances available to implement the M&E program, particularly within a host governments with limited tax base to support any services including development activities. Associated with this is the stated vs. underlying objective of the M&E effort. As I understand M&E it is designed to be an independent effort on behalf of the underwriting taxpayers to assure the development money is well invested and not wasted as well as guide future projects to more effectively address the objective and allow projects to effectively evolve to better serve the intended beneficiaries. It should not be a propaganda tool to promote projects as successful when by all normal business standards, they are complete failures. As I have listened to many of the USAID MEL (Monitoring, Evaluation, & Learning) webinar and reviewed numerous project reports I am left with the distinct impression the MEL effort is primary to deceive the American public and the elected members of Congress into thinking they are making major contributions to rural poverty alleviation of smallholder farmers, while most of the intended beneficiaries are avoiding active participation like it were the plague, or perhaps the current coronavirus. This may effectively attract continued funding but does nothing for beneficiaries other than keep them deeply entrenched in poverty. More likely M&E activities diverted to project propagandizing will have a substantial negative impact as it reinforces failed programs into future programs preventing them for evolving to better serve the beneficiaries.
Allow me to illustrate use poverty alleviation for smallholder farmers as reference beneficiaries:
Missed question: One question that has been overlooked for most of the last 50 years is the timing of agronomic activities starting with crop establishment. This gets to the limitations of Agronomy research which does an excellent job of determining the physical potential of an area but says nothing about the operational requirements to extend the small plot research across the rest of the farm or community. The assumption is that labor or contract mechanization is readily available, and the farmers only need to be “taught” the value of early planting. A valid M&E effort 50 years ago seeking information including simple field observations on the timing of agronomic operations would have noted that under manual operations crop establishment was spread over 8 weeks or more with rapidly declining yields with the delay to the point it is impossible for manual agriculture to meet basic family food security. If the M&E programs did pick that up the whole poverty alleviation effort would have shifted from badgering smallholders on the importance of early planting and focus on providing access to the operational requirements such as access to contract tillage that would make timing of crop establishment discretionary. It would also note to the extent the current emphasis is on value chain development as a means to promote additional production with the underlying assumption there is surplus operating capacity, is premature until after the operational capacity is increased so farmers could get their crops planted in a more timely manner, produce enough to meet family food security and still have ample production to justify the improved value chain.
The failure to identify the operational limits of smallholder farmers has a major impact on the current emphasis for quality nutrition. Here the underlying question is what are the calories needed to optimize the economic opportunities which are largely associated with heavy manual agronomic field work? This requires at least 4000 kcal/day but with the manual agriculture most farmers are limited to 2000 to 2500 kcal/day barely meeting the 2000 kcal/day basic metabolism, with only enough work calories for a couple diligent hours of manual labor. That may go a long way to explaining the 8-week crop establishment time. But it does mean that it will be difficult to accept a diversified diet if it requires less calories which in turn will reduce their economic opportunity including crop production. However, the nutrition M&E emphasis the impressive number of “beneficiaries” informed without looking at their compromises in being able to utilize the information and the affordability of the improved nutrition vs. available income. The whole concept of dietary requirement to meet economic opportunities seems completely lost to the nutrition improvement effort but certainly not the “beneficiaries” being badgered with nutritional information they cannot use. How often are our proposed agronomic interventions more labor intensive and thus an attempt to obligate hungry exhausted smallholders to exert energy will in excess of their available calories, possibility as much as twice their available calories. When doing so have we met the definition of genocide or come very close to doing such?
Data Manipulation: Even if quality M&E data is collected it could be tabulated either as a propaganda tool to promote programs even if they are basically a failure or as a guide the evolution of future programs. There basically two ways of tabulating M&E data. If you are interested in using M&E for propaganda promotions of the project regardless of how successful it is you simply report to aggregate totals of the data. With a large multi-nation with multi-programs within a nation it can easily provide some highly impressive number, often in the hundreds of thousands. This will appease the public perhaps assure future funding but be meaningless as evaluation tool. A more effective evaluation tool for guiding future project would the expressing the same data as a percent of the potential. Using my pet concern of the overemphasis on farmer organizations to funnel assistance to smallholder farmers. It is possible and the USAID MEL program routinely does, claim they are assisting several hundred thousand smallholder farmers, but a more detailed analysis would indicate rarely do they have more than 10% of the potential farmers, within the smallholder communities they are claiming to assist, actively participating and even then they will divert most of their business to alternative service providers. Thus, they may be assisting a few hundred thousand but have a few million potential beneficiaries avoiding the project. Then since most of the active members will side sell all but what is needed for loan repayments, the total market share from the community will be a trivial less than 5% and virtually no impact on the overall communities’ economy. Not what would be considered a success by most business standards!! What is urgently needed here is an upfront statement of what will constitute a minimal successful project in terms of percent of potential beneficiaries actively participating in the project and the market share funneled through the project. All expressed in percent of the community.
This target for a success project needs to be consistent with what the underlying taxpayers are expecting. In the case of farmer organizations, the expectation is over 60% participation and over 50% market share. Can anyone come close to this level of success particularly when projects are openly attempting to compete with private service providers?? A good sincere M&E program, either individual or together, should have realized this some 30 years ago and if so the development effort would have carefully looked at alternatives, including accepting that the much vilified private service providers were effective efficient business models that actually provide the farmers the financially best and most convenient business services. Also, it would not take a lot of effort to appreciate the farmer organization model is administratively too cumbersome to compete with private service providers. The overhead costs of operating a farmer organization substantially exceeded the private service profit margins so relying on farmer organization would drive the smallholder members deeper into poverty despite the massive rhetoric about poverty alleviation. Thus, the farmers wisely avoid the farmers organization so that they require continuous external support and facilitation and fully collapse once external support ends.
Financial Limits: The concern with financial limits is that host government are working with a limited tax base to provide support services for which the administrative nature of a national M&E program just cannot get the priority needed for quality data collection. The overall economic environment in most host countries is what I refer to as financially suppressed. This is an economy with such a high level of poverty that most families spend 80% of wages or production just to obtain a meager diet for their family even when the consumer prices are only 1/3rd to 1/5th the USA or EU prices. With that percent of income devoted to survival there just is not enough “discretionary” income to make an effective tax base for the government to raise funds for civil services. Sorry but no taxes no services. The result is that for all practical purposes the government is financially stalled. I think this is why, in previous comments, most of the M&E was done through NGOs with access to external funds. The problem is when you try and compel a government to undertake an administrative task that cannot be fully funded including the operational costs for the field trips necessary to collect reliable information, the quality and reliability of the task become questionable. With no funds but pressure to provide the data, civil officers will often simply complete the information as best they can according to what they perceive is taking place. It is the best they can do but could be far from reality. It should be noted that virtually 90% of all smallholder farmers in most host countries have never interacted with a civil officer including village agricultural extension officers. It is really better not to ask for a M&E program than have one completed by perception instead of facts.
The bottom line of this comment is to be very careful and make certain any M&E program alone or combined is mechanism for representing the beneficiaries and advancing the development effort and does not become a means for embellishing programs that may be socially desirable but for which the beneficiaries are avoiding like the current virus. It will be very difficult to meet the SDG when the M&E activities are more promoting and instituting failed programs instead of guiding the evolution of programs to better serve the beneficiaries.
Another issue I have with getting farmer participation is, too often it is virtually essential to leverage their final assessment to conform with what a project has to offer. It must be recognized that before you can solicit effective farmer input, the project will be 2 or more years past conception with over a million US$ expended. That is the typical gestation period from initial conception to issuing an implementing contract and fielding an implementing team, specifically geared to the innovation incorporated in the project proposal and contract. With that much time, effort and money already spent no one wants to hear the farmers are not interested in the innovation and the implementing team as no alternative to leverage the results to the predetermined innovation or go home. The result is very small percent of potential beneficiary’s actual participate, well below the 60+% the underwriting taxpayers assume are participating and those participating only rely on the project of minimum benefits. Ultimately, the projects require continuous external facilitation and collapse immediately after the external support ends leaving little, if any, lasting impact.
Please review the following webpages and links:
Effectively involving farmers in the participatory process has always been a challenge both for diagnostic and evaluation. However, much of this might be due to the tendency of relying on the interview process to solicit their input. Thus, it might be noteworthy that much of agronomy is highly visible and thus it can be easily and perhaps more accurately passively collected with some good field observation. Once the farmer plants the seeds, he has a several month commitment the is easily seen for the entire cropping season. This be good for identifying the crops being produced, and the varieties being grown. With some good observations or questions, it would be possible to get planting dates and determine the biggest oversight in agronomy and smallholder development. This is the timing of crop establishment and other time sensitive activities. The overall time spread being a critical indicator of the severe operational limits smallholders face in terms of limited labor, energy to fuel that labor sufficiently for a full day of manual agronomic field work, or access to contract mechanization. Once this oversight is recognized it hopefully will be addressed instead of being ignored for over 40 years.
With the quality of Google earth and other satellite imagery it is now easily possible to get high enough resolution imagery of project areas to easily plot where target crops are produced, measure the hectarage involved and sum that up get percent of acceptance. Likewise, get an estimate of planting date and you can quickly plot a cropping calendar to see the overall cropping pattern of the project area, including the time spread in activities. This can be both a diagnostic tool and monitoring tool. While all this may have only minimal direct connection with the farmers, it should give you a quick appreciation as how effective your project is. Please see the example attached of a crop calendar that was developed to document actual irrigation use in Egypt as a guide for bring irrigation issues in line with actual use.
Of course, you will need to have targets as to what will qualify as a successful vs. unsuccessful project and this will need to be in line with what your underwriting taxpayers funding the project expect and will accept as success.
If the operational limits mentioned above were identified 30 or 40 years ago our overall emphasis would have shifted from concentrating on specific crops and crop management to facilitating access to contract mechanization, expediting crop establishment sufficiently to meet food security with enough surplus production to accommodate the value chain we promote as a means of stimulating production.
As I look at the theory of change (TOC), particularly as it applies to development and more particularly smallholder agriculture, I wonder if we are getting an accurate analysis of its effectiveness, or is it getting caught in the implementers' need to appease the donor to assure project extensions and future projects. This results in projects appearing far more successful than they really are with a lot money invested for limited benefit and the intended beneficiaries being left with little impact.
I think a lot of the TOC being applied to smallholder communities is derived from academia and based more on what is socially desirable than effectiveness in sustained change and implying a wider use in the donor home country than reality which is misleading to the host country. Much of the TOC being imposed on smallholder communities is based on organizing and relying on farmers organizations or peopleware instead of hardware or software. The idea is people can be organized to work in their collective vested interest, even when this conflicts with individual vested interest. Unfortunately, individual vested interest usually will eventually take priority, so these TOCs require continued external facilitation, if not direct subsidies to survive and collapse as soon as the last advisors departs, perhaps before the they clear the departure lounge for the flight home.
The best example of this is the nearly 40 years imposition of cooperatives to provide business services to smallholders in terms of consolidated inputs and markets. This is done with blanket, but never substantiated thus slanderous, vilification of private traders. This is imposed even while the cooperative movement in the USA, which had some major positive impact a century ago, has been in decline for several decades and when last reported 20 years ago represented less than 30% of agriculture business activity. I would image the same would be true for cooperatives in the EU or other donor bases. The problem with the cooperative model is that it is administrative cumbersome which translates to high overhead costs that, in the financially suppressed developing world, will quickly exceed the financial benefits of bulking inputs and marketed produce. When this happened relying on cooperatives will force smallholder farmers deeper into poverty. The result is that cooperative based projects only attract a small percent of the potential beneficiaries, perhaps 10 to 12%, and even then, the bulk of the members production is side sold to the vilified private traders, leaving the cooperative with little more than in-kind loan repayments. Not what you could objectively refer to as a successful project. Yet, with some creative accounting they are always considered successful at least while someone is facilitating them.
Another example is the use of Water User Associations (WUA) to manage irrigation water and maintain canals. They are based on the Ditch Companies of Colorado (where I am writing from) and other western USA states. Ditch companies’ success is based on very strict and rigidly enforced water law that is not available in most irrigation schemes in which WUA are imposed. Thus, while there is a common vested interested in maintaining irrigation canals, the individual vested interest does a 180-degree shift once you pass the individual’s inlet. Again, they can exist while there is external assistance but will collapse once that ends.
Please review the following webpages including internal links:
I can not but agree with John Weatherhogg analysis that access to drinking water is a critical factor that can free women to assist with more economic activities. Really all forms of domestic drudgery relief should be addressed as a key to economic assistance. I was very interested to note that the original rural mechanization in many parts of Africa was the introduction of maize mills. I think this has totally replace the heavy pounding of maze and other grains. It is sometime more effective to address some of these side issues than concentrating of crop and animal husbandry, most of which is badly compromised.
Please allow me to elaborate on my previous comment and add a few web references. Carefully consider how technology, particularly agronomic technology, is developed and what it does and what it cannot do. Most agronomic technology is developed though replicated small plot trials. These do a very good job of determining the physical potential of an area and technology, but days nothing about the operational requirements, in terms of labor or access to mechanization, to extend that innovation across the rest of the field, farm or smallholder community, with the regrettable assumption that it is not a problem and the only thing needed is a good extension educational program.
Unfortunately, the operational requirements for agriculture production falls into an administrative void between the agronomist and the social scientists. Who in an agriculture development project is responsible to determine the labor requirements the availability of the necessary labor and the rational compromises crop or animal husbandry when that labor is not available? When you accept these short coming in the development effort you can quickly appreciate that smallholder farmers, both men and women, are massively overextended and cannot take full advantage of the technology promoted for their benefit in the timely manner required. There is really a genocide component of our agriculture assistance effort as we try and obligate smallholder farmers to exert over 4000 kcal/day when they only have access to 2000 to 2500 kcal/day. Resulting in limited work days and prolong time to complete any agronomic activity including up to 8 weeks for basic crop establishment.
Thus the critical need across the board is drudgery relief by any means. This will then expedite crop establishment, yield potential and food security for the entire family. As mentioned in the previous comment relieving the domestic drudgery through better access to water, grain mills, easier access to fuel can free up a lot of time for women to become involved in more economic activities either assisting their partners with farm work or other enterprises, but without reduction in domestic drudgery how much time do women have to become involved in economic activities. Unfortunately, the daily need for domestic chores has to take priority over the economic opportunities.
As I review the continued emphasis on women in agriculture and other development efforts I have a couple overall concerns.
First the emphasis on women empowerment has an undercurrent that most women are in an adversarial relationship with their partners. Thus my question is what percent of women in rural smallholder communities are in a adversarial relationship and in need of assistance vs. what percent are in a more collaborative relationship with their partners? I would think the majority of women are in a more collaborative relationship. This then leads to, if most women are in a collaborative relationship with their partners what percent of the communities women are interested in participating in an empowering activities vs. continuing to operate collaboratively with their partners.
Second, given that domestic chores of child raising, cooking, collecting water and fuel are largely the responsibility of women and these daily activities have priority over economic activities, what percent of women time and energy are consumed in meeting the daily domestic chores, leaving how much time to assist in economic activities either in collaborative with their partners or independent as part of an empowerment project. I would suspect that most of the time and energy are consumed with domestic chores with very little time available for economic activities. This then leads to question as what percent of the women in a community will get involved with empowerment efforts, vs. continuing to work in collaboration with their partners.
Just a couple of initial thoughts.
My biggest concern for all host government activities such as managing evaluations is "is there sufficient financial resources to do an effective job or if not will tasked be highly compromised by officers attempting to complete evaluation based on personal expectations rather than solid fact". The concern is based on most host governments FAO works with are what I refer to as financially suppressed economics where so much of individual wages or labor goes to meet basic subsistence needs that there is not sufficient discretionary funds to form a viable tax based to support government services such as evaluations. This then results in a lot of informal income activities interfering with effective governance. Governments are for all practical purposes financially stalled. Please review the webpages:
Sorry for joining this discussion late, but I have been tied up the last couple weeks.
I think the major reason youth are losing interest in farming is that most manual operated farming in Africa can not provide food security for the family. This get back to an issue that has been overlooked by the development effort but I keep harping on. That is the shortcoming of agronomy, my discipline, in that agronomy does an excellent job of determining the potential of an area, but says nothing about the operational requirements to extend the small plot results across the rest of the area. It just assume it is not a problem and all that is needed is education on improved technology that is often more labor intensive, in what is usually a severe labor deficit environment, in which most people are operating at a 50% calorie deficit. The solution is to remove as much of the drudgery as possible, starting with facilitating access to contract private sector tillage services. Once this is done, the crop establishment time will be substantially reduced, potential yield will go up, food security more assured, with potential to for crop diversification to more nutrient rich foods, as well as greater surplus to market up the value chain. Once the smallholder agriculture can provide a reasonable quality of living the youth will be more interested in staying in the rural areas.
You might want to compare the manual operations in Africa to the paddy lands of Asia, such as Thailand, Laos, Viet Nam, Malaysia where some 30 years ago the made the major shift from water buffalo to power tillers. This more than halved the crop establishment time, leading to higher yields, increased crop intensity to 5 crops in 2 years, etc. and lead to a more middle class living standard. Since this enhancement of the operational capacity was independent of the development effort, it has been overlooked, but I would contend that it is major reason for the success of the "green revolution" in Asia, with IRRI's yield increase a secondary reason. The 2 were happened concurrently.
Please review the following webpages and linked pages :
Mesresha, please be very careful on how much you rely on " natural fertilizers" as opposed to chemical fertilizer and taxing their use to make certain there is enough to accommodate all agricultural needs, and the labor energy in using them is fully recovered by the increase in yield. I don't think this is possible, as we have already greatly exceeded the earth's natural carrying capacity. Please review the following webpages:
In response to Kelvin comment I would like to offer my opinion as to the role of M&E. That is primary reason for M&E is to represent the underwriting taxpayer or other funders to make certain funds invested are effectively used. To do this by default it become the representative of the beneficiaries. The secondary purpose of M&E is to guide future projects to more effectively identify and serve the needs of the beneficiaries. What it should not be used for is being a propaganda tool to deceive the funders in making what by normal standards would be complete failure appear unqualified success. Unfortunately, as best I can determine that is exactly what the USAID MEL (Monitoring, Evaluation, and Learning) program appears to do. This distortion of M&E does not provide extra guidance for future projects, but entrenches failures and reinforces them into future project while doing nothing for the unfortunate beneficiaries, Please review the following webpages:
Please allow me to add to my original comment a concern about the quality of the government services even if well funded. I have a major concern with the extension education effort to promote many innovations for smallholder particularly if they are more labor intensive. The problem is that agronomist, including myself, do an excellent job of determining the physical potential of an innovation, but say nothing about the operational resources needed to extent the small plot result across the rest of the field, farm or smallholder community. It is just assumed the labor is available and has the dietary energy to handle the extra effort. Unfortunately, this assumption is not correct and labor and dietary energy to fuel that labor are in short supply so the farmers have little prospects to adopt the extension recommendations. The question I have is who within the government or development project is responsible to determine the labor requirements, more importantly the availability of labor, and the rational compromises farmers have to make when the labor is in short supply.
Meanwhile and largely independent of government effort, the farmers look for access to mechanization for land preparation and crop establishment as well as threshing and other high labor intensive activities.
Please consider the following webpages:
This is a very interesting and challenging question. Unfortunately, I cannot be optimistic about solutions. it all depends on tax basis to generate the revenues needed to finance government services. Please visit the webpages on Financially Suppressed Economy and Financially Stalled Governments and see how accurately they describe your situation in Uganda. You might also look at the Uganda page on the Consumer Price Comparison. The problem is "no taxes, no services". Trying to provide services that are not adequately funded usually results in the services becoming corrupt and civil officers looking seeking necessary supplemental income by claiming to provide the services. Please note that in countries like Uganda about 90% of the smallholder farmers have never been involved with government programs or met with an agriculture officer either extension, research or other. Yet they still survive and evolve. The links:
Thank you for your second set of comments. What you describe are massive cumbersome administrative procedures, not only to facilitate the initial project establishment but also to sustain it as a viable competitive business model, that will continue to serve the smallholder community well past any external financial support and advisory facilitation. Administrative producers create overhead costs that a usually greatly underestimated and rarely included in evaluations. For the cooperative business model to be sustainable these costs will eventually, after the initial donor assistance ends, have to be paid for by the farmers and come from the bulking differential you mentioned as the financial benefit of the cooperative model. Can that be done? I seriously doubt it and would expect the overhead cost will exceed the financial benefits you mentioned. When this happens relying on your cooperative model will only force the farmers deeper into poverty. Thus, my question to you is to take 30 minutes to an hour and do the detailed financial analysis to clearly demonstrate that the cooperative business model you advocate and can provide sustainable financial benefits to the smallholder farmers once the donor facilitation support ends.
I think this will be very difficult to obtain as the overall suppressed economy puts tremendous downward pressure on consumer prices to only 1/3rd to 1/5th that of the USA and seriously limiting the profit margins of most competing private service providers. Thus, your bulking advantage will be too small to cover the overhead costs associated with the more cumbersome administrative procedures required for running a cooperative. Disturbing to me is the continued promotion the financial benefits without ever mentioning how much that is and what the costs are to obtain it. This still does not address the inconvenience of consignment sales in a society in which the basic financial management strategy is to retain goods in-kind selling only what is needed to meet immediate cash needs but needing immediate cash. This also prolongs the marketing season to 10 months after harvest.
Please review the following webpages and provide the detailed cost accounting to clearly demonstrate your cooperative model is not only socially desirable but also competitive, sustainable past donor assistance, and relying on it will not force farmers deeper into poverty. You might also add a detailed cost of doing business comparison with the competing private service providers. Why these basic business parameters are not included in the evaluation process is a mystery to me. I do think when done it would prove embarrassing. It these parameters were insisted upon 30 years ago the evaluation process would have successfully identified the non-competitive failure of the cooperative model and guided the poverty alleviation effort to more effective means of assisting the smallholder farmers. Without such a business analysis I am inclined to conclude that the development community is more interested in imposing on smallholder farmers what might be a socially ideal business model that is more likely to push them deeper into poverty, despite the massive rhetoric to the contrary, then sincerely assisting them out of poverty.
Thank you for the interesting comment. I am not certain I fully agree with you. I think what you may be referring to a ridged “way of life” is a result of the limited capacity of smallholders to adopt innovations intended for their benefit because most innovations are more labor intensive, while the farmers are severely labor limited and already substantially overextended. This is compounded by access to only half the calories needed for a full day of manual agronomic field work. There is a rather ugly word for expecting people to routine exert more energy than they have access to. The word is genocide and yes, we have had a genocide component in the smallholder development effort for about 40 years. This is an oversight that has fallen into an administrative void between the agronomists, who do an excellent job of determining the physical potential of an area, but say nothing about the operational resources, such as labor and contract mechanization, needed to extend their small plot result across the rest of the field, farm or smallholder community, and the social scientists who may determine the labor requirements, but stop short of determining the availability of the labor and most important what are the rational compromises smallholder farmers have to make in adjusting the more labor intensive innovations to their limited available labor and energy.
I am inclined to consider most societies to be more dynamic than you imply. I have a difficult time thinking smallholder farmers are content with their life style as it implies limited food security, normally a couple bouts of malaria each year, limited life expediency, infant mortality, etc. I think once we start conceptualizing on the limited operational resources available to smallholders and address this issue you will find the smallholders lifestyle will be very dynamic even if headed toward the Western Materialism which appears the more desire lifestyle. Without addressing the limited operational capacity, the farmers will remain entrenched in their impoverished lifestyle that you described.
Unfortunately, the evaluation process does not normally include a timing component which would quickly identify the operational constraints and encourage looking at how to address them.
Thank you Emem and Paul for your contributions. They are most appreciated.
1. Enem, please don't think it is just Nigeria where joint enterprises have been mostly a failure, but it is really across the board for anyone who take a closer look. You might be interested that the garri example mentioned in the original statement is from Benue, Nigeria. The main question is how do we convey to the powers that control project design that joint enterprise are waste of good donor money, and might up that to a scandalous waste. I would hope that this money could be more effective when devoted to alternative activities. You might like to visit my website and look at several pages devoted to how ineffective they are and how they get perpetuated in the donors imposed program design. The website link is: https://smallholderagriculture.agsci.colostate.edu/
2. Paul, you are approaching the problem from a more social scientist perspective that I, as a crop production agronomists, normal don't focus on. Have you ever looked at joint enterprise from a pure business perspective and computed the sustainable overhead costs including the time an effort for consensus building or conflict resolutions, and compared this to individual enterprises and their profit margins. I think if you did you will note that relying on joint enterprises would force the members more deeply into poverty.
Again you might look at the website mentioned above for some more detailed discussion.
3. While I very much appreciate the comments on group vs individual enterprises, I noted no one as attempted to address the other issue. That of operational limit such as labor and how that impacts on timing of operations, decline in potential yield, and ultimately food security. Is the lack of comment to date a confirmation that operational limits still falls into an administrative void between the agronomist and the social scientists? Can we ever assist smallholder farmers out of poverty until this concern is addressed.
What can we do to improve the quality of development projects? Comments
This is a very interesting and tough question, at least for smallholder agriculture projects. To get a full appreciation of the problem it is necessary to take a detailed look at how projects are formulated, the time & costs of formulating a development project, the degree the project developers are isolated from the beneficiaries, the representation the beneficiaries have during project development, the extend the donor is expecting the implementing team to appease their concept of what needs to be done and the cost of not appeasing the donor in terms of contract extensions and future projects.
Most agriculture development projects take a least 2 years from conception to signed contract & implementation with a cost at least one million US dollars. This is before there is an opportunity to objectively meet and interview beneficiaries and become aware of what their real needs and interest are. With that much time and money invested no one want to hear the project is mis-conceived and the beneficiaries are not interested and avoiding active participation. During this time the basic conceptualization is done by donors and often based on academic idealism, heavily promoted of social desirability rather than solid business accounting etc. This is done with the cooperation of host governments, who claim to represent the beneficiaries, but often don’t have the financial resources to be close contract with the beneficiaries and may have some personal vested interest in the project. During this period any contact with the beneficiaries are usually limited to very quick highly orchestrated appraisal visits and relying on pre-arranged and well briefed beneficiaries for interviewing. The beneficiaries are actually isolated from the donors claiming to arrange assistance for them by a 4-level development hierarchy. The end result is often very rigidly contracts with little flexibility for modification, leaving the implementors with little choice but to leverage the beneficiaries to comply with the donors conceptualization of what they need, or simply ignore project and proceed as best they can, which most opt for. The implementing team is also virtually hostage to appeasement reporting to show a successful project, regardless of the actual acceptance by the beneficiaries. Fail to do so will be interpreted as failure of implementing team resulting in limited project extension or future projects. After all the project design was perfect.
The best example of this is the nearly 40 years and billions invested in producer organization or cooperatives. However, an objective evaluation would show well over 90% failures collapsing immediately after external funding and facilitation end, with less than 15% of potential beneficiaries participating, marketing little more than the intuitional in-kind loan repayment, representing only 5% of the community’s market volume. Also, if the beneficiaries really relied on the project business model the overhead costs would exceed the bulking volume benefits which would than force the farmers deeper into poverty, despite the massive rhetoric of poverty alleviation.
Please review the following webpages from the smallholderagriculture.agsci.colostate.edu website:
Recurring Errors in Public Policy and Major Development Projects: Contributions and Solutions from Evaluation (Commentary)
Public Policies are only as good as the available finances to implement and enforce them. This then requires a viable tax base to support the civil services. Unfortunately, too often the host country does not have the necessary tax base. This is what I refer to as a financially suppressed economy. That is an economy serving a mostly impoverished society in which up to 80% of all income or production must be spent on a meager diet to feed the population. This leaves so little discretionary funds that can be taxed. No taxes, no services!! This leaves most host governments financially stalled, barely able to meet their salary and fringe benefit obligations for the civil officers with virtually no operational funds to implement programs or support policies. Thus, most public policies are paper polices, show a governments’ good intentions, and perhaps appease donors interested in providing financial assistance and funding development projects.
The result is:
One must be very careful in building a technical capacity that can not be operationally supported with adequately supported financially. This can be a major disservice to the intended beneficiaries and possible the general public. While not meeting certified seed quality standards may not be particularly deferential, claiming a food safety program based mainly on the honor/gratuity/baksheesh system can easily result in contaminated food getting into the supply chain, resulting in multiple sickness and even death. Better to have no system than a misleading system to prevent people becoming over confident in the food safety and avoiding taking personal caution in preparing food.
There may not be much you can do about this other than be aware that it can and mostly likely is occurring. Also, it might be better to minimize the involvement of host governments in projects as the marginally paid civil officers are usually looking for some informal (politely avoiding the horrible “C” word) opportunities from within the project and can be more a hinderance than an assistance to project implementation. Unfortunately, too often the informal income is a financial necessity for civil officers to obtain any reasonable professional live style.
Please review the following webpages from the https://smallholderagriculture.agsci.colostate.edu/ :
I think this is an excellent example of the consequences of over extending the financial resources of a host country. As I look at the overall economic conditions that are common to and effectively define developing countries, it is they represent financially suppressed economies. By this I mean economies that serve a generally impoverished population in which the bulk of the populations spend up to 80% of income or farm production on basic subsistence needs such as marginal food, clothing and shelter. While this can put tremendous downward pressure on consumer prices, there is also no discretionary funds to form a tax base that will fund civil services such as statistical analysis. Sorry, but no taxes; no services. Thus, governments are for all practically purposes financially stalled, barely meeting personnel financial obligations of salary plus benefits with virtually no operating funds for implementing programs, such as statistical surveys. This results in surveys being filled in by the enumerators in accordance with their biases or expected survey biases. However, the results are accepted as accurate by the government with policy decisions being based on the results. Governments then heavily rely on external donor funded NGO to do much of the development activities. Such NGO often have their own objectives or are contracted to impose donor motivated solutions that may or may not be consistent with the needs of the beneficiaries such as smallholder farmers. Please review the following webpages and links: https://smallholderagriculture.agsci.colostate.edu/financially-suppressed-economy-2/ and https://smallholderagriculture.agsci.colostate.edu/financially-stalled-governments/
Looking at the specific Benin survey case, I sense that being somewhat aware of the budget limits the organizers tried to compensate by making the survey more complex and more demanding on the enumerators. As an agronomist my discipline does not normally rely on survey information and are not fully familiar with survey technology. However, I am somewhat unusual for an agronomist and have often relied on survey information to get a fuller understanding of farming systems. Thus, I am familiar with some of the guidelines for conducting a quality survey. My normal guidelines are that a survey should take no more than an hour, an enumerator should conduct only 4 or at most 5 interviews per day. Thus, expecting them to do a 15-page survey would be very difficult in an hour and 10 per day virtually impossible. Thus, the enumerators will have to resort to group interviews that I tend to dismiss as having individual biases of outspoken minorities in them, or simply fill out the surveys on behalf of the interviewees. If you are expecting results from remote areas and not able to provide transportation, is there an alternative to having the enumerators fill out the surveys and still meet the limited deadlines for completing the survey? I don’t think so.
The result is the data is full of bias and unreliable, but consistent with the financial state of the country. It also means that the government is substantially out of touch with most of the populations. I recall hearing a few years ago that about 90% of smallholder farmers have never had any contact with agriculture officials and they are perhaps better off because of that.
A more specific case that I am personally knowledgeable of would be the Ruaha river basin in Southern Tanzania. Here there were many complaints that the river was drying up earlier each year and endangering the wild life in the Ruaha National Park, which was blamed on climate change. According to the government there were 4 rice schemes above the National Park totally some 11,000 ha. However, when they got an aerial or satellite view the total rice acreage was some 40,000 ha. The difference was many small illegal informal schemes build by the farmers even including some permanent concrete abutments to divert the water. Now what civil officers is going to wonder through the area on a rough road that will take at least 6 hours to fully transvers the 200 km and tell the poor smallholder farmers that must stop growing rice so foreign visitors can enjoy the wild life in the National Park. More likely they will visit the area, inform the farmers the rice cultivation is illegal, but with a modest gratuity allow them to continue until they return in 3 or 4 years. What government official will acknowledge this? Easier to simply continue to blame the river drying on climate change.
So, what can be done to get more reliable survey information? I don’t really know. I would suggest extrapolating from donor assisted NGO administered projects where they do have enough budget to manage a reliable survey. This would make for a patchwork of quality information across the country, but I would expect it to be fairly representative. Until the economy can expand to provide a reasonable tax base to finance civil services I fear this is the best means forward.
I hope this is helpful.