RE: Conducting MEL with digital agriculture service providers – lessons from leveraging MEL to develop farmer-centric digital services | Eval Forward

Dear Emily,

Many thanks for the interesting and thought-provoking blog. 

In reading it, I also skimmed the documents to which you provided links. The report on the Digitalisation of African Agriculture by the Technical Centre for Agricultural and Rural Cooperation ACP-EU report on was particularly revealing. First my thoughts on this, then those on the blog itself. 

1.    The CTA Report on the Digitalisation of African Agriculture

The hope that that D4Ag could be a game changer in boosting productivity, profitability, and resilience to climate change. This assertion is riddled with many assumptions. 

Two points: 

First, potential jobs for 75% of unemployed African youth may be, what about how this transforms African agriculture and the lives and livelihoods of farmers? I question the significance of how the absence of digital solutions offers a significant reason as to why smallholders are disconnected from input and product markets. The absence of a solution rarely explains the underlying problem. I also worry about what Varoufakis calls Techno feudalism  - the tyranny of big tech - and the effects of D4Ag going to scale, who the main beneficiaries are, and who pays the rent. A bit of a counter-culture, and pours water on the tech parade, but Yanis Varoufakis makes some intriguing points.

Second, if the  EU is serious about supporting the transformation of African Agriculture, it would: 

  • Do a lot more than co-finance the African Continental Free Trade Area’s (AfCFTA) Secretariat in Dakar - one of Agenda 2063’s flagships - and look beyond mimicking China’s belt and road support through the $150 investment allocated for the Global Gateway. A good start would be to cancel its main trading mechanism with the continent  - the EPAs and pressure African governments to stop taxing its farmers: a policy that contradicts what Africa needs as defined by the AfCFTA; and a policy that largely explains the constraints to achieving yield gains. 
  • Reform its CAP that facilitates the dumping of food on domestic African markets; inhibits Africa’s aspirations to grow its agricultural economies through extortionate non-tariff barriers to African exports; and, equally important, taxes European consumers and wreaks havoc with European ecosystems through biodiversity losses and greenhouse gas emissions.   

The adverse effects of the above are arguably the main reasons explaining Africa’s food import bill, their continuation its projected increase.

In addition to the above comment on youth, technofeudalism, and the policy and regulatory constraints, I found the YouTube clip  - Key Figures from the Report - to be cleverly presented but thin. For example, the projected 22% decline in yields on the continent, I submit, is not the main and most important consequence of climate change as much as the growing presence of D4Ag can resolve so stimulate increases in yield, let alone those in farmer incomes. Biodiversity loss is the main significant consequence. Why? Fallow areas have virtually disappeared in Africa. On average the rate of fallow is 1.2 percent with fallow having all but disappeared except for Tanzania (7,8%) and less so Uganda (5%). The result of African farmers more than doubled annual increases in cropped land from 1.7Mha in 2004- 2007 to just under 4Mha in 2016-2019. For the most part, production gains have been through an increase in area under cultivation; that is, as opposed to gains in productivity. This is in stark contrast to the rest of the world where production increases have been realised exclusively through increases in physical yields. This expansion of agricultural land has taken over natural ecosystems and has been the biggest driver of the destruction of Africa’s biodiversity. Defining success has to do more than claiming incremental increases in farmer yields and associated gains in smallholder incomes typically reported by many NGOs and donors that are typically used to justify the project investment. I don’t believe the issues facing Africa’s farmers can be resolved by projects anymore.   

2.    The blog itself. 
A great problem statement: “But MEL is often imposed by donors to track the impact of their funding, and service providers often associate it with tedious reporting and struggle to see its value.”.  

But isn’t it too easy to blame donors, and assume they know what information they need and when and what decision uncertainties they face? Consulting companies, the agents of donors, rarely negotiate information requirements. They see the donor as the ultimate client and lack a balanced accountability arrangement with those in need/those they are paid to support. Projects often resemble a traded commodity trapped in a client/agent relationship. 

And the default measure of crop yields? The obsession, almost an indicator fetish, with using crop yields as a valid measure of success for agricultural projects, was called out back in the early 1990s’. See a blog I wrote on the pitfalls of having this as a valid and useful pursuit. Its use is a cockroach M&E policy measure: you think it was flushed away; yet keeps coming back!!! (Sorry)

It would be really interesting to learn more about the MEL approach you designed for the GSMA Innovation Fund for the Digitisation of Agricultural Value Chains – who did you read/talk to on developing the approach? 

I like how you saw and pursued the need to re-brand M&E and adapt its tools to effectively collaborate with private sector partners. As part of a study on impact investing back in 2020, I stumbled across what I thought was a great example developed by Leapfrog, an impact investor in the financial services sector. Its approach to capturing customer experience and making services more service user/client-centric reminded me of the pioneering work of Robert Chambers and Lawrence Salmen in the development aid sector back in the 1980s. Feedback loops and treating farmers as the subject of conversations on issues that matter to them; as opposed to objects of a survey on matters that concern the donor. Leapfrog’s approach is documented here if you are interested.

I completely agree that when designing MEL frameworks, it is useful to reflect on the value proposition of MEL; one that balances the information requirements among a “hierarchy of users” and doesn’t divorce itself from so is seen in isolation with other people and processes – financial control, learning, decision-making, and delivery. More often than one would hope or expect, the process starts on the wrong foot by developing a theory of change and/or a results framework and “slides downhill’ from there. But, I was left wondering why GMSA did this with the private sector. Why isn’t this done for all M&E frameworks?  

Your approach involved running three quantitative surveys per service to gain smallholder feedback on services, which also helped you report on outputs and early outcomes KPIs such as farmer satisfaction with services and behaviour change in farming practices.

Why three  - to capture seasonality and why quantitative? Won’t this many encourage fatigue to set in just as it appeared to be the case with farmers being swamped with SMS? And, concerning your last screenshot - High SMS reading rate and understanding of advice, but behaviour change challenges remain and the frequency of SMS is too high – how do the numbers inform answers from the enumerators in response to the two questions on the left-hand side? Did your survey design test the assumptions made explicit in the Impact Roadmaps or Project Blueprints as much as measure movements in the relative values of pre-defined indicators (of adoption, for example)? 

Apologies for the ramble, yet I hope some of the above observations are helpful, and many thanks again.

Best wishes,