Digital agriculture services provide a key opportunity to improve smallholder farmers’ access to information, markets and financial services. However, these services are hard to scale, often meeting low levels of adoption and usage. In donor-funded programmes, Monitoring Evaluation and Learning (MEL) can play a key role in exploring service adoption challenges among smallholder farmers and identifying lessons to make services work for rural, low-income populations. But MEL is often imposed by donors to track the impact of their funding, and service providers often associate it with tedious reporting and struggle to see its value. So how can we rebrand MEL and adapt its tools to effectively collaborate with private sector partners?
This question greatly influenced the MEL approach I designed for the GSMA Innovation Fund for the Digitisation of Agricultural Value Chains. This FCDO-funded innovation fund provided technical assistance and results-based grants to 6 companies developing and scaling digital agriculture services. You can read more about the project, the MEL methodology and the findings in the report Improving Farmer Livelihoods Through Digitised Agricultural Value Chains. A similar approach is now being implemented in the GIZ-funded GSMA AgriTech Accelerator, which provides Technical Assistance to 10 agritech companies scaling digital services that improve farmer livelihoods and climate resilience.
This blog provides key takeaways from designing an engaging MEL approach tailored to digital agriculture service providers.
Rebrand MEL to resonate with the private sector
Many MEL tools can serve a dual purpose, meeting reporting and learning needs while being useful for companies. When designing MEL frameworks, it is useful to reflect on the value proposition of MEL that will be communicated with private sector partners. Differentiating between internal objectives such as results verification or donor reporting, versus learning objectives for private sector partners helps to shape this MEL “brand”. Aligning MEL learning questions with the vision and objectives of service providers is key to generate a clear value proposition. For example, instead of emphasizing the role of MEL in monitoring project performance and reporting to donors, we rather framed it as technical assistance that would help them better understand their users, and that would allow them to improve and scale their services. Tools can also be renamed to resonate with the private sector, for example our Theory of Changes have become Impact Roadmaps or Project Blueprints. Finally, getting buy-in from and working with senior executives, as well as with people close to implementation such as product managers, is critical to ensure buy-in and that MEL findings will be acted on to improve business models and make services more farmer-centric.
The GSMA MEL brand communicated to private sector partners
Develop MEL tools that are also useful for private sector partners
The MEL skillset lends itself well to projects focused on testing or scaling new services, and to commercial entities seeking to raise further investments. Designing impact strategies, setting KPIs, collecting and analysing service data, and reflective management are familiar to both MEL specialists and commercial product managers. Conducting a MEL needs assessment workshops at the start of a project is a good way of identifying private sector partners’ priorities, potential synergies and how MEL tools can help them meet strategic objectives. We used the Theory of Change to map potential business plan risks for services to scale up, and to identify impact areas that align with environmental, social, and governance (ESG) investing, as companies are often asked to demonstrate the value and the impact of their service to raise funds or for sustainability reporting. Specifically, we explored overlaps between donor-facing impact KPIs we needed to collect and agriculture ESG data, such as the agriculture metrics identified by the IRIS+ benchmark tool developed by the Global Impact Investing Network. In summary, KPIs and data collection should include data points that are useful for companies in their reporting, communication activities and their efforts to raise investment.
GSMA AgriTech MEL tools, focusing on their value proposition for private sector partners
Establish feedback loops to leverage findings for product iteration
Higher outreach and activity rates are key commercial objectives for service providers and human-centric design can help address some of the challenges of smallholder service usage. It requires collecting regular feedback from users to inform changes in services or their delivery. MEL can leverage qualitative and quantitative research expertise to provide independent user journey analysis and identify smallholder pain points and opportunities for service improvements. We dedicated up to three quantitative surveys per service to gain smallholder feedback on services, which also helped us report on outputs and early outcomes KPIs such as farmer satisfaction with services and behaviour change in farming practices. Setting up regular touchpoints with private sector partners to debrief and brainstorm results from data collected by MEL helps articulate lessons and identify concrete actions to be taken as a result of user feedback. For example, one of our surveys found that farmers using digital advisory services provided by our Indonesia and Pakistan agritech partners were not aware of new farming advice added to the service or new features such as weather forecasts. We shared the findings with both agritechs and user experience specialists and discussed potential ways to address this challenge. As a result, push notifications from the mobile apps and SMS advisory were introduced to pull users to specific content.
Screenshot from an anonymised brainstorm session to identify learnings and product improvements based on MEL data
---
Rebranding MEL, developing tools that were useful to partners and establishing feedback loops to make sure findings were actionable greatly helped MEL’s image with private sector partners. Feedback gathered during workshops found high perceived usefulness of MEL, and data collection was seen as leading to concrete learnings. This buy-in ultimately helped with more efficient collaboration in implementing MEL activities and allowed us to improve our processes.
What is your experience of tailoring MEL to work with digital agriculture service providers?
Feel free to provide your comments and feedback below.
Daniel Ticehurst
Monitoring > Evaluation Specialist freelanceDear Emily,
Many thanks for the interesting and thought-provoking blog.
In reading it, I also skimmed the documents to which you provided links. The report on the Digitalisation of African Agriculture by the Technical Centre for Agricultural and Rural Cooperation ACP-EU report on was particularly revealing. First my thoughts on this, then those on the blog itself.
1. The CTA Report on the Digitalisation of African Agriculture
The hope that that D4Ag could be a game changer in boosting productivity, profitability, and resilience to climate change. This assertion is riddled with many assumptions.
Two points:
First, potential jobs for 75% of unemployed African youth may be, what about how this transforms African agriculture and the lives and livelihoods of farmers? I question the significance of how the absence of digital solutions offers a significant reason as to why smallholders are disconnected from input and product markets. The absence of a solution rarely explains the underlying problem. I also worry about what Varoufakis calls Techno feudalism - the tyranny of big tech - and the effects of D4Ag going to scale, who the main beneficiaries are, and who pays the rent. A bit of a counter-culture, and pours water on the tech parade, but Yanis Varoufakis makes some intriguing points.
https://www.theguardian.com/world/2023/sep/24/yanis-varoufakis-technofeudalism-capitalism-ukraine-interview
Second, if the EU is serious about supporting the transformation of African Agriculture, it would:
The adverse effects of the above are arguably the main reasons explaining Africa’s food import bill, their continuation its projected increase.
In addition to the above comment on youth, technofeudalism, and the policy and regulatory constraints, I found the YouTube clip - Key Figures from the Report - to be cleverly presented but thin. For example, the projected 22% decline in yields on the continent, I submit, is not the main and most important consequence of climate change as much as the growing presence of D4Ag can resolve so stimulate increases in yield, let alone those in farmer incomes. Biodiversity loss is the main significant consequence. Why? Fallow areas have virtually disappeared in Africa. On average the rate of fallow is 1.2 percent with fallow having all but disappeared except for Tanzania (7,8%) and less so Uganda (5%). The result of African farmers more than doubled annual increases in cropped land from 1.7Mha in 2004- 2007 to just under 4Mha in 2016-2019. For the most part, production gains have been through an increase in area under cultivation; that is, as opposed to gains in productivity. This is in stark contrast to the rest of the world where production increases have been realised exclusively through increases in physical yields. This expansion of agricultural land has taken over natural ecosystems and has been the biggest driver of the destruction of Africa’s biodiversity. Defining success has to do more than claiming incremental increases in farmer yields and associated gains in smallholder incomes typically reported by many NGOs and donors that are typically used to justify the project investment. I don’t believe the issues facing Africa’s farmers can be resolved by projects anymore.
2. The blog itself.
A great problem statement: “But MEL is often imposed by donors to track the impact of their funding, and service providers often associate it with tedious reporting and struggle to see its value.”.
But isn’t it too easy to blame donors, and assume they know what information they need and when and what decision uncertainties they face? Consulting companies, the agents of donors, rarely negotiate information requirements. They see the donor as the ultimate client and lack a balanced accountability arrangement with those in need/those they are paid to support. Projects often resemble a traded commodity trapped in a client/agent relationship.
And the default measure of crop yields? The obsession, almost an indicator fetish, with using crop yields as a valid measure of success for agricultural projects, was called out back in the early 1990s’. See a blog I wrote on the pitfalls of having this as a valid and useful pursuit. Its use is a cockroach M&E policy measure: you think it was flushed away; yet keeps coming back!!! (Sorry)
It would be really interesting to learn more about the MEL approach you designed for the GSMA Innovation Fund for the Digitisation of Agricultural Value Chains – who did you read/talk to on developing the approach?
I like how you saw and pursued the need to re-brand M&E and adapt its tools to effectively collaborate with private sector partners. As part of a study on impact investing back in 2020, I stumbled across what I thought was a great example developed by Leapfrog, an impact investor in the financial services sector. Its approach to capturing customer experience and making services more service user/client-centric reminded me of the pioneering work of Robert Chambers and Lawrence Salmen in the development aid sector back in the 1980s. Feedback loops and treating farmers as the subject of conversations on issues that matter to them; as opposed to objects of a survey on matters that concern the donor. Leapfrog’s approach is documented here if you are interested. https://leapfroginvest.com/press-release/creating-impact-with-leapfrogs-cx-launchpad-program/
I completely agree that when designing MEL frameworks, it is useful to reflect on the value proposition of MEL; one that balances the information requirements among a “hierarchy of users” and doesn’t divorce itself from so is seen in isolation with other people and processes – financial control, learning, decision-making, and delivery. More often than one would hope or expect, the process starts on the wrong foot by developing a theory of change and/or a results framework and “slides downhill’ from there. But, I was left wondering why GMSA did this with the private sector. Why isn’t this done for all M&E frameworks?
Your approach involved running three quantitative surveys per service to gain smallholder feedback on services, which also helped you report on outputs and early outcomes KPIs such as farmer satisfaction with services and behaviour change in farming practices.
Why three - to capture seasonality and why quantitative? Won’t this many encourage fatigue to set in just as it appeared to be the case with farmers being swamped with SMS? And, concerning your last screenshot - High SMS reading rate and understanding of advice, but behaviour change challenges remain and the frequency of SMS is too high – how do the numbers inform answers from the enumerators in response to the two questions on the left-hand side? Did your survey design test the assumptions made explicit in the Impact Roadmaps or Project Blueprints as much as measure movements in the relative values of pre-defined indicators (of adoption, for example)?
Apologies for the ramble, yet I hope some of the above observations are helpful, and many thanks again.
Best wishes,
Daniel