Struggling to prove the obvious – How can evaluations show that community-driven development works?

CDD
©IFAD

Struggling to prove the obvious – How can evaluations show that community-driven development works?

Community-driven development (CDD) emphasizes community control over planning decisions and investment resources, operating on principles of transparency, participation, accountability and enhanced local capacity.

Although CDD is recognized as an efficient way of delivering public goods, in particular in marginalized or fragile situations, evaluations often struggle to find strong evidence to demonstrate the empowering impact of this approach. This challenge may be tied, in part, to the wide range of contexts involved and to expectations of what these programmes should deliver.

The debate

Expectations of what community-driven development can deliver, in terms of social and political impact, have been high and sometimes overly optimistic.

The first evaluation of the World Bank’s support to CDD (2005) found that outcome ratings for such projects were better than for non-CDD projects. But the evaluation also showed that CDD projects achieved more on quantitative goals such as construction of infrastructure than on qualitative goals such as capacity enhancement. [i]

The 2018 evaluation synthesis on CDD prepared by the International Initiative for Impact Evaluation (3IE) made major headlines within the development community. It stated that these programmes had had little or no impact on social cohesion and governance, triggering an ongoing debate over how to evaluate CDD. [ii]  More recently, in 2019, the Independent Office of Evaluation at IFAD conducted the first synthesis of its work in this arena over forty years. As a first step, the synthesis reviewed more than a thousand projects approved since 1978 and identified 243 projects with CDD elements, which ranged from having elements such as a community development fund or a specific focus on community empowerment as a project objective. The synthesis shed some interesting light on how to gauge the contributions of CDD.

Apples and oranges

A common issue in evaluation is how to avoid comparing apples with oranges. CDD projects may differ widely, as they have to be tailored to the capacities in place and to local conditions. For example, in IFAD’s work, participatory community development--where communities participate in specific stages of a project, usually in the planning and implementation--is probably the most common approach. More specifically efforts to strengthen participatory local governance are usually applied in natural resources management and agricultural development projects where communities are empowered to work as partners with local government

The evaluation synthesis of the Fund’s work focused on those approaches which used a community development fund (CDF) to enable the community to take control of funding decisions and the investments made. CDFs have funded public infrastructure, but also economic investments undertaken by groups of rural people.

The definition of "community" poses another challenge in the evaluation of CDD projects. Often, a community is seen as linked to a territory, in particular in the African context. However, this view would not encompass a whole range of non-territorial communities targeted in IFAD operations, such as: self-help groups in India and groups of agro-pastoralists and non-sedentary pastoralists in Ethiopia. The use of the term "community" as such is sometimes problematic because it can mask the level of diversity and inequality.[iii]

Barking up the wrong tree?

Evaluations of IFAD-supported CDD projects universally report that they strengthened social capital in rural communities. Yet there is little “hard” evidence to support these assertions. Measuring and comparing concepts like social capital and social cohesion can be especially challenging. Specific, measurable interpretations vary according to context, even if there is broad understanding, for example, of social capital as trust, relations, and networks.

Qualitative methods seem to be better suited to the measurement of social capital. In Bolivia, for example, an evaluation of the Plan VIDA-PEEP showed that findings on social capital can vary significantly between qualitative and quantitative assessment. The qualitative study looked at the ‘softer aspects’ of social capital, such as mutual trust, social rules and how they are applied, and the creation and strengthening of solidarity networks between participating families and communities. It found that CDD strengthened mutual trust and solidarity networks between participating families and communities. Meanwhile, the quantitative study looked at ‘factual’ indicators, such as group membership and the number of times the respondent has stayed with non-family members outside of the community in the past year. Based on the results of these indicators, it concluded that there was little to no impact on the social capital of rural communities.

But are we right to assume that CDD should have an impact on social capital? Are we perhaps confusing cause and effect? The Plan VIDA-PEEP impact assessment found that in Bolivia, where social relations are strong and CDD projects drive policy initiatives, social capital could be both a contributor to and a result of, good project performance.

… and yet it seems to work

The evaluation synthesis found that CDD delivered better results, in particular, in marginalized and/or fragile situations. They were  more successful in such areas as promoting gender equality, functional and vocational training of women, formation of groups like savings and credit groups, income-generating activities, and indirectly through investments in projects like drinking-water systems.

There was also evidence that community ownership helped to ensure the sustainability of natural resources and physical assets, but that long-term sustainability also depended on government support. Insufficient government budget allocations to pay for equipment, utility services and staff housing sometimes reduced the sustainability, for example, of schools and health centres.

So CDD remains relevant

This latest synthesis supports the overall view that CDD as a form of people-centred and locally owned development has the potential to address mainstreaming issues such as the formation and strengthening of farmer groups, gender equality, food security and nutrition, natural resources management. It also shows that community-driven development can make a major contribution to effective institutions and participatory decision-making (as envisaged by SDG 16).

The synthesis shows that in our evaluations we need to focus on what we can realistically expect CDD to deliver and consider the specific context in order to know what works under what conditions.

***

[i] OED World Bank. 2005. The effectiveness of World Bank support for community-based and –driven development

[ii] Guggenheim, Scott. 2018. Comments on 3IE Study on Community Driven Development. In FP2P, Blog by Oxfam GB maintained by Duncan Green.

[iii] Cornwall, Andrea. 2000. Beneficiary consumer citizen: Perspectives on Participation for Poverty Reduction. Sida studies no 2.