How are we progressing in SDG evaluation?

Dear colleagues,

Thank you so much for your active participation and engagement in this discussion. A few words on the latest contributions:

Nea-Mari, thank you for the information and links of the Finnish SDG M&E at subnational level.  Inspiring examples for other countries and cities! Thanks Esosa for highlighting the critical role of evaluation in evidence-based development programs. And Mark, yes, acknowledging limitations of our studies/evaluations is always a good practice, thanks for highlighting this point.

This discussion comes to a close for now, but there will be more opportunities ahead to further exchange ideas and knowledge on supporting progress towards the SDGs through evaluation.

Wishing you all the best, and stay tuned for future updates. 

Emilia

Dear Emilia,

Regarding your last point, I'm glad you raised the issue of why our report found the patterns we did in terms of which SDGs are being evaluated (with quantitative impact evaluations) and which aren't. For context, we generally tried to stay away from "why?" questions because (a) our data don't really give us insight about that, and (b) we aren't experts in all the sectors covered. 

But obviously, over the course of writing the report we wondered about why we were seeing the patterns we did. Like you, I'd be very curious to hear thoughts from environmental sector experts. I'd also be interested in hearing your own hypotheses!

Thanks,

Mark

Dear members,

My contribution.

Evaluation is a critical factor and core subject when assessing the progress towards UN SDGs. The emphasis placed on the role of evaluation in SDGs' achievement is justified because its contribution to making a difference and the institutionalized data and statistics developmental strides enhancing SDG-related progress across nations. Evaluation is vital in research and in programmes/projects embarked by individuals, institutions and nations in line with SDGs. The outcomes of programmes/projects are better known thanks to evaluation. Ascertaining the levels of progress and how it can upscale the necessary gap bridging between or among nations can be perfectly done by those involved in evaluation.

For example, the inequality in world's economy between global north and south can be measured and better understood using reliable data sources through evaluations. Only when a nation's economy has been evaluated with reliable data/evidence, assistance can be provided to bridge the inequalities. Without data validation by the evaluators, no help can be sought to make progress towards the UN SDGs. To understand economic progress  the evaluation of relevant data is required to validate the findings. As an example, a research study on data from USA and China provided me a sound comparison of their development levels and progress. Evaluation is key to UN SDGs accomplishments and success. Evaluation is knowledge, findings . Furthermore, evaluationis what validates the findings and empirical facts of research and relevant programmes.

Empirical facts can be validated by economy indexes and indicators. In the case of USA and China the researcher got its information from World Bank and IMF. These are reliable sources which can validate the strenght of their economies  andtheir global hegemony. This research is an important verdict to ascertain the global north and south economy inequality before calling for programmes/projects’ implementation to bridge the gap. When the issue(s) is known, fixing it won't be difficult, that is the stands of evaluation.

However, sustaining SDGs progress, countries most know the fundamental issues for developmental strides of their economies. This is why evaluation is paramount to SDGs developmental goals in the world. This will help to validate the progress made so far and how to improve on it or them. In addition the 17 SDGs need proper institutionalized monitoring and evaluation to make a difference in world's economy trajectory growth and developmental sustainability. Evaluation contribution to SDGs target is imperative whether as research or programmes/projects. Nevertheless, I think there is need to have an institutionalized governing data body that evaluates the progress of nation's development from local areas, states' level, national, sub-national and international levels. This will really help and assist UN SDGs targets. The local level areas will give the true pictures of communities development at national level of a nation in SDG programmes/projects and research.

Moreso, identifying what works, begins with local level evaluators to national and international levels. It trickles from down up. If truly there is an institutionalized governing body for data evaluation and validation, what works will be easy to identify and lessons learned will be made known for effective SDGs implementation and progress. All these factors have to do with cultural differences which means a lot in developmental strides of a nation. This is why local and national evaluation is paramount to SDGs targets. In other words, what works for country A might no work for country B, that is to say, environmental factors and differences and influence are subject and better understood through local and national evaluators for progress of SDGs programmes/projects or goals. Evaluation is a key component to reach out to maximize a possible change of world's economy development, peace, justice, zero hunger, quality healthcare, good education and no poverty etc.

Finally, to have a true comprehensive nature and effective development across board in nations, let there be mixture of UN, sub-national, national, local evaluators integration, comprises of different bodies from federal, state, local levels, private sector, civil society organizations, NGOs and UN agencies in monitoring and evaluating SDGs targets implementation and sustainability. This will bring policies of inclusiveness for all stakeholders for SDGs achievements globally.

Thank you.

Esosa.

Thanks for the questions regarding the Finnish experience. To give some examples on the subnational level, the Ministry for Environment’s programme Sustainable City (2019-2023) piloted a co-created set of municipal indicators for sustainable development. The Association of Finnish Local and Regional Authorities has a flagship project for the six largest cities in Finland on the strategic steering of sustainable development goals. The network has co-created SDG tools. 

Some links of interest might be: 

Website: The Association of Finnish Local and Regional Authorities; SDGs: https://www.localfinland.fi/network-strategic-management-sdgs-cities

Website: Sustainable City

https://kestavakaupunki.fi/en/-/indicators-for-sustainable-urban-development-kekanua-

We are working together with partners to produce the second edition of the SDG Evaluation Guidebook https://www.iied.org/17739iied. Stay tuned to read more later this year! 

Best regards, 

Nea-Mari 

Dear colleagues,

Thanks for contributing to a lively discussion. 

I comment on a few points, not with the intention of exhausting the conversation (neither to fully summarize it!), but hoping to provoke some additional reflection. 

1. We should go beyond the focus on measuring contribution or progress to SDGs: there is a range of dedicated studies/evaluations and indicators (including proxy indicators) which also contribute to understanding development progress. SDGs do not sit in isolation, and there can be several pathways and ways leading to the same direction. Dorothy and John Akwetey particularly articulated this topic, but it is present various contributions. They also emphasize the significance of evaluations at national, institutional, and sub-national levels, beyond large-scale SDG evaluations. 

The approach used by the study on evaluation evidence shared by Mark Engelbert, which used impact evaluations as a key input, seems to speak to this last point.  

On the same line is the work that the The Global SDG Synthesis Coalition is conducting. The synthesis can be used either as an alternative to an SDG focused evaluation or as part of a larger study. The syntheses follow a systematic and transparent approach to identifying, collating and appraising the quality of individual evaluations, and then synthesizing findings and lessons from bodies of evaluative evidence. The approach includes evidence gap maps and other tools, including a rigorous process (and corresponding framework) to include or exclude studies. 

2. The challenges to evaluate SDGs encountered by most countries and development actors, and shared with different lenses by Ram Khanal Lovemore Mupeta and Hadera Gebru include: limited resources, insufficient data, lack of appropriate evaluation techiques and complex interlinked targets. In light of these challenges, we should (i) consider/search for other approaches (synthesis is one of them), rather than launching ourselves into potentially daunting evaluations, (ii) start small and (iii) scope wisely for studies that can be useful. Engaging country-based professionals (evaluators and implementers from different sectors) in the process, could support increasing awareness and build evaluative capacity. 
Unfortunately, major political unrest and challenges can result in a complete setback for any attempt to evaluate progress towards the Sustainable Development Goals (SDGs), as exemplified by the situation in Ethiopia where the post-Covid crises and civil war have undermined all developmental progress.

3. The subnational level (local, in particular), is another recognized challenge shared by many. Nea-Mari, I am curious to know a couple of examples of what Finland has been doing at local level – which types of digital solutions  have you adopted for the M&E of SDG progress? I am also positively surprised by the influence of the evaluations into parliamentary elections and in the planning of the new government programme. What would you say, Nea-Mari, are the key-elements that make these evaluations powerful in Finland? 

4. Examples of reports: Pelagia Monou, Fabandian Fofana, I wonder if the reports of the evaluations you have been involved are public and you could share the link with us? Pelagia, were you able to go beyond the number of projects and budget to tap into contributions or result? Fabandian, did you measure the contributions to SDGs at local level? Who was involved and how? 

5. And last but not least (but on a kind of a side note), a comment about the finding of the 3ie report shared by Mark, that evaluation work on the “Planet” SDGs (SDGs 6 and 12 to 15) has been neglected. The report informs that very little (impact) evaluation research was found covering SDGs 12 (Responsible Consumption and Production), 14 (Life Below Water), and 15 (Life on Land). While I have my own hypothesis as an explanation for this finding, I wonder if Stefano D’Errico, Ram Khanal and other colleagues with expertise in the environmental sector would like to chip in on the reasons? 😊

Still a long way to go: Chris, Olivier and Lal remind us that the post A2030 Framework is rapidly approaching!

Thanks all for contributing!! 

Warm regards

Emilia 

Greetings!

Here is my contribution to the on-going discussion topic: how are we progressing in SDG evaluation?

I appreciate and thank all who have contributed/share their experience/ to this important discussion topic. I also extend my thanks to the UN and relevant parties who have put efforts and minds together way back in 2015 to establish the SDGs (17 ambitious and inter-linked goals) for the betterment of the world people and the planet. 

From my experience development evaluation works are often encountered with constraints such as lack/inadequate data, limited budget, time constraint and lack of appropriate evaluation techniques and etc. These constraints can also be compounded/ aggravated with natural and man-made crises (e.g. Covid- 19; War). Likewise, SDG evaluation works if/when conducted can face similar constraints.

Let me reflect the situation in Ethiopia regarding the progress in SDG evaluation. In 2015 Ethiopia proactively mainstreamed, aligned the SDGs with the Second National Growth and Transformation Plan (GTP II), which was very encouraging. In fact Ethiopia then  had good track record in some development initiatives, similar to some of the   GDGs.

However, the encouraging momentum of development, monitoring and evaluation initiatives/works related to the achievement of the SDGs have either almost shattered/stagnated/reversed due to the natural crises/e.g Covid-19) and devastating on- going  civil war in most parts of Ethiopia. To my understanding, it has and is hard to conduct/even think about evaluating achievements of the of SDGs since the last six years. Thus, I guess the progress of evaluating SDGs in Ethiopia given the on-going devastating civil war compounded with the prior Covid-19 effect is poor. Due to the on-going civil war, let alone to achieve the SDGs, the situation is in a reversal mode. To mention few: millions of Ethiopians are internally displaced and are under poverty/ hunger; millions of students cannot attend school; millions have poor access to clean water and sanitation; there is high human right violation/particularly of women in many parts of the country.  Thus, even with the absence of evaluation work of the achievement of SDG in Ethiopia by evaluation experts, ordinary people can understand the achievement of SDG goals is generally not good, which requires attention of all relevant parties, at national, regional and international levels.

With regards,

Hadera Gebru
Senior Development and Evaluation Consultant
Addis Ababa, Ethiopia

Dear Emilia

Thank you for bringing up this discussion. That's an interesting thought! I have been reading the contributions with interest and I believe that moving beyond assessing progress towards the SDGs is crucial for achieving them by 2030. While tracking progress is important, it paints only a partial picture. Here are some potential areas in which I assist organizations and institutions in exploring beyond traditional assessments:

1. Deepen Understanding of Challenges and Gaps:

• Go beyond averages: Disaggregate data to understand how different demographics, regions, and groups are experiencing progress or setbacks.
• Analyze root causes: Don't just measure outcomes, delve into the underlying factors hindering progress, such as inequalities, governance issues, or lack of resources.
• Explore unintended consequences: Assess the potential negative impacts of actions taken towards reaching some SDGs, and how they might affect others.

2. Facilitate Action and Implementation:

• Identify actionable solutions: Use data and analysis to recommend practical interventions and policies that can accelerate progress.
• Develop implementation pathways: Bridge the gap between policy and action by outlining concrete steps to achieve desired outcomes.
• Engage stakeholders: Partner with communities, businesses, and civil society to co-create and implement solutions, fostering ownership and accountability.

3. Promote Innovation and Collaboration:

• Explore disruptive solutions: Go beyond traditional approaches and embrace innovative technologies, social models, and business practices that can significantly accelerate progress.
• Foster multi-stakeholder partnerships: Break down silos and encourage collaboration between different sectors, leveraging diverse expertise and resources.
• Support knowledge sharing and capacity building: Share best practices and lessons learned across regions and countries to accelerate global progress.

4. Address Systemic Issues:

• Tackle underlying inequalities: Recognize that unequal access to resources, opportunities, and power hinder progress, and prioritize actions that address these imbalances.
• Strengthen institutions and governance: Invest in building strong, inclusive, and accountable institutions that can effectively implement the SDGs.
• Promote sustainable development: Recognize that environmental degradation and climate change threaten progress on all SDGs, and integrate sustainability into all efforts.

Let’s remember that achieving the SDGs requires a holistic approach that goes beyond just numbers. By understanding the challenges, fostering action, embracing innovation, and addressing systemic issues, we can move beyond mere assessment and create a more just, equitable, and sustainable future for all.

Do you have any specific areas or challenges within the SDGs that you'd like to explore further? I'm happy to assist in any way I can.

Best regards,
John

---
John Akwetey (he/him/his)
Doctoral Graduate Associate, The Evaluation Center
Interdisciplinary Ph.D. in Evaluation (IDPE) 
Western Michigan University
1903 W. Michigan Avenue
Kalamazoo, MI 49008-5237
+ 1 (269) 210-9349

Editorial Board Member, American Journal of Evaluation (AJE) 
 

Greetings!

Chris’s point that it would be an eminently  sound idea to formulate in broad strokes, what ought to be undertaken after 2030 is well taken.

However, such an undertaking would be useful if and only if we have a reasonably clear notion of what has been done and not done with reference to the current set of SDG’s in spite of their lack of logical cohesion. Here, it is difficult to see how such knowledge may be acquired without evaluation.

Once this knowledge is at hand, we may hope that the next set of post 2030 development goals would be formulated in a way that is going to embody a greater logical cohesion with reference to the six fundamental human needs within people’s own cultural norms.

Knowledge of those fundamental needs are self-evident, hence jargon-free viz., nutrition with culinary enjoyment – after all, we are not yet akin to machines that just need fuel – good health, security in its broadest sense, education in its jutifiable sense, procreation and what may be called non-material goal which embraces aesthetic enjoyment, games and sports one takes part in, various other forms of entertainment. This is non-material for its satisfaction does not entail any material gain.

Most of the current SDG’s are subsumable under our fundamental needs since the former gain a value as a direct or an indirect means of enabling us to attain some fundamental need; hence they are secondary or tertiery needs constituting an interconnected hierarchical network of human needs whose ramifications reflect a society’s current state of development. This development may or may not be justifiable, for instance, promotion of rampant consumerism to achieve economic growth.

Cheers!

Lal.

To Chris' question, I am not certain that there will be a successor global development agenda, after 2030. Multilateralism is not in a good place right now. The MDGs were approved after the end of the cold war, but unfortunately, the cold war is making a come back...  So, whether there will be enough good will between nations in 2030 to arrive at a successor agenda remains to be seen. 

Further to Dorothy Lucks' comments headed "Beyond the SDGs", I had hoped for some suggestions about the next development framework, but this was clearly not her intention. 

One can trace a history of these from then-WHO DG Halfdan Mahler’s “Health for All by the Year 2000” via UN SG Kofi Annan’s Millennium Development Goals (noting that Kofi Annan had been a former WHO staff member) to the SDGs. Since the value of these frameworks lies more in the fact that they provide a framework rather than the rate of achievement of their goals and individual targets, it is important to start working well before the 2030 finish line for the SDGs on the next development framework – and particularly on the things left out so far. These include a separate treatment of knowledge and information in development. 

It is not enough to treat knowledge and information as “cross-cutting” in development. We are seeing what misinformation/disinformation/malinformation is doing to the edifice of development evidence. Information disorders are killing people at an increasing rate, as happened in the COVID-19 pandemic. Even when they are not fatal, they slow the pace of evidence-based human development. We need to be able to monitor and evaluate the health of knowledge as a specific goal in the next development framework – with goals, targets and indicators. 

Best,

Chris

Chris Zielinski 
Centre for Global Health, University of Winchester, UK  and 
President-elect, World Association of Medical Editors (WAME) 
Blogs; http://ziggytheblue.wordpress.com and http://ziggytheblue.tumblr.com 
Publications: http://www.researchgate.net and https://winchester.academia.edu/ChrisZielinski/

Dear Emilia,

Thank you for sparking this vital conversation regarding the role of evaluation in achieving the Sustainable Development Goals (SDGs). Evaluation is undoubtedly crucial in assessing progress, identifying effective strategies, and learning from experiences to hasten progress towards the SDGs. Here are some insights, specifically considering Zambia's context and its challenges in evaluation:

Incorporating SDGs in Evaluations: While we strive to align project objectives with specific SDGs in our evaluation frameworks, we acknowledge the unique challenges faced in Zambia. Limited resources and capacity constraints often hinder the comprehensive assessment of contributions to SDGs. Efforts are underway to enhance the integration, but resource constraints remain a significant obstacle.

Synthesis Studies: Conducting synthesis studies in Zambia has proven challenging due to gaps in data availability and quality. The lack of a centralized repository for evaluation findings makes it difficult to distill common lessons, best practices, and challenges across different projects. Strengthening data infrastructure is crucial to overcoming this limitation.

Demand Beyond UN Agencies: Zambia has shown a growing interest in SDG-related evaluations beyond UN Agencies, particularly at the national level. However, challenges persist at the sub-national levels, where there is a need for increased awareness and capacity building. The commitment is evident, but there is a necessity for targeted efforts to involve diverse stakeholders.

Engagement at Sub-National Levels: Sub-national entities in Zambia, including local governments and regional organizations, are gradually becoming more involved in SDG-related evaluations. Tailored approaches are essential, given the diverse local contexts, but there is a need for increased support and resources to ensure effective engagement at this level.

Challenges and Opportunities: Zambia faces challenges such as limited capacity, insufficient data, and complex interlinked SDG targets. The opportunities lie in leveraging international partnerships and experiences to address these challenges. Collaborative efforts, knowledge-sharing, and targeted capacity-building initiatives can turn these challenges into opportunities for sustainable development.

We are eager to participate in the upcoming workshop and webinar on evaluating contributions to SDGs. Through sharing experiences and insights, we hope to contribute to a more nuanced understanding of the role of evaluation in advancing sustainable development, with a specific focus on addressing Zambia's unique challenges.

Best regards,

Mupeta Lovemore

Dear Emilia

Thank you for bringing up this discussion.  I have been reading the contributions with interest and would like to add another perspective. The question that you raise about delving into concrete evaluation practices got me thinking about the depth and breadth of practice related to the SDGs. 

In my work as an evaluation external reviewer for several different organizations, in meta-evaluations, institutional level evaluations across national and multilateral organizations and in evaluation syntheses, I am involved in or read deeply at least a hundred evaluation reports in a year. The responses so far to this thread provide some really good practice examples that are at the pinnacle of SDG evaluation and these are hugely valuable, but we also don’t want to miss the less visible practice that is also contributing to the SDGs. 

To illustrate this, we can think of an atoll, an iceberg, a mountain range. The tips are visible but underneath there are masses that connect to the peaks. This led me to consider three key points, but there are undoubtedly more.

Beyond the SDGs

The SDGs do not sit in isolation. They were crafted as part of the 2030 Agenda for Sustainable Development – Transforming Our World. The SDGs are only flags on the pathway to a bigger summit. The SDGs are not the only processes that contribute to sustainable development – but they do help to provide focus. The tendency to focus on the few “SDG evaluations” does not account for the increasing number of evaluations that arise within countries and organizations that relate to strategic work that occurred through national and institutional responses for the 2030 Agenda commitments. These responses are reflected in national development plans, institutional change, shifts to multi-sectoral approaches to the SDGs or have more participatory approaches, to name a few influences of evaluation work. An example is the Multilateral Organisation Performance Network (MOPAN) that has incorporated assessment of the extent to which a multilateral organization, funded by the 22 countries that are members of MOPAN, has shifted its strategy and systems related to its mandate to align with the 2030 Agenda. These assessments are used by the organization to consider strategic and systematic improvements in line with the 2030 Agenda and other global commitments.

Below the SDG indicators

As countries and organizations shift, countries like Nepal, Ghana and many more plus organizations at all levels, have integrated the SDG indicators into their plans and acknowledged other factors of culture and country that were important, leading to a suite of indicators that are relevant to different contexts. As Ram says so clearly, these are now normal process and therefore we can look beyond and more deeply.  The VNRs are only one part of the process. The effect of the SDG indicators is only one part of the visibility of what is being done towards SDG achievements. Some evaluations I read are clearly linked to SDG response but may barely mention a link to a specific SDG, but together they generate a body of evaluative work that is valuable in progressing the 2030 Agenda.

Wider than the evaluation sector. The work that has been done through the National Evaluation capacities conferences and other evaluation capacity development initiatives has built evaluation capacity that has expanded and flowed down into other national and sub-national systems and local contexts.  If we subscribe to the principle that evaluations are designed to support accountability and learning for better design, implementation, performance and outcomes that lead to progress towards a more sustainable future, then the impact of the many evaluations that are being carried out at all levels are contributing to the 2030 Agenda results.

A realistic view. The above points are made, not to be idealistic and say that the evaluation sector is making good progress in evaluation related to the SDGs.  There are many crevasses and cracks and fault lines. Some areas are hidden and others are crumbling.  I, like so many others are disappointed that more is not being done. But let’s not be short-sighted and think only in terms of large-scale SDG evaluations and miss the mass of other valuable work that is going on.

With kind regards

Dorothy Lucks 

Executive Director, SDF Global,

Dear Emilia, 

I would like to share the experience of Finland, and how evaluations of the SDGs are being carried forward at different levels of government. 

Finland has been evaluating its progress in implementing the SDGs and has conducted two national evaluations of its 2030 Agenda implementation, both led by the Prime Minister's Office.

The first one was the independent evaluation called PATH2023, commissioned in 2018 by the Finnish government. This evaluation delved into policy documents applying a political economy framework to understand the underlying theory of change and scrutinized the state of sustainable development using the 4Is approach: institutions, interests, ideas, and information. It considered national sustainability indicators, policy objectives, and the global dimension of sustainable development. The analysis involved data from surveys, interviews with experts, and inputs from national workshops. Oversight came from a steering and a support group, with input from the Expert Panel on Sustainable Development and advice from the international evaluation community, in particular EvalPartners. 

In 2023, a second evaluation took place, emphasizing progress made since the previous one. This evaluation used qualitative methods, including document analysis, interviews, surveys, and benchmark analysis. Specifically, it involved 45 expert interviews from central government and workshops with 40 participants focusing on the Government's role and societal engagement in implementing Agenda 2030. A survey gathered opinions from various sectors, and a benchmark analysis of the Netherlands, Sweden, and Germany provided further insights. The evaluation identified, among others, bottlenecks in the governance model and made recommendations for further development of the Agenda2030 implementation. 

The national SDG evaluation process in Finland is intertwined with the Voluntary National Reviews (VNR): these are seen as a channel to feed findings into the strategic and planning cycle and to increase visibility and use of the evaluation in the country. The findings from the 2019 evaluation influenced political parties during parliamentary elections and shaped the 2019 Government program and the 2020 VNR. Similarly, the 2023 evaluation played a vital role in planning the new Government Program and will contribute to the 2025 VNR. 

Beyond the national level, the Development Evaluation Unit of the Ministry for Foreign Affairs commissions centralised evaluations of Finland’s development policy and cooperation, where SDGs are integrated through the various development policy objectives. The unit has also issued a guidance note on national evaluation capacity development and country-led evaluations.   

Finally, at sub-national and institutional levels, Finnish municipalities and local governments have taken an increasingly active role in incorporating the SDGs and their monitoring and evaluation into their work. Some local governments have developed digital solutions for SDG-related data. Similarly, government institutions and bureaus such as Social Insurance Institution of Finland and Finnish Tax Administration have also embarked on this path.

I am sharing some links and references below, and hope this experience can help other governments in approaching SDG evaluation. 

Nea-Mari

Lead Evaluation Specialist; Deputy Director 
Development Evaluation Unit (EVA-11) 
Ministry for Foreign Affairs of Finland
 

----

Berg, Annukka; Lähteenoja, Satu; Ylönen, Matti; Korhonen-Kurki, Kaisa; Linko, Tyyra; Lonkila, Kirsi-Marja; Lyytimäki, Jari; Salmivaara, Anna; Salo, Hanna; Schönach, Paula and Suutarinen, Ira (2019) PATH2030 – An evaluation of Finland’s sustainable development policy. Prime Minister’s Office, Helsinki.http://urn.fi/URN:ISBN:978-952-287-655-3

D'Errico, S., Geoghegan, T. and Piergallini, I. (2020). Evaluation to connect national priorities with the SDGs. IIED, London. Available at https://www.iied.org/17739iied

Haila, Katri; Salminen, Vesa; Roiha, Ulla; Uitto, Heidi; Vikstedt, Elina; Vinnari,  Eija; Vakkuri, Jarmo; Oreschnikoff, Aleksis and Uusikylä, Petri (2023) Assessing the implementation of the 2030 Agenda in Finland. Prime Minister’s Office, Helsinki. Report available (Finnish) and policy brief in English at: https://urn.fi/URN:ISBN:978-952-383-381-4

Ministry for Foreign Affairs (2022) HOW TO? Practical tips for supporting evaluation capacity and systems in partner countries. Available at: https://um.fi/documents/384998/0/Practical_tips_for_supporting_evaluation_capacity_and_systems.pdf/d1ed61b4-4e5a-0198-6f86-4c6766ae8887?t=1678449920584

Räkköläinen, Mari and Saxén, Anu (2022). ”Pathway to the Transformative Policy of Agenda 2030: Evaluation of Finland’s Sustainable Development Policy. In: Uitto, J.I., Batra, G. (eds) Transformational Change for People and the Planet. Sustainable Development Goals Series. Springer, Cham. https://doi.org/10.1007/978-3-030-78853-7_16

Prime Minister’s Office, Finland (2023) POLICY BRIEF 2023:7. Available at: http://urn.fi/URN:ISBN:978-952-383-381-4 (Accessed 07 December 2023). 

I conducted an evaluation on the Sustainable Development Goals (SDGs) in Mali. To reach our objectives, we conducted a participatory evaluation. 

Dear Emilia,

Thanks for raising this important issue.

1.      I have had some opportunities to be involved in the evaluation of some aspects of SDGs in Nepal. I find evaluating specific objectives or targets within the SDGs is a normal process and we can learn from the existing evaluation approaches. Evaluating sustainability at a higher order considering a space-time dimension of certain interventions during or at the end of the intervention may however be a challenging task. With my limited experience, I have encountered some challenges while evaluating SDG-related interventions. I feel it was good to view the scope of interventions at a higher level and consider the whole ecosystem of the selected interventions (as SDGs are intersectoral, collaborative and many more utopian terms), this theoretically appealing concept however posed some noted difficulties in evaluation. Some of them are narrow scope or project-based interventions (short term, thinly spread, too focused on activity delivery, isolated, weak MEAL systems and many more); traditional approaches and mindset of commissioning and managing evaluation (such as lack of system thinking approach & anticipatory views that intervention may lead in future); and operationalizing sustainability evaluation (such as weak intervention design, critical data gap, attribution gap and the limited time allocated to assess the condition for sustainability). In addition, there are substantive evaluation policy gaps, institutional failure and critical capacity gaps to assess sustainability at national and sub-national levels. In addition, I do not see much interest/enthusiasm to carry out ex-post evaluation of programmes even by large donors and development partners.

2.      Carrying out SDG evaluation is still an emerging thinking for many organizations including UN agencies. As SDGs are still new at the sub-national level, demand from the sub-national level is so far low. 

With best regards, 

Ram Chandra Khanal 

Hello, Emilia!

Let me begin with a comment which is going to be extremely unpopular which I have made in several other fora, viz., that the current list of SDG’s is logically flawed with respect to the justifiable needs of the people. Having said that, evaluation in this context faces two basic challenges:

1. At what level should evaluation be carried out to be useful to the policy makers and those who design policy implementation? It is difficult to envisage this done except nationally, regionally and locally within a country. International inputs ought to be within this framework.
2. There seem to be diverse foci of evaluation; however, as SDGs were intended to enhance the quality of life of real people, it is imperative that changes in quality of life should be the focus of evaluation. Obviously, this varies significantly from land to land, hence no standardisation is possible here.

It is a great pity that environmentalists of every ilk, nutritionists, health lobby etc., has not placed sufficient emphasis on halting the global population growth, and indeed reducing it. Unless this is done, the rest would remain chimeric.

Best wishes!

Lal.

Super helpful! thanks a lot for sharing the report Mark.

Best,

Stefano D’Errico

This may be coming at the question from a slightly different angle than Emilia intended, but as it happens, my team has just released a report on evaluation evidence (impact evaluations in particular) for the SDGs. We classified over 7,000 impact evaluation studies by SDG, with the goal of answering the question: "if I'm a policymaker working to improve progress on a particular SDG, how much evaluation evidence do I have to go on?". (The research questions are stated much more formally in the report, but that's the basic idea.)

We found that there has been a lot of evaluation work on "People" SDGs (e.g., poverty, food security, education) but the "Planet" SDGs have been largely neglected in the literature (with the exception of a big recent push by Chinese researchers to evaluate environmental regulations in China). 

We argue for a coordinated "evidence for sustainable development" agenda, which will ensure that a broad base of evaluation evidence is available to support evidence-informed decisionmaking across all the SDGs.

Just mentioning this here as it may provide some useful info for folks interested in the role of evaluation in achieving the SDGs.

Dear Emilia, I have assessed the extent to which a European Environmental Programme has been coherent with the Green Deal objectives and the SDGs. This analysis was based in matching the programme's thematic areas with the Green Deal’s actions and SDGs' targets. The results were very positive and revealed specific thematic areas of the programme which contribute the most (in terms of number of projects funded and budget allocated) to the Green Deal and the SDGs.