Self-evaluation and internal evaluation.
One of the lessons shared during the United Nations Evaluation Group (UNEG) session on national evaluation capacity development at the Forum International Francophone de l’Evaluation (FIFE) in December 2021 and outlined in a recent blog post in EvalForward is:
“If we wish to attract and maintain the interest of more stakeholders in evaluation, we need to nurture it with open participatory processes and frequent evaluation exercises, including self- and internal evaluation. This will help to strengthen understanding of evaluation and advocate for the use of evaluative evidence. It is not contrary to the notion of independent and external evaluation to ensure rigorous methodology; the two can go hand-in-hand to help advance the cause and practice of evaluation in a country”.
I would like to subscribe to this conclusion and welcome the credibility given to self- and internal evaluation. Their potential to provide evaluative evidence to inform decision-making through a rigorous methodology is acknowledged. I have always noticed that self- and internal evaluation were discredited and sacrificed on the altar of the sacrosanct principle of the independence of evaluation. Indeed, it is in the name of this principle that Benin's 2012–2021 National Evaluation Policy favoured external evaluation. However, in practice, this option led to at least three consequences.
The limits of exclusive recourse to external evaluation in Benin.
First, this has not helped develop the evaluation capacities of managers of public projects and programmes, let alone those of central and local government officials. As proof, after ten years of implementation of this policy, the level of professionalism in evaluation still remains low.
Second, the independence of external evaluators is not absolute because, even if the mechanisms for granting consultancy contracts do not suffer from any formal procedural flaws, they are not always fully transparent. Under these conditions, the evaluation cannot be completely independent even if there is an external evaluator.
Third, the methodological rigour sought by external evaluations is not always demonstrated. Indeed, a meta-evaluation conducted by Benin’s Office of Public Policy Evaluation in 2021 showed that the public policy evaluations carried out in the country from 2007 to 2019 are of “acceptable” and modestly satisfactory quality.
The benefits of rapid assessments in responding to the needs of decision-makers.
Since 2020, Benin's Office of Public Policy Evaluation has started rapid evaluations adopting a "facilitated" approach whereby the evaluation is conducted by the internal project team with the support of an external evaluator. The experience has been very positive not only in terms of learning and capacity strengthening, but also for the quality of the evaluation. Indeed, the developed rapid evaluation methodology reduces the time needed to produce the report (6 to 12 weeks) and allows decision-makers, especially when under constraints of time and financial resources, to make timely decisions. It uses data provided by the project information system, enriched by qualitative primary data collected, processed and analysed in record time (one to three weeks) using collection tools (SurveyMonkey, Google Forms, etc.) and online interviews.
Furthermore, the case of Benin shows that pooling the efforts and experiences of internal project management teams with an external evaluator helps to produce better quality evaluations.
Innovative approaches such as rapid evaluation may therefore be the current solution to attract and maintain the interest of a larger number of stakeholders in evaluation and to adapt to the time needs of decision-makers, allowing them to make decisions in real time, as opposed to traditional assessments. On the other hand, conventional evaluations are more costly and therefore cannot be conducted frequently on the same intervention, beyond a mid-term evaluation and a final evaluation in the best case. As for impact evaluations, they are rare because they are even more costly.
The role of evaluation and monitoring in public decision-making.
Evaluation, as it is practised today, is essential for learning from our successes and failures in managing public interventions. It is also valuable for research and scientific progress thanks to the wealth of empirical knowledge that emerges from evaluative studies, especially in the economic and social fields. It is essential in the context of prospective studies, planning, programming or reviewing an intervention.
However, in order to help decision-makers make rapid and informed decisions based on evidence, especially in emergency situations, monitoring, based on a good statistical system, is often the most appropriate tool. In 2015, for example, the Government of Benin commissioned an evaluation of two agencies’ performance before awarding them operating grants. The two evaluation reports produced in 2018, three years after the Government's request, did not respond in time to the need expressed.
Thus, the Government of Benin has decided that in its institutional reform for the implementation of its 2021–2026 action programme, it will place the supervision of the monitoring system under the responsibility of the Presidency of the Republic and entrust the evaluation to the Ministry of Development in charge of planning. Indeed, the Presidency needs daily information on the level of implementation of projects, programmes and reforms, as well as rapid feedback on problems and bottlenecks in execution on the ground.
Evaluation has the advantage of providing and analyzing qualitative data that allows for lessons to be learned, and promotes learning in management practices. This is an important aspect that is missing in monitoring, which is limited to mostly quantitative data on the rate of physical and financial implementation of projects. This, however, has the advantage of meeting the needs of decision-makers, who are most often concerned with achieving their strategic objectives and being able to report to their constituencies on the progress of projects.
The challenge is therefore to combine the elements of monitoring and evaluation in order to bring together quantitative and qualitative data. The experience of rapid assessments in Benin has shown that it is possible to combine information from the monitoring system provided by project managers with primary qualitative data collected using technology tools, in order to inform decision-makers with actionable evidence in a reasonable time.
Monitoring and Evaluation Officer
Presidency of the Republic of Benin