Thanks to Malika for bringing up this issue of rapid evaluation, which shows once again the difficulties we often encounter in the practical implementation of certain theoretical notions.
My point of view on the issue is that of an institutional actor and not that of a consultant. In Benin we have started to conduct rapid evaluations, a new concept to which we have been exposed in South Africa with the "Twende Mbele" Programme (a cooperation programme in evaluation that we have initiated with South Africa and Uganda to strengthen our national monitoring-evaluation systems through the sharing of experience and the development of collaborative tools).
It is within the framework of this programme that we have developed a specific methodological guide on this type of evaluation and have simultaneously undertaken 4 rapid evaluations, 3 of which concern public interventions and the 4th relates to the effects of COVID-19 on the informal sector.
First of all, it must be said that the major difference between rapid and traditional evaluation lies in the constraints of time and limited resources that characterize rapid evaluation. In Benin, for example, a normal evaluation (excluding impact assessments which can take up to 5 years) takes on average 9 months to 1 year, or even longer, due to many factors related to procedures (notably administration, procurement, institutional management especially when there are many stakeholders), and sometimes even due to the data collection and analysis phase, which is often lengthy. Rapid evaluation therefore calls for new processes to reduce the time of the pre- and post-data collection phases. With the adaptation we made in Benin, the overall time for a rapid evaluation was estimated at 12 weeks maximum. This implies a data collection period of a maximum of two weeks to a month, to allow time for initial activities, organising data collection, analysing results, writing a draft report, obtaining observations and finalising the report. However, it is a reality check to see if this is realistic.
In terms of tools, the gap in time and scope can be filled by using rapid methods:
- Collection with groups (rather than individuals), workshops,
- Use of routine data or other evaluations,
- Team work to carry out different steps at the same time (Collection and processing)
Furthermore, we believe that in order to effectively save time, it is preferable for the rapid evaluation to be carried out by an internal team, because this is the only option that does not require a procurement procedure. But it does require that appropriate organizational mechanisms are put in place, for example:
- a project organisation chart for the team,
- the organisation of the weekly working time to be devoted to the evaluation mission and strict delivery deadlines,
- support measures for the team, etc.
In addition, as the evaluation team did not reside in the communities, we identified focal points in the data collection settings to facilitate with community members in order to save time.
Malika was not specific enough to enable us to propose solution approaches adapted to her context. But from the little I have retained, here are a few ideas that I share for the moment. I could provide more factual elements once we have learned from the experience currently underway at the Benin Public Policy Evaluation Office.
Thank you and good luck to all of you.
Elias SEGLA
[the original contribution is available on the French page]
RE: Évaluation rapide pour mesurer l'impact de la pandémie COVID dans les zones montagneuses
Dear community,
Thanks to Malika for bringing up this issue of rapid evaluation, which shows once again the difficulties we often encounter in the practical implementation of certain theoretical notions.
My point of view on the issue is that of an institutional actor and not that of a consultant. In Benin we have started to conduct rapid evaluations, a new concept to which we have been exposed in South Africa with the "Twende Mbele" Programme (a cooperation programme in evaluation that we have initiated with South Africa and Uganda to strengthen our national monitoring-evaluation systems through the sharing of experience and the development of collaborative tools).
It is within the framework of this programme that we have developed a specific methodological guide on this type of evaluation and have simultaneously undertaken 4 rapid evaluations, 3 of which concern public interventions and the 4th relates to the effects of COVID-19 on the informal sector.
First of all, it must be said that the major difference between rapid and traditional evaluation lies in the constraints of time and limited resources that characterize rapid evaluation. In Benin, for example, a normal evaluation (excluding impact assessments which can take up to 5 years) takes on average 9 months to 1 year, or even longer, due to many factors related to procedures (notably administration, procurement, institutional management especially when there are many stakeholders), and sometimes even due to the data collection and analysis phase, which is often lengthy. Rapid evaluation therefore calls for new processes to reduce the time of the pre- and post-data collection phases. With the adaptation we made in Benin, the overall time for a rapid evaluation was estimated at 12 weeks maximum. This implies a data collection period of a maximum of two weeks to a month, to allow time for initial activities, organising data collection, analysing results, writing a draft report, obtaining observations and finalising the report. However, it is a reality check to see if this is realistic.
In terms of tools, the gap in time and scope can be filled by using rapid methods:
- Collection with groups (rather than individuals), workshops,
- Use of routine data or other evaluations,
- Team work to carry out different steps at the same time (Collection and processing)
Furthermore, we believe that in order to effectively save time, it is preferable for the rapid evaluation to be carried out by an internal team, because this is the only option that does not require a procurement procedure. But it does require that appropriate organizational mechanisms are put in place, for example:
- a project organisation chart for the team,
- the organisation of the weekly working time to be devoted to the evaluation mission and strict delivery deadlines,
- support measures for the team, etc.
In addition, as the evaluation team did not reside in the communities, we identified focal points in the data collection settings to facilitate with community members in order to save time.
Malika was not specific enough to enable us to propose solution approaches adapted to her context. But from the little I have retained, here are a few ideas that I share for the moment. I could provide more factual elements once we have learned from the experience currently underway at the Benin Public Policy Evaluation Office.
Thank you and good luck to all of you.
Elias SEGLA
[the original contribution is available on the French page]