Thank you Ibtissem for raising this interesting discussion topic.
As external evaluator, my contribution is the following: I agree with you that “the quality of an evaluation isn’t solely shaped by the technical expertise of the independent evaluation team; and an effective evaluation management is also critical to its success”. With the objective to ensure the validity and reliability (to mention few) of evaluation results, Independent Evaluation Offices within development agencies quite often establish evaluation frameworks that provide a structured approach to manage evaluations and assure consistency across different projects.
To conduct effective evaluations IEOs commission external evaluators with cleat Terms of Reference (TOR) and also assign an evaluation manager from their office with a clear mandate. Now the core issue is: how should the independent evaluation team and the evaluation manager effectively shoulder their respective roles and responsibilities during the evaluation period? This is to guarantee the evaluation results are of high standard, credible, and dependable for the way forward and decision making.
1.Involvementof evaluation manages: The involvement of evaluation managers across the evaluation phases, should be strategically calibrated as per their mandate/ roles and responsibilities provided by the IEOs/their office/. As such, the independence of evaluations can be preserved. What is required from them is to effectively employ appropriate approaches to manage the evaluation process, which are indispensable for quality/high standard/credible evaluation results. In fact, it is very important that the roles and the responsibilities of the evaluation managers need to be clearly communicated with the external evaluation team from the outset, to avoid confusion (although, this communication is quite often overlooked. In conclusion, I strongly believe that if the evaluation team and evaluation managers perform a good job as per their mandate, the evaluation results are usually of high quality.
2. Role of evaluation managers: As noted in the above section, I believe that the role of an evaluation manager should not encompass active participation as a team member, to any degree regardless of their relevant technical expertise. Actually, to my understanding the role and responsibility of evaluation manager is not just supervising. There are diverse approaches developed by the IEOs that the evaluation manager should effectively employ, without which the findings of the external evaluation team cannot be of high standard (reference can be made to the diverse approaches developed by IEOs to manage evaluation). I do not see significant advantage from encompassing active participation as a team member in the role of evaluation managers. The disadvantages include: (i) there could be conflict of interest/biases, as they come from the same organization; (ii) independent evaluation may not be preserved; and (iii) their active participation can lead to lack of focus on their very mandate, which can affect the quality of the evaluation results.
3. Collaboration with evaluation managers: As an external evaluator my insight and experience in this regard is that there is often collaboration with evaluation managers and functions. The diverse approaches employed by the evaluation managers have been very instrumental to perform quality evaluation work. For sure, their participation has usually enhanced the relevance and utility of evidence for decision-making processes (the diverse approaches developed by IEOs to manage evaluations can be referred, which are documented in the IEOs. I strongly believed that collaboration with evaluation managers provided the managers effectively shoulder their roles and responsibilities is a must, and not a choice, for quality/credible evaluation results.
RE: Management matters: exploring the link between management models and the use of evaluations
Thank you Ibtissem for raising this interesting discussion topic.
As external evaluator, my contribution is the following: I agree with you that “the quality of an evaluation isn’t solely shaped by the technical expertise of the independent evaluation team; and an effective evaluation management is also critical to its success”. With the objective to ensure the validity and reliability (to mention few) of evaluation results, Independent Evaluation Offices within development agencies quite often establish evaluation frameworks that provide a structured approach to manage evaluations and assure consistency across different projects.
To conduct effective evaluations IEOs commission external evaluators with cleat Terms of Reference (TOR) and also assign an evaluation manager from their office with a clear mandate. Now the core issue is: how should the independent evaluation team and the evaluation manager effectively shoulder their respective roles and responsibilities during the evaluation period? This is to guarantee the evaluation results are of high standard, credible, and dependable for the way forward and decision making.
1.Involvementof evaluation manages: The involvement of evaluation managers across the evaluation phases, should be strategically calibrated as per their mandate/ roles and responsibilities provided by the IEOs/their office/. As such, the independence of evaluations can be preserved. What is required from them is to effectively employ appropriate approaches to manage the evaluation process, which are indispensable for quality/high standard/credible evaluation results. In fact, it is very important that the roles and the responsibilities of the evaluation managers need to be clearly communicated with the external evaluation team from the outset, to avoid confusion (although, this communication is quite often overlooked. In conclusion, I strongly believe that if the evaluation team and evaluation managers perform a good job as per their mandate, the evaluation results are usually of high quality.
2. Role of evaluation managers: As noted in the above section, I believe that the role of an evaluation manager should not encompass active participation as a team member, to any degree regardless of their relevant technical expertise. Actually, to my understanding the role and responsibility of evaluation manager is not just supervising. There are diverse approaches developed by the IEOs that the evaluation manager should effectively employ, without which the findings of the external evaluation team cannot be of high standard (reference can be made to the diverse approaches developed by IEOs to manage evaluation). I do not see significant advantage from encompassing active participation as a team member in the role of evaluation managers. The disadvantages include: (i) there could be conflict of interest/biases, as they come from the same organization; (ii) independent evaluation may not be preserved; and (iii) their active participation can lead to lack of focus on their very mandate, which can affect the quality of the evaluation results.
3. Collaboration with evaluation managers: As an external evaluator my insight and experience in this regard is that there is often collaboration with evaluation managers and functions. The diverse approaches employed by the evaluation managers have been very instrumental to perform quality evaluation work. For sure, their participation has usually enhanced the relevance and utility of evidence for decision-making processes (the diverse approaches developed by IEOs to manage evaluations can be referred, which are documented in the IEOs. I strongly believed that collaboration with evaluation managers provided the managers effectively shoulder their roles and responsibilities is a must, and not a choice, for quality/credible evaluation results.
With best regards,
Hadera Gebru
Addis Ababa, Ethiopia