I followed the conversations and contributions on this topic with much interest. I have been a mixed methods practitioner for several years both as a consultant and as part of my day job in the international development sector.
My experience is that applying mixed methods in evaluations is easier said than done. The main challenge is misaligned expectations and or understanding of mixed methods between commissioners of evaluations and those assigned to conduct evaluations. As a consultant regularly develop technical proposals or expression of interests for evaluation tenders. This process requires me to review several evaluation ToRs per day and I find that at least four in five ToRs specifically suggests or require mixed methods approach to be used. However, in the majority of the cases, there is often not sufficient time and or budget allocation to align with minimum necessities for carrying out a decent logical mixed methods approach.
Good evaluations require a good balance of complementary and, or supplementary quantitative and qualitative evidence. This therefore means regardless of whether the evaluation commissioner or the project has specifically asked for mixed methods, it is difficult in most cases to sufficiently address the standard evaluation questions without considering both quantitative and qualitative data. This means as an evaluator, you have only two choices; (1) either to operate within the time and budget limitations which might compromise the quality of evaluation results and impact on your professional integrity or (2) pay the price difference and go over and above the allocated time and budget in order to deliver quality.
Finally, I would say that the demand for mixed methods or expectations in program evaluations is here to stay but the development sector is still a long way in realising the need to aligning those expectations with necessary enablers; particularly, pricing and timeframes.
RE: How are mixed methods used in programme evaluation?
Dear Evaluators and Colleagues
I followed the conversations and contributions on this topic with much interest. I have been a mixed methods practitioner for several years both as a consultant and as part of my day job in the international development sector.
My experience is that applying mixed methods in evaluations is easier said than done. The main challenge is misaligned expectations and or understanding of mixed methods between commissioners of evaluations and those assigned to conduct evaluations. As a consultant regularly develop technical proposals or expression of interests for evaluation tenders. This process requires me to review several evaluation ToRs per day and I find that at least four in five ToRs specifically suggests or require mixed methods approach to be used. However, in the majority of the cases, there is often not sufficient time and or budget allocation to align with minimum necessities for carrying out a decent logical mixed methods approach.
Good evaluations require a good balance of complementary and, or supplementary quantitative and qualitative evidence. This therefore means regardless of whether the evaluation commissioner or the project has specifically asked for mixed methods, it is difficult in most cases to sufficiently address the standard evaluation questions without considering both quantitative and qualitative data. This means as an evaluator, you have only two choices; (1) either to operate within the time and budget limitations which might compromise the quality of evaluation results and impact on your professional integrity or (2) pay the price difference and go over and above the allocated time and budget in order to deliver quality.
Finally, I would say that the demand for mixed methods or expectations in program evaluations is here to stay but the development sector is still a long way in realising the need to aligning those expectations with necessary enablers; particularly, pricing and timeframes.
JT