Great discussion question from Jean and very insightful contributions!
First, I think Jean's question is very specific - that is how mixed methods are used not just in evaluations but PROGRAM evaluations, right? Then, we know that a program consists of two or more projects i.e. a collection of projects. Therefore, programs are rarely simple (where most things are known) but potentially complicated (where we know what we don't know) or complex (where we don't know what we don't know). Oxford English dictionary tells us that a method is a particular procedure for accomplishing or approaching something. Tools are used in procedures. I am from the school of thought that believes that when something is too complicated or complex, simplicity is the best strategy!
Depending on the context, program design, program evaluation plan, evaluation objectives and questions, the evaluator and program team can agree on the best method(s) that helps achieve the evaluation objectives and comprehensively answers the evaluation questions. I like what happens in the medical field, in hospitals where, except in some emergency situations, a patient will go through triage, clinical assessment and historical review by the Doctor, laboratory examination, radiology e.t.c then the doctor triangulates these information sources to arrive at a diagnosis, prognosis, treatment/management plan. Based on circumstances and resources, judgements are made whether all these information sources are essential or not.
Mixed methods is great, but the extent of using mixed methods and sequencing should be based on program and evaluation circumstances, otherwise instead of answering evaluation questions of a complex or complicated program, we end up with data constipation. Using all sorts of qualitative methods at once i.e. open ended surveys, KIIs (Key Informant Interviews), community reflection meetings, observations, document reviews e.t.c in addition to quantitative methods may not be that smart.
In any case, perhaps, the individual projects within the program have already been comprehensively evaluated and their contribution to program goals documented, and something simple like a review, is what is necessary at the program level.
When complicated or complex, keep it simple. Lean data.
RE: How are mixed methods used in programme evaluation?
Greetings to all!
Great discussion question from Jean and very insightful contributions!
First, I think Jean's question is very specific - that is how mixed methods are used not just in evaluations but PROGRAM evaluations, right? Then, we know that a program consists of two or more projects i.e. a collection of projects. Therefore, programs are rarely simple (where most things are known) but potentially complicated (where we know what we don't know) or complex (where we don't know what we don't know). Oxford English dictionary tells us that a method is a particular procedure for accomplishing or approaching something. Tools are used in procedures. I am from the school of thought that believes that when something is too complicated or complex, simplicity is the best strategy!
Depending on the context, program design, program evaluation plan, evaluation objectives and questions, the evaluator and program team can agree on the best method(s) that helps achieve the evaluation objectives and comprehensively answers the evaluation questions. I like what happens in the medical field, in hospitals where, except in some emergency situations, a patient will go through triage, clinical assessment and historical review by the Doctor, laboratory examination, radiology e.t.c then the doctor triangulates these information sources to arrive at a diagnosis, prognosis, treatment/management plan. Based on circumstances and resources, judgements are made whether all these information sources are essential or not.
Mixed methods is great, but the extent of using mixed methods and sequencing should be based on program and evaluation circumstances, otherwise instead of answering evaluation questions of a complex or complicated program, we end up with data constipation. Using all sorts of qualitative methods at once i.e. open ended surveys, KIIs (Key Informant Interviews), community reflection meetings, observations, document reviews e.t.c in addition to quantitative methods may not be that smart.
In any case, perhaps, the individual projects within the program have already been comprehensively evaluated and their contribution to program goals documented, and something simple like a review, is what is necessary at the program level.
When complicated or complex, keep it simple. Lean data.
My thoughts.
Thanks.
Gordon