Principles for a healthy relationship between evaluators and experts

A common dilemma is striking a balance between team members who understand the technical subject and those who have expertise in evaluation methodologies and processes. In most evaluations, resources are scarce to hire big teams to cover all fields. Finding people who are experts in at least one discipline, have the right language skills and are skilled evaluators is challenging.

Here are some principles for creating a good working relationship between the roles of experts and evaluators in development evaluations based on my own experience as an evaluator, and on informal conversations with colleagues, notably in the last European Evaluation Society (EES) conference.

Manage expectations and reduce confusion

All evaluation actors should work to build a common understanding of the evaluation framework (objectives, approach, focus, methods, etc.). You will save a lot of time if, from the beginning, you spend time and effort to make sure you speak the same language and agree on roles and contributions. Do not assume that evaluation jargon is universal; other professions might have different definitions of the same words and concepts.

Ensure the evaluation team leader has the appropriate competencies to do the job

All evaluators, especially the team leader, should demonstrate competencies to do the job – a set of knowledge and skills that a practitioner needs. There is an ongoing debate on “professionalizing” evaluation practice in different contexts. None of the proposed frameworks appear to have been systematically derived or empirically validated through consensus built among diverse professionals across the field, but some are being developed, such as the development and adoption of competencies in Canada. Created as a key foundation of the Credentialed Evaluator designation under the auspices of the Canadian Evaluation Society (CES), five competencies domains were identified:

• 1.0 Reflective Practice: Competencies focus on the fundamental norms and values underlying evaluation practice and awareness of one’s evaluation expertise and needs for growth.
• 2.0 Technical Practice: Competencies focus on the specialized aspects of evaluation, such as design, data collection, analysis, interpretation, and reporting.
• 3.0 Situational Practice: Competencies focus on the application of evaluative thinking in analyzing and attending to the unique interests, issues, and contextual circumstances in which evaluation skills are being applied.
• 4.0 Management Practice: Competencies focus on the process of managing a project/evaluation, such as budgeting, coordinating resources, and supervising.
• 5.0 Interpersonal Practice: Competencies focus on people skills, such as communication, negotiation, conflict resolution, collaboration, and diversity.

Make it easy for “accidental evaluators”

If you are an experienced evaluator and have to work with experts new to the evaluation job, including many team members, make the process clear and engage everyone from the beginning. Create templates, provide resources, share timely feedback and listen to expert ideas and concerns.

Make sure experts are aware of evaluation norms and standards, and follow ethical guidelines such as confidentiality, respect of views and opinion, etc. Unlike traditional professions, the evaluation community has yet to see a proper “code of conduct”. Nevertheless, established evaluators do adhere to norms, ethical values and standards; usually gathered in documents, either with a universal scope, such as the UN Norms and Standards for Evaluation or tailored to a regional context, such as Evaluation Standards for Latin America and the Caribbean of the Latin American and Caribbean Network of Monitoring, Evaluation and Systematization (ReLAC).

Fight biases and check the validity of findings

Subject experts may have an inclination towards specific methodologies, techniques, and practices in the technical area of study, which is a real threat to the validity of evaluation findings. Experts as evaluators tend to focus on the technical side more than other equally important aspects.

If it is not within your means as an evaluation team leader to check the quality of what experts deliver, ensure that their work is reviewed both by peers and the evaluand. The larger the spectrum of peer reviewers, the more insightful the feedback collected can be and the credibility of findings strengthened.

Nevertheless, as experts are established authorities in their field they can continue the advocacy work, use evaluation findings in their field and shape the research and development agenda. On the other hand, evaluators tend to move on to the next evaluation, reducing the potential impact.

What are your thoughts on working with experts in evaluations? Leave a comment below.