Many evaluators work in contexts where there is little to no existing data to help identify persons with disabilities for participation in evaluations, and other consultation or assessments.
Even in countries where information is more easily accessible, evaluators may need to be more creative about who they contact, and how they adapt traditional evaluation methods to be more inclusive.
In response to the United Nations Disability Inclusion Strategy, the Office of Evaluation of the World Food Programme recently developed specific guidance on how to address disability inclusion in evaluation practice and organised an EvalXchange session which prompted a rich discussion on disability inclusion in evaluation.
One of the panellists, disability inclusion consultant, Kevan Moll, shared some practical tips on how evaluations can be more inclusive of persons with disabilities, captured below.
What are some of the key considerations to take into account when identifying persons with disabilities for an evaluation?
When identifying persons with disabilities as potential evaluation participants, a common tendency is to engage only with those who are the easiest to reach, the most visible and the most vocal.
To ensure the broadest possible perspective, range of views and experiences in any evaluation, it is important to engage with the full diversity of persons with disabilities by sex, impairment and age to the maximum extent possible.
Disability movements are not necessarily homogenous or cohesive structures. National Organisations of People with Disabilities (OPDs) may have a cross-disability focus in that they are theoretically representative of people with the most common impairments. However, it is important to also consider contacting impairment-specific organisations to ensure the widest possible scope.
It is also important to be aware that other organisations may exist at national, sub-national or local level for women, men, young people and children with disabilities.
Inclusion of the voice and perspective of people with speech, hearing, intellectual and psychosocial impairments can be more challenging. In this respect, the wider disability sector (NGOs working for or with people with disabilities) as well as parents or carers groups – where they exist – can also be valuable sources of support.
Can you suggest some practical strategies to identify persons with disabilities where limited data is available?
National or local OPDs can be a helpful starting point in identifying local contacts – whether individuals, groups, or networks – to provide basic information, advice and support. Web-based research, knowledge from non-government organisation (NGO) staff, local authorities and community leaders are all potential sources of information in this process. Parents and carers of persons with disabilities can also be a useful source of local contacts, knowledge and support.
In humanitarian situations, people with disabilities may already be registered as vulnerable for enhanced access to services. From this, groups can organise and leaders can emerge spontaneously or through facilitation who can then be engaged with.
In Timor-Leste in 2022, I was involved in a research and analysis study on the prevalence of people with disabilities by sex, impairment and age as well as general demographic data for an inclusive agricultural livelihoods programme. This included appointing the Executive Director of the national OPD as a Steering Committee member, as well as the direct involvement of local OPD personnel in the identification and survey of people with disabilities for interviews on attitudes and behaviour among and towards them in their communities. This process also enabled the OPD to expand their membership and better represent their rural constituents.
What advice can you share on the adaptation of more traditional evaluation methods to be more inclusive of persons with disabilities?
First of all, consider including people with disabilities in any and every aspect of the evaluation process wherever possible and not just as interviewees: this could mean in planning and design; as steering committee members, evaluators and enumerators; as practical support in the identification of, and communication with, participants; and as an audience for feedback of findings and input into any follow up response.
Women and men with different impairments as evaluators and enumerators can significantly enhance the quality of the discussion and information generated on disability issues and experiences.
Interviewees are more likely to share thoughts and experiences and be better understood and articulated by someone with a similar lived experience. As a result, the evaluation is better able to communicate, analyse and interpret findings and implications. In mainstream programmes, an evaluator with a disability opens the discussion on equality of access and inclusion.
With group meetings, think about the desired composition(s) of consultations: a mix of people with and without disabilities; women and men with a range of impairments; single impairment groups; single sex groups; youth or child exclusive groups. Different combinations generate additional insights.
Can you give any practical tips for preparing for an evaluation including persons with disabilities?
Irrespective of whether individual or collective exercises, preparation is essential. It is crucial to know your audience in advance and make any necessary provisions and adaptations.
Unless conducting one-on-one interviews through home visits, ensuring suitable access and inclusion may have financial implications in terms of finding appropriate venues, hiring of interpreters, transportation support and ensuring support from family members.
Ideally people with disabilities should speak for themselves if at all possible. If this is not possible, ascertain in advance who they want to represent or speak for them.
Interviews with people with communication impairments may be more productive in a one-to-one environment, together with their nominated advocate. These interviews will take longer, so decide beforehand whether quality or quantity of information is the priority (and any preferred mix of these) and allow additional time for such interviews.
For collective consultations, it is important to visit the venue before the meeting and ask yourself the following questions:
- Is the venue accessible and appropriate for people with physical/mobility, visual, speech/hearing and intellectual impairments? If not, what adjustments are needed?
- Is the venue close enough to participants’ homes to reduce the need to travel long distances? If journeys are long, over difficult terrain or in remote areas, are you willing and able to organise or refund any travel costs?
Are there specific guidelines to consider for people with hearing, visual or intellectual impairment?
For people with hearing impairment, ascertain whether participants are partially or completely deaf. What is their preferred form of communication: Sign Language, their own gestures or lip reading? Sign Language interpreters can often be accessed through local organisations of and for people with hearing impairment and other OPDs. For someone using their own gestures, ensure that they are accompanied by someone familiar with their individual communication style. During meetings, ensure that faces and hands are clearly visible (well-lit and with clear sight lines). Drawings and written information may also help with accuracy, clarification and confirmation.
For people with visual impairment, familiarise participants with the venue layout and facilities on arrival. At the beginning of any session, go around the group and ask all participants to say their name to facilitate an understanding of who is present and where they are placed. During the meeting, ask people to state their name before speaking so that everyone knows who is talking.
For people with intellectual impairment, the ability to understand depends on the degree of impairment. Simple language, short messages, repetition and patience are essential. Support from a family member or carer may also be necessary.
Some evaluations may be about programmes which already include people with disabilities, while others may not have a specific disability focus. Either way, it is good practice for all evaluations to consider inclusion and engagement of people with disabilities as a cross-cutting theme in any evaluation process.