RE: The pervasive power of western evaluation culture: how and in what ways do you wrestle with ensuring evaluation is culturally appropriate and beneficial to those who legitimise development aid? | Eval Forward
The discussion on cultural issues and evaluation, put forward on the EvalForward platform by Daniel Ticehurst, reminds me of an anecdote told in some African countries: the story of a hole on a main road, 3 km away, which was repaired every year and which reopened every year during the rainy season and therefore caused many serious accidents. And although the hospital was just 3 km away, the lack of ambulances made it even more difficult to deal with the problem. So the government decided to close the hole with concrete so that it would never open again... and open another hole right next to the hospital to solve... the problem of lack of ambulances.
By starting my contribution with this anecdote, I want to be a bit provocative in saying that trying to address cultural issues at the time of the evaluation is in fact a rather late debate in the socio-economic development process, especially if one agrees that any development action must be inherently participatory, and that the involvement of beneficiaries must be concretised quite early in two important initial stages of a development process (elaboration of the development problem, and identification of development/change objectives). Such a strategy is essential if beneficiaries are to be actively involved in the later stages of the development process (implementation/realization, monitoring, evaluation, hand-over, etc.).
Reading some of the contributions, many of which remained rather technical and methodological, I felt that in the early 1980s, when Robert Chambers' book "Rural Development' Putting The Last First" was published, many of the cultural aspects in development processes had been discussed from the outset. This identifies the six major prejudices (or biases) hindering the contact of outsiders with rural "reality" in general, and with deepest rural poverty, in particular.
The discussion on cultural aspects in the context of an evaluation is certainly interesting, but it cannot be denied that it remains a rather reductionist debate presenting the evaluation of a development action as if it were an independent island, whereas it is just one stage in the process of that development action and, moreover, intervenes at a moment when that process is quite advanced in time and space, and has perhaps reached a point of no return in its evolution.
Among the contributions on this theme, there are 3-4 that have - in my humble opinion - identified the problem of the frequent lack of consideration of cultural aspects not only during the evaluation, but also during the whole process of the development action: these are, according to the temporal order of the contributions.
(1) Njovu Tembo Njovu does not comment, rather he makes a case for the philosophy of development in the world being quite dominated by Western ethnocentrism; this remains an undeniable truth and I must say that I largely agree with the content of this argument. He states that "the global evaluation system is dominated by notions of investigation emanating from the Western, patriarchal, white-privileged Global North... and that national evaluation systems are controlled by donors". This refers to the seriousness with which the monitoring and evaluation of development actions are taken into consideration in Southern countries, which remain highly dependent on Western countries for financial resources to be allocated to monitoring and evaluation activities. And the iron law of Western countries in terms of development is that if they open budgets for Southern countries for development actions and related monitoring and evaluation activities, then part of these budgets must be used in the form of human resources from these donor countries. And then we fall back into the vicious circle of the expatriate who will formulate the development action with his Western blinders having only very ephemeral knowledge of the socio-cultural aspects of the environment in which the development action in question will evolve, and then what about our present debate on the cultural aspects and evaluation. It is therefore necessary to work, as Silva Ferretti mentions, to develop the necessary tools that can enable expatriates to integrate local knowledge into their development actions, giving more consideration to the fact that the beneficiaries can be both learners and teachers at the same time (dixit Njovu). This would lead to the decolonisation of development work in general, and evaluations in particular, as Ventura Mufume suggests, and avoid perpetuating "the culture of white supremacy" (dixit Silva Ferretti).
(2) Eriasafu Lubowa believes that "the challenge of cultural sensitivity would be partly resolved in the design phase by a thorough participatory stakeholder analysis, undertaken during the development of the results framework and indicators". For this person, the active participation of the main stakeholders of a development action, especially the beneficiaries, in the design phase, and in the monitoring of the implementation, should minimise/counteract the problems of cultural sensitivity that would arise during the evaluation. Thus, this statement adds weight to my argument that it is somewhat late to address cultural aspects at the time of evaluation; they certainly need to be addressed at the very beginning of the development action process, i.e. at the design stage.
(3) Ram Chandra Khanal states that "For various reasons, cultural issues are less represented in the evaluation design and subsequent phases. When designing an evaluation, most survey and observation methods and tools do not take into account context, space and time, and are mainly focused on outcomes and their associated indicators." This is the dilemma of evaluation, which quite often remains within the results framework of the development action, elaborated years before by a team of expatriates who did not have enough knowledge about the cultural and social environment in which the development action would evolve (example of a World Bank project on animal traction for land ploughing in Africa). And as Silva Ferretti says so well, if we accept that evaluation means "results, indicators", we may be killing the possibility of a cultural appropriation of the evaluation, and perhaps even of the development action as a whole, from the outset. This is especially true as many practitioners equate evaluation with the documentation of results and indicators, which can distract them from other avenues of analysis.
So, in the end, the evaluation stage alone cannot take responsibility for addressing the cultural aspects of the environment in which a development action is carried out, although it may be able to do so within the strict heuristic framework that an action research activity such as an evaluation can offer. And so the real discourse on these aspects needs to be planted in the design phase of that development action by allocating more space and active role to the beneficiaries and reducing the white supremacy of expatriates who can in no way hold the "absolute truth".
RE: The pervasive power of western evaluation culture: how and in what ways do you wrestle with ensuring evaluation is culturally appropriate and beneficial to those who legitimise development aid?
The discussion on cultural issues and evaluation, put forward on the EvalForward platform by Daniel Ticehurst, reminds me of an anecdote told in some African countries: the story of a hole on a main road, 3 km away, which was repaired every year and which reopened every year during the rainy season and therefore caused many serious accidents. And although the hospital was just 3 km away, the lack of ambulances made it even more difficult to deal with the problem. So the government decided to close the hole with concrete so that it would never open again... and open another hole right next to the hospital to solve... the problem of lack of ambulances.
By starting my contribution with this anecdote, I want to be a bit provocative in saying that trying to address cultural issues at the time of the evaluation is in fact a rather late debate in the socio-economic development process, especially if one agrees that any development action must be inherently participatory, and that the involvement of beneficiaries must be concretised quite early in two important initial stages of a development process (elaboration of the development problem, and identification of development/change objectives). Such a strategy is essential if beneficiaries are to be actively involved in the later stages of the development process (implementation/realization, monitoring, evaluation, hand-over, etc.).
Reading some of the contributions, many of which remained rather technical and methodological, I felt that in the early 1980s, when Robert Chambers' book "Rural Development' Putting The Last First" was published, many of the cultural aspects in development processes had been discussed from the outset. This identifies the six major prejudices (or biases) hindering the contact of outsiders with rural "reality" in general, and with deepest rural poverty, in particular.
The discussion on cultural aspects in the context of an evaluation is certainly interesting, but it cannot be denied that it remains a rather reductionist debate presenting the evaluation of a development action as if it were an independent island, whereas it is just one stage in the process of that development action and, moreover, intervenes at a moment when that process is quite advanced in time and space, and has perhaps reached a point of no return in its evolution.
Among the contributions on this theme, there are 3-4 that have - in my humble opinion - identified the problem of the frequent lack of consideration of cultural aspects not only during the evaluation, but also during the whole process of the development action: these are, according to the temporal order of the contributions.
(1) Njovu Tembo Njovu does not comment, rather he makes a case for the philosophy of development in the world being quite dominated by Western ethnocentrism; this remains an undeniable truth and I must say that I largely agree with the content of this argument. He states that "the global evaluation system is dominated by notions of investigation emanating from the Western, patriarchal, white-privileged Global North... and that national evaluation systems are controlled by donors". This refers to the seriousness with which the monitoring and evaluation of development actions are taken into consideration in Southern countries, which remain highly dependent on Western countries for financial resources to be allocated to monitoring and evaluation activities. And the iron law of Western countries in terms of development is that if they open budgets for Southern countries for development actions and related monitoring and evaluation activities, then part of these budgets must be used in the form of human resources from these donor countries. And then we fall back into the vicious circle of the expatriate who will formulate the development action with his Western blinders having only very ephemeral knowledge of the socio-cultural aspects of the environment in which the development action in question will evolve, and then what about our present debate on the cultural aspects and evaluation. It is therefore necessary to work, as Silva Ferretti mentions, to develop the necessary tools that can enable expatriates to integrate local knowledge into their development actions, giving more consideration to the fact that the beneficiaries can be both learners and teachers at the same time (dixit Njovu). This would lead to the decolonisation of development work in general, and evaluations in particular, as Ventura Mufume suggests, and avoid perpetuating "the culture of white supremacy" (dixit Silva Ferretti).
(2) Eriasafu Lubowa believes that "the challenge of cultural sensitivity would be partly resolved in the design phase by a thorough participatory stakeholder analysis, undertaken during the development of the results framework and indicators". For this person, the active participation of the main stakeholders of a development action, especially the beneficiaries, in the design phase, and in the monitoring of the implementation, should minimise/counteract the problems of cultural sensitivity that would arise during the evaluation. Thus, this statement adds weight to my argument that it is somewhat late to address cultural aspects at the time of evaluation; they certainly need to be addressed at the very beginning of the development action process, i.e. at the design stage.
(3) Ram Chandra Khanal states that "For various reasons, cultural issues are less represented in the evaluation design and subsequent phases. When designing an evaluation, most survey and observation methods and tools do not take into account context, space and time, and are mainly focused on outcomes and their associated indicators." This is the dilemma of evaluation, which quite often remains within the results framework of the development action, elaborated years before by a team of expatriates who did not have enough knowledge about the cultural and social environment in which the development action would evolve (example of a World Bank project on animal traction for land ploughing in Africa). And as Silva Ferretti says so well, if we accept that evaluation means "results, indicators", we may be killing the possibility of a cultural appropriation of the evaluation, and perhaps even of the development action as a whole, from the outset. This is especially true as many practitioners equate evaluation with the documentation of results and indicators, which can distract them from other avenues of analysis.
So, in the end, the evaluation stage alone cannot take responsibility for addressing the cultural aspects of the environment in which a development action is carried out, although it may be able to do so within the strict heuristic framework that an action research activity such as an evaluation can offer. And so the real discourse on these aspects needs to be planted in the design phase of that development action by allocating more space and active role to the beneficiaries and reducing the white supremacy of expatriates who can in no way hold the "absolute truth".
Mustapha Malki, PhD
Specialist in monitoring and evaluation
Freelance consultant
[Originally posted in French]