Lal - Manavado

Greetings!

As for an evaluator’s ability to suggest a better approach to solving a problem, I think one must take two aspects of the matter into consideration.

First, an evaluation is undertaken to ascertain how successful a given approach is to achieving some pre-determined objective. In my example, it was improving public health of an unnamed country. The political authorities opted for an ultra-modern cardiac unit in the capital of a land where there was hardly any primary health care for the majority.

Durinng pre-project evaluation, this would be obvious to an evaluator who looks at reality as it is, rather than as an academic exercise. True, it is not always as simple as this seems to be. Even so, I believe an evaluator who is not afraid to apply his common sense to the existing local realities of a given place would be able to make some sensible suggestions on some generic changes to a plan intended to attain a goal. The evaluator may not be competent to recommend an specific action, but generic changes ought to be within his ken.

In the ‘public health’ example, it is obvious to an informed evaluation that primary health care has a logical priority over a fancy cardiac unit of limited utility. Of course, he would not be competent to recommend the nuts and bolts of how such a health care system may be established.

Cheers!

Lal.

PS:

Let us remember evaluation is concerned with enhancing the quality of life of real people in some way, and it is not to be conflated with some abstract enterprise dealing with theoretical entities.

Greetings!

I have followed this discussion with interest, and it seems to me that the point one tries to make here is that evaluation ought to bring about a desirable change in the way a policy/strategy/tactic i.e., a field implementation is intended to attain its objective. Otherwise, evaluation would be just ‘much ado about nothing.’ Be it an impressive report or a set of colourful graphics. Here, I cannot agree more with Sylva.

Other participants have already noted several obstacles to progress such as political expediency, incompetence, corruption, indifference among the decision-makers, lack of resources, unacceptable donor interference, etc. All these assume that a given evaluation has been understood, but ...

We can hardly take this ‘understood’ for granted; I think this is the point Sylva is raising here. If I am right, the question then is what precise form an evaluation ought to take in order to facilitate such an understanding while hoping that it might induce the policy makers/strategists/field planners to revise their approach towards achieving a pre-determined goal.

In other words, evaluation would then guide the revision of the previous approach towards attaining the same objective. This process may have to be repeated as other conditions influencing achievement of a goal could change. An extreme example of such an influence is the present Corona infection.

Here, we have identified two basic problems:

1. How to make ‘planners’ understand an evaluation.
2. How to induce them to revise their plans in line with an evaluation. It seems that this is far more difficult, especially in view of the obstacles we have just mentioned earlier.

However, restricting ourselves to our first question, I might suggest an evaluation take the form of a short critique of the generic actions a plan embodies. As a concrete example, let us saya plan suggests that in order to improve public health, the authorities plan to put up an ultra-modern cardiac unit in the capital of a country. The donor is full of enthusiasm and endorses the project. Meanwhile, the country involved hardly offers primary health care to its citizens.

Here, in my view, the pre-project evaluation would be short and lucid, and would run as follows:

“This project would have an extremely limited beneficial effect on the public health of the country, and it is proposed that the available funds and human resources are deployed to provide primary health care at centres ocated at X, Y, Z etc.” This is something that has actually happened and I have suppressed the country and donor’s names. I do not think the actual evaluation report looked anything like my version, but it must have been impressive in its thickness and uselessness.

So, are the evaluators willing and able to concentrate on the practical and guide the hands that feed them towards some common good with few lucid jargon-free sentences?

Cheers!

Lal.

Greetings!

Rather than answering the set of questions posed here, one would think that it might be useful to offer a possible way forward an approach that may help one to overcome the difficulties previously discussed. 

Naturally, it will require a great deal of work as the number of people one will have to influence is very large. 

Best wishes!

Lal Manavado

(see attachment) 

Greeting!

Incorporating evaluation into national policies faces several challenges. It is not very easy to distinguish clearly the differences in evaluation at national, regional and local i.e., field levels. This is very important, because what evaluation involves at those levels are very different. The attachment included here might prove useful in navigating through those difficulties.

Cheers!

Lal Manavado

Greeting!

Quite a few instances of 'racism' in evaluation have been presented in this discussion. However, I find it hard to understand just how this awareness could enable one to deal with it. Of course, one could compile a long document on such examples and make it widely available. Even so, how is that going to help?

My next point is that the term 'racism' is not well chosen. What it does is to confine a certain sub-type of discrimination to the differences in skin colour. There are many instances of such discrimination even within groups of same skin colour.

For instance, say among people of X coloured skin, this kind of discrimination may occur across the following barriers:

   1. Urban and rural divide.

   2. What school/university one attended.

   3. Religion.

   4. Sex.

   5. Caste/social class.

   6. Nepotism and corruption.

Discrimination of the kind under discussion can be motivated by any one or more of the six reasons given above within a group having the same skin colour. I believe that it would be unsound to ignore those, but to avoid excluding them, one has to view the problem as an instance of discrimination rather than racism.

As for how to resolve this problem, it must be noted that it is a social issue arising from lack of ethical standards. Please note I am speaking of secular ethics which I prefer to call standards of common decency. It would be naïve to believe that legal measures could be of any help because there is a huge difference between having the 'right laws' on a country's statute books, signing of international conventions on one hand, and their actual enforcement on the other. So, I believe it would take some time to deal with this problem and it would require public education now and the incorporation of personal ethics into school education. Much learned talk may give one a sense of having done something, but that would hardly address the problem in the real world.

Best wishes!

Greeting!

We shall continue from where we left off in our previous contribution, which is given below. Avoiding all arcane expression typically in use, let us point out a few important facts that are important to real disabled people with reference to agricultural pursuits:

   • Certain disabilities like blindness or severe visual impairment will exclude people from engaging in agricultural work in any meaningful way.

   • This will also apply to people with severe to limited mobility owing to leg injury or Mysthenia gravis, etc.

   • Certain mental dysfunctions may preclude the sufferers from participation in agricultural pursuits.

   • Thus, the opportunities available to disabled people in agriculture are limited owing to certain physical or mental handicaps from which they suffer. This happens to be the situation in real life.

   • However, deaf people and those who suffer from other disabilities  may be able to fruitfully engage in agriculture provided that what they plan to do do not impose an undue strain on them.

   • What those people may usefully do depends on the real life conditions that exist in the area they live. We cannot suggest any meaningful generic improvements unless we have area specific information.

We hope project planners and evaluators begin right there and start from the beginning rather than basing their work on theoretical or academic papers. After all, we are trying to ameliorate the lives of real people, arn't we?

Best wishes!

Lal Manavado.

Greeting!

Speaking in general terms about evaluation, there does not seem to be a wide-spread agreement on what exactly one intends to evaluated. Avoiding the use of any jargon, we would have thought that what really matters is to evaluate whether a project/programme has succeeded in improving the quality of life of the target group it was designed to benefit. This is not the same as the successful physical conclusion of any development effort. For instance, in an earlier discussion successful completion of a modern motor way and a bridge to join an island and the main land had been cited as vain efforts.

We have pointed out that meeting the following criteria is essential if a project is to benefit its target group:

1. It is sustainable by the target group with respect to the physical resources available to it.

2. The competence required to run and maintain it are within the available skill set of the target group.

3. The project makes a significant contribution to improve the education in its real sense, health, nutrition, security in its broadest sense, procreation and what we have termed non-material needs. This last includes aesthetic enjoyment i.e., literature, music, etc., participation in games and sports, etc. Improvements in any one or more of them will increase one's quality of life.

After these introductory remarks, it is clear what we need to find out is how and to what extent agricultural pursuits may contribute to the quality of life of the disabled people. We hope that work on this area will succeed in determining the what is possible to achieve in clear and concise terms.

Best wishes!

Lal Manavado.

Dear Mr. Molloy,

I am happy to see that you have raised the question, does evaluation ascertain the changes in the quality of life of the disabled following the completion of a programme/project? Naturally, such changes can be positive or negative. Unfortunately, some efforts at 'modernisation seem to have a negative effect on the disabled, especially on those living in less affluent countries.

For instance, pdf documents are supposed to be 'better looking' than simple text files or doc files. But this is not easily accessible to the visually impaired living in poor countries, because the software needed to make such documents accessible to the visually impaired is just too exprensive for most of them. Moreover, the pc's needed to run them are too expensive for the target group named here.

I suggest you get in touch with the organisations for the disabled in as many countries as possible via e-mail and request their views. Of course, this may provide some linguistic difficulties as personnel in many such organisations do not speak standard English or only know their own language.

Be that as it may, you might find some useful bits of information from the real world from this source. Meanwhile, I am a little sceptical about published materials, because they are usually written by the local non-handicapped people who are educated in cities and have little notion about how the disabled in the rural areas live.

I hope this would be of some use to you, and wish you every success.

Best wishes!

Lal Manavado.

Yes, monitoring is important in evaluation, but it is necessary to understand that unless one has decided in advance what exactly one is going to monitor, it serves no useful purpose. This 'what' is determined by achievement of what result one is going to ascertain. It is easy to overlook this vital logical fact, and often this happens. Thus, monitoring is logically subsumed by the evaluation for which it is intended.

Best wishes!

Lal Manavado.

Greetings!

I hope that I may be forgiven for presenting a few justifiable reasons to demonstrate that the much-maligned top-down approach does not mean dictating to a target group be they farmers or any other category of actual workers who really do the job.

For instance, take the case of a single farmer; if he is experienced, not necessarily loaded with impressive diplomas in agriculture, would not he first consider the type of soil in his property, access to water, climate, kind of crop or animals fro which he is certain to have a demand, etc., before he begins his work? If anyone should deny this, one is just assuming that farmer is just ignorant, which is very often unjustifiable. 

If one should agree that our farmer does  so, then, he is using top-down approach to see what is most suitable for him to grow and/or raise on his land, because that will enable him to achieve his goal of succeeding in procuring enough food for himself and selling the rest to meet his other needs.

The problem is not in the method, but in its fragmented application. What happens is it is incompetently applied at highest level i.e., policy formulation and even less competently at the strategic implementation level. At regional and local levels, what comes from the top is just passed over as it is. This is a common chain of incompetence very wide-spread throughout the globe. The term top-down' is bandied about often by those who do not understand what it is, or what it involves.

It is a staggered approach where at each level both the 'big picture goal' which is general becomes more and more area-specific on implementation. Finally, at our farmer's level where it matters most, it becomes a process of reconciliation between his goal through its integration into the national whole. It is as simple as that when shorn of jargon and the clichés of vested interests.

So, farmer's wishes and feed back is an integral part of a competently applied top-down solution.

Cheers!

Lal Manavado.

Dear all,

I cannot agree more with Silva's comments, and if I may, I would like to add a few remarks.

I think it would be reasonable to maintain that the fundamental purpose of evaluation would be to ascertain whether an action, a project for instance, has succeeded in contributing to the quality of life of its target group.

This quality enhancement can be brought about by enabling them to satisfy one ore more of their fundamental needs. Nutrition is one of those needs. Its satisfaction obviously depends on the availability and affordability of wholesome local food, which in turn depends on the adequacy ofthe the local food system. The same applies to the set of needs associated with the other fundamental needs.

Often, the adequate satisfaction of a fundamental need requires the prior satisfaction of the needs associated with the satisfaction of some other fundamental need. For example, An adequate nutrition frequently depends on transport of food. Transport thus becomes a necessity not only for adequate nutrition, but also for other fundamental needs like education, health, security, etc.

So, I think a holistic empirical approach to evaluate/acertain the completeness of a project during planning stage would be critical for the success of any action. After all, we undertake a project to bring about a desired change, viz., to enhance the quality of a group's life in some way. This not theoretical, and it is eminently practical. Ideally, it ought to be pro-active, but there is too much below par today to think about that.

Target group is supposed to take over the running and maintenance of a project on its completion to some degree. They are not an isolate, but a part of a larger community. Thus, it is important to ascertain whether the target group and/or the community to which it belongs are able to continue the adequate operation of the project on its completion.

I think the variables involved here are just too numerous to be taken into account by any theoretical norm. Even after establishing its environmental and social impact with reference to its result and the tools a project proposes to use, and a community's willingness and ability to use them with sufficient skill, there are many more imponderables one will have to face. These have to be examined on the spot. 

Best wishes!

Lal

Dear Ines,

Many thanks foryour comments, and my apologies for my delayed reply.

Perhaps, I might mention that I am very reluctant to use the term 'theory' on what is obviously an approach used to achieve some concrete result, which in this case is to get youth to engage in agricultural pursuits to a greater extent than now. I prefer to use 'theory' to describe in generic terms fundamentals that govern gnomic phenomena, i.e., interactions among non-thinking entities like planets, atoms, molecules, etc.

Having said that, I think the approach we are interested in makes one crucial distinction, viz., the difference between the 'desired result' and the 'actual result'. In a previous discussion on this forum, a contributor mentioned a very expensive modern highway built in a developing country. The desired result included its regular use by the locals to transport goods, which in turn would improve the 'local economy'. But the actual result was that it was a success as far as the road itself went, but was hardly used by the poor target group who did not has sufficient access to the required wheeled transport, nor yet enough local produce to send away!

This is where the pre-evaluation of any endeavour should come in, i.e., a project, programme, etc. And it should be holistic. Had it been done in the case of that road, first thing that would have come to mind is the annual surplus to be transported, the need for it, availability of appropriate transport vehicles, etc. If those did not obtain in certain minimal amounts, the road project would have been either reduced to a more realistic level or scrapped altogether.

As you will notice, this approach entails that there are so many variables one has to take into account for a meaningful pre-evaluation. Hence, it is logically impossible  to draw universally applicable evaluation guidelines that are pragmatically justifiable. After all, what we want is a pragmatic result that benefits a target group, and not an academic activity. In the present case, the acid test is, has the project or the programme  resulted in a significant increase in the numbers of young people taking up agricultural pursuits in a given area, and will they continue to do so? If the answer to these questions is a no, the endeavour has been a failure. We shall always have to keep this in mind.

I'm not familiar with the situation in India, and you point out that most young people in developing countries lack appropriate skills in agriculture.

I am not at all certain that cultivation of Himalayan foot hills would be such a good idea in the long term, for the geology of the sub-continent makes it totally dependent on monsoon rain as its principal source of water.

Snow and ice on the mountain ranges in the area from Hindookush range via the Himalayas and to the Chinese ranges depend on the monsoon moisture.

Unless sufficient water goes into the soil through seasonal rainfall and snow and ice melts, rivers in the area would dry up and this would lead to a catastophe. I am told that the major rivers in the area from Kablll river eastwards have shrunk considerably during the past century. Thus, deforestation of the Himalayan foot hills would retard soil uptake of monsoon rains even though the 'organic farming' would enable some people to earn large profits for a short time. According to the Imperial Gazeteer of India the soil in that area is not particularly fertile. Incidentally, no geological and climatological project of greater comprehensiveness and complexity other than that undertaken by the British in India has been carried out anywhere else in the world. I know it is not fashionable to say this, but it remains a fact. (Ref. Determination of the geode of the Indian sub-continent in order to ascertain the height of Mt. Everest.)

Best wishes!

Lal.

Greetings!

How appropriate is our current approach to evaluation?

In my previous comments to this forum, I have underlined some aspects of evaluation, which I find difficult to justify. As it is currently understood, evaluation seems to be restricted to the result of a project/programme with reference to a formal list of what it is intended to achieve. Shorn of graphs, tables, etc., and a descriptive text, this is the gist of evaluation today. But can it really inform us of anything more than its methodological incompleteness? We would prefer to think otherwise, but our desires and hard reality are two different things.

I think it will be generally agreed that the purpose of any rational agriculture project to promote youth participation in agricultural pursuits would be the following:

   1. Induce greatest possible number of young people to engage in agricultural activities.

   2. Make a significant contribution to a sustainable local food production not opposed to the food culture of the area. Sustainability of this depends on its being environment friendly and supporting bio-diversity in agriculture.

The above two points are logically  inseparable.

The contributors to the present discussion have emphasised two points, viz., education and financing, while making a tangential reference to 'making agricultural pursuits appealing' to 'modern educated youth'. I do not quote verbatism here, but the contributor's meaning is clear. It is concerned with 'educated' youth.

As far as I know, most young people who flee into the cities throughout in the world come from predominantly agricultural rural areas, where the educational and health facilities are of low standard. Thus, most of the youth who reject agricultural pursuits are not well-educated either to procure employment that would pay them enough to live out of poverty. This accords with the reality; one only needs to take a cursory glance at the million-dweller slums around the big cities in the developing world and the surrounding villages that are generally populated by the elderly and small children.

I think this is the backdrop against which a fruitful discussion of this issue may be undertaken. Do please note that it is in such countries that food shortages, hunger and malnutrition predominates in the world even though their leaders boast of some of their citiesas paragons of 'economic growth'.

Other things being equal, it is reasonable to ask will diployment of materail and technical resources would induce the young people to take up agricultural pursuits even if decent incomes are assured? Would they be then willing to stay in situ and take up the plough or leave slums to do so? I am perfectly aware that no one seems to dare ask this question, let alone answer it even though everybody knows this is true and it is an incontrovertible fact.

Senn in this dismal light, evaluation with respect to the present purpose offers us an indirect insight, viz., it is vital to carry out a sound pre-implementation evaluation of any undertaking to ascertain its probable success. This must be carefully distinguished from its feasibility which is merely mechanical.

Let me hint at a possible recommendation a pre-implementation evaluator might make. Let us ask the question, what apart from poverty drives the youth from food production? This is not political rhetoric, for every big city in industrialised countries has its own slum where slum-dwellers have lived for generations in poverty, and crime is a common feature there.

Even though this cannot be turned into colourful  graphics, impressive figures or into a learned dissertation, I maintain the rural youth are drawn into cities and away from agricultural pursuits fro three reasons, viz., 'the bright lights' of the city as portrayed in fiction, films, tv., videos, etc., belief that one could quickly get rich there, and most of all, denigration by the rest of the society of agricultural persuits as inferior work. One only needs to look at how various European languages  call their farmers in informal speech to see how ingrained such beliefs are.

Whilefully agreeing on the importance of appropriate infra-structure, financing, health care,education and above all on-the-job training,, I believe that it is essentail to bring about a radical change in our social attitude to food production and those who are involved in it. At the same time, it is necessary to portray the life of an average city-dweller as it is, but not as the make-belief city-dweller known to many a rural youth.

Perhaps it might be salutory for the general public to understand that provided that they had air and water, nothing else could have any value for them unless they have enough food, for then they all would be dead. Hence, it is those who are involved in food production who makes everything we value as civilised possible to create and sustain. Hence, engagement in agricultural pursuits should be highly esteemed. All this is obvious, but as one of the wisest men of our times once said, "it is the obvious that is most difficult to understand."

So, any chance of launching a carefully re-evalued undertaking to re-mould the current public perception of agricultural pursuits? Any possibility of inducing those responsible to portray city-life as it is for most city-dwellers? Unless these happen, the prospects of success seem dismal, and yet there are a few bright spots where dedicated people have succeeded in inducing youth not just to engage in food production, but to do so in line with the local food culture because they care. Perhaps they may serve as beacons to the future as the monasteries did during the dark ages.

Best wishes!

Lal Manavado.

Dear Carlos,

Just a few supplementary remarks on the subject; strictly speaking, they may not seem relevant, but I think one has to be a pragmatist in order to use a synthetic approach to attain a goal. In my view, pragmatism is implicit to synthesis.

Therefore, taking every factor that has a bearing on a successful outcome becomes a logical necessity every synthesis should embody. This entails a comprehensive analysis of not only documented data, but also the specific external contextual variables like the needs and capacities of the potential beneficiaries (real people in the target area) and the resources at one's disposal to implement and sustain a given project or plan. Obviously, this has to be undertaken before planning a project based on a holistic synthesis of the relevant facts as revealed by the pre-project analysis. At this point, I think it would be very useful to carry out a pre-implementation evaluation of the planned project so that its strengths and weaknesses could be ascertained before actual implementation.

If this is tenable, then meta-analysis by itself would not  be sufficient to underwrite the success of a project, and it may under certain conditions give one a false sense of completeness and correctness that could be undesirable. Hope this would be of some use.

cheers!

Lal.

I am happy to see this most timely question raised at this time, soon after the previous discussion on the difficulties in ensuring adequate funding for agriculture.

A synthetic approach by definition, involves paying due attention to every influence, positive and negative that bears on achieving a clearly defined objective. In my experience, this seems to be the most difficult thing to achieve even when sufficient material and guidance is clearly available. Let me be explicit.

First of all, there is a common belief that evaluation ought to be on some result with reference to a plan or a 'project specification' if you will. This is akin to checking whether some product meets a manufacturer's specifications and costings. This approach seems to be attractive because of its 'clear-cut' and scientific' nature. But, how useful is this in real life?

When applied, this method would evaluate as successful any 'prestige project' as long as it meets the project specifications and budget. Its inadequacy lies in that it ignores taking into account its utility to its supposed beneficiaries. accruing Such potential benefits depends on the beneficiaries' need for it and their ability to use the project product with sufficient skill. An up-to-date cardiology unit in a remote area of a poor country where health personnel a problem is an example of such a potentially successfully completed but rather a useless project.

A synthetic approach would have ascertained the greatest health needs of the people of such an area, the available man-power, ensured a sustainable funding mechanism, maintenance  capabilities before a project design is undertaken. I firmly believe what is most important is such a pre-project evaluation before one can carry out a meaningful post-project one.

You would have noticed that in a holistic synthetic approach one distinguishes clearly between the 'theoretical' project on paper and the real one that represents people doing their assigned tasks. Without the latter, a project is nothing but paper, but when what is on paper takes into account not only the abilities, but also the frailties of the people involved, it would be justifiable to evaluate a project plan a very useful guide to action.

I know my views are not those of the majority of the experts. But as a synthetist and an analyst, I think my approach is justifiable, for it looks at a project as a set of actions by some people  to enable a target group to achieve some desirable goal in a sustainable way.

Thank you.

Cheers!

Lal Manavado.

Greetings!

I've read the discussion so far with great interest. It would be fair to say that the question boils down to two aspects of the matter, viz.,

   I. Total amount of funds actually at the disposal of the government (or for that matter any organisation).

   II. Willingness and the ability of the 'fund allocators' to do their job honestly and skillfully.

This condition, 'honestly' is not very easy to achieve even in the so-called 'mature democracies'. I will not try to address the issue of how to obtain funds in the first place even though it is very important as Prof. Tinsley has noted in his contribution.

Assuming that the funds are available, we now face the question of 'willingness' of the allocators to provide support in a justifiable way. The ability involved here is an allocators capacity to assign funds in a justifiable way. However, the willingness to do so may not obtain for several reasons, viz.:

   1. Corruption in its many forms.

   2. Incompetence and indifference.

   3. Near fanatical belief in 'development theories'.

This list may not be exhaustive, but we would be naive to ignore its awful effect on our way forward.

Finally, the question of justifiability; a justifiable allocation of public funds (aid or tax income) represents provision of funds to various efforts in proportion to their significance towards enabling people to meet their fundamental needs, which I have fully described elsewhere. These are nutrition, health, education, security,procreation and what I have called our non-material needs. The last is so called because their satisfaction does not involve any material gain, eg. aesthetic enjoyment, playing games, etc. As agriculture is the principal means of meeting our nutritional needs, it should receive due priority. After all, after air and water, food is the most important thing for us. Without it, political or religious creeds, rights, etc., are only of an academic interest.

Best wishes!

Lal.

Greetings!

I think your questions can be given generic answers, whose importance may vary according to the prevailing local conditions. I would like to underline that no 'development effort' will be free of some or all of them irrespective of the political maturity and economic status of a country. After all, even though great deal of lip service is done to deal with 'thinking in silos', precious little is being done to guard against 'working in silos' throughout the world. Let us always keep this distinction in mind if we wish to be realistic.

Now to the questions:

• What are the most common mistakes made in your country?

The greatest mistakes are the failure to ascertain the following before a project/plan is designed and executed:

1. Are the beneficiaries willing and able to derive sufficient real benefits from it with reference to their abilities and expectations and not according to those of some distant planners?

2. Are there sufficient local resources both physical and human, needed to make the best use of what has been planned? For instance, there is no need for a multi-million Dollar bridge to transport a couple of tonnes of vegetables to a nearby city.

3. Does the area/country has sufficient technical skills and financial resources to maintain the end-result on its completion?

4. Are there other better alternatives to the current proposal? For instance, in an area where high unemployment rates obtain, it would be more appropriate to select a labour-intensive alternative than a capital-intensive high tech one. After all, the purpose of every development effort should be to enable as many people as possible to secure a decent livelihood.

This list is not exhaustive, but I think its general drift is quite clear.

• To what extent do you think these mistakes could have been avoided with better use of evaluation, or that evaluation could contribute to the success of policies and major development projects?

This is indeed a tricky question. If we speak of evaluation in a very narrow technical sense as it is often done, it can not be of much use here. However, if we are willing to work out of silos as it were, and opt for a holistic notion of evaluation, it could make a significant contribution. That 'if' is logical.

Let me explain; if we are planning to evaluation not the mere hardware of en effort say a hospital or a bridge, and extend our activities to its intended purpose, i.e., benefiting a group of real live people, then it would be invaluable. This might be called pre-effort evaluation of possiblebenefits made with reference to the 4 points above.

• Have the results of evaluations allowed to amend the failures of public policies and development projects?

If evaluation is only concerned with 'the hardware', it could influence policies and implementation strategies only insofar as they are concerned with the end-result, but never with its benefit yield to real people. For example, better bridge building strategies can result in better bridges, but that does not address the question of their utility. So, it would be reasonable to suggest that only holistic pre-evaluation could be of use in better policy design and implementation strategies.

• Have public policies and projects developed subsequent to the conducted evaluations taken into account previous errors and corrections?

Another contributor has already made many perspicacious remarks on this.

Best wishes!

Lal.

Greetings!

I think an answer to Dr. Houngbo that reflects reality as it is, and not what we would like it to be, would sound very discouraging. I  believe that contrary to the common belief, it would be unwise to trust implicitely data on nutrition even from the affluent and technically advanced countries.

Perhaps, this question may underline with sufficient force the limitations one would have to face in policy formulation in general and that with respect to nutrition in particular. Consider the general methods in use to ascertain the adequacy of nutrition even in a small area.

1. Bio-metrics with reference to age, sex, etc.

1. Does one has a ‘valid’ baseline for comparison?
2. How long should one monitor to arrive at such a baseline?
3. What guarantee does one have that food intake of the participants could or would remain constant during the monitoring period?
4. In the absence of prior group specific values, how does one determine what would be an adequate diet for each test category during the monitoring period?

Well, I could describe some more difficulties, and this is only on establishing a baseline for comparison.

Then of course, we have the usual difficulties regarding transport, monitor competence, inadequate numbers, not to forget people’s willingness to participate.

Some have proposed a work-around or an indirect method, which would have been amusing had it not been put forth as a solution to an important problem. It is to monitor the consumption of various food items in an area. The untenability of this method is too obvious to be elaborated.

I know that I sound most discouraging. But, haven’t we perhaps placed too much trust in numbers because of their ‘seeming’ objectivity? After all, numbers are no more objective than any man-made symbol. Do we think something is automatically the way forward, because it looks like being scientific?

Our notion of science can totally mislead us just as any other belief system could. As soon as we say, X is f and that is an absolute fact with the firmness of any fanatic, out flies the science.

So, can we think of some other approach? It could use statistics for what it is worth as an adjunct, but let us base our policies on agriculture and food distribution on common sense and the applicable norms of nutrition. By applicable, I mean food needs of the local people in line with their food culture as much as possible. Then of course, the powers that be should do all they can to ensure that the facilities are available to local people to produce enough food, a fair distribution and most important making it available at affordable prices. A well-planned and sustained cooperative endeavor free from monopolies seems to be the only way forward, if none is to be left behind starving.

Value of food stems from it being essential to life, not because  it forms ‘value chains’ that enrich a host of intermediaries.

Best wishes!

Lal.

Greetings!

While I fully appreciate the evaluation problems caused by the mismatch between the achievement of 'deliverables' and their actual human benefits, I nevertheless cannot help thinking this is a problem we have created for ourselves. It's just another instance of the difficulties every reductive approach entails.

Consider for a moment what would have happened with that 'road' if the planners asked themselves a few simple questions like:

  1. What's the likely daily volume of wheeled traffic on it?

  2. How many living in the vicinity of that road will be using it? And for what purpose? Etc, etc...

In a very affluent industrialized country in the North, a similar thing happened. It involved a very expensive bridge in a distant area intended to link a moderately inhabited island with the main land. The intention was to enable the people living on the island to travel to work on the main land without having to take the regular ferry. The outcome was interesting to say the least.

The islanders used the bridge to move their goods and chattels and settle down in the main land closer to their places of work, while keeping their old homes as summer houses! It was hoped to finance the bridge at least in part, by the daily toll drivers would have had to pay, but this became less than insignificant.

So, the lesson is obvious, but then, what is obvious seems to be the most difficult to understand.

If before planning begins, one achieves a clear understanding of what would really help the potential beneficiaries of a project and balance it against their actual ability to derive those benefits from it, one would arrive at some realistic set of goals. Then, it would be easy to design a project where the gap between the abstract 'deliverables' and real benefits is minimal, thus making the evaluator's task easier and pertinent.

At the risk of being accused of unseemly levity, a fairly unusual example here would be a project to supply mountain mules to the farmers in High Andes cultivating say quinoa in their fields. This seems to be the most effective way to help them to transport their surplus food to the nearest market. Lack of good roads, high expense in road construction and maintenance, length and cost of training people, and most of all, the time all these take will make the traditional beast of burden a not so comical a choice.

Best wishes!

Dear Mustafa,

Thank you very much for a brilliantly reasoned analysis of the issues Natalia presented. My interest in the field is not as a practioner, but rather as someone who is aware of the importance of continuous monitoring and evaluation as a necessary condition for the success of any project. Your clear distinction between the ‘deliverables’ and their actual usefulness is crucial, and as you point out, often overlooked.

Cheers!

Lal

Greetings!

Naturally, it is important to enhance the skill of evaluators; but apart from some general considerations applicable to every evaluator, one must not overlook the wide variety of projects involved, makes it necessary for an evaluator to develop certain skills specific for each project type. For instance, the skill needed to assess a road is categorically different from what is needed to evaluate the successful completion of a health facility, say, a hospital.

Let us assume that a given project has been successfully completed, and an excellent evaluation has been made. Is it reasonable to assume that is all what's required? Some may be tempted to say, what else? We've done what we've been hired for, and now our job is done well. True, as far as it goes.

If we are content with that, I think we have missed something crucial. That is simply this; when the celebrations are over and project personnel and the evaluators depart, how well will the beneficiaries utilize what has been put in place? Would they be able to undertake necessary maintenance and improvements on their own? Would they be able to make good use of it? Or would it remain a successfully completed monument to the planners' lack of sense of proportion? In other words, a white elephant or a prestige project of little or no utility.

It is this aspect of capacity building I tried to bring to the fore in my first comment on this subject. I believe it is the duty of an evaluator to ascertain the public's ability to use what is planned, and if necessary to induce the planners to incorporate into project plans measures to enhance user's competence to benefit from it.

Cheers!

Lal Manavado

Senior advisor

Norwegian Directorate of Health

Dear Luisa and Lavinia,

The Extent of Capacity Development as an Indicator of Success.

I am happy to see this long neglected aspect of evaluation has received the attention it deserves. Other things being equal, one has too often seen otherwise successful endeavours quietly fizzle out when the outside professionals had left it. The reason is simple: when a project has been completed, the locals in charge simply lacked the know-how and skills needed to run it efficiently, maintain it or a combination of both.

It is impossible for a pragmatist envisage a just ‘one off’ project, i.e., when it is successfully completed, no further human effort is needed to keep it going. Of course, one may argue that running a refugee camp provides a good counter-example, because once all the refugees have been properly assimilated into the host society or repatriated, the project is truly finished. But, in real life, one seldom sees such except in a few rather affluent countries. Besides, vast majority of projects evaluated are concerned with enhancing the daily lives of ordinary citizens of a country.

Therefore, it stands to reason that when planning a project, it is vital to its success to begin with the overall purpose of the effort. It is simply to improve some aspect of daily lives of some target group. At this point, it is so easy to let a planner’s reductive imagination soar above the rosy clouds. We have already seen two examples of that in the previous EVAL-ForwARD forum, viz., a road and a bridge.

I think it is crucial that the evaluators come in at this point to emphasize that unless it can be established beyond any reasonable doubt that the potential beneficiaries of a project are willing and able to derive its benefits, it would be futile to initiate it.

Never under estimate their willingness and ability. Many successfully completed public health projects languish unused, for the culture of the intended beneficiaries does not value good health as highly as it is done by other cultures. Likewise, desire for prestige has driven some to plan advanced telecom networks to provide cellular telephony to rural youth. Here, their ability to use them for ‘developmental purposes’ has been overlooked. Facts are simple; areas involved lack good basic road transport and the target group is hardly literate. So, cell phones will provide a source of entertainment and long-distance gossip. Hardly a benefit especially in view of its cost and the consequences.

After these longish preliminaries, let us assume that the project involved is indeed appropriate i.e., it will really benefit the target group because the members of it are willing and able to use it. Capacity building cannot influence this willingness for it belongs to another category, but it is vital to one crucial aspect of this ability viz., the ability to run the project well and to keep it in running order while undertaking the improvements it needs in the long term.

I am not certain to what extent the capacity of the public to benefit may be enhanced unless it is integrated as an essential component of a project. This is especially true in cases where the overall objective has been to improve public nutrition. Other things being equal, a project to increase food production would not lead to better nutrition unless the target group has an adequate dietary competence, i.e., knowing what to eat, how to prepare it, etc.

So, it would be reasonable to affirm capacity building is an indicator of success in evaluation, and ought to be incorporated into a project at its inception. However the question, whose capacity and to do what, needs careful consideration. At the theoretical level, one can distinguish between two sub-groups in a target population, the overall beneficiaries and those who are expected to continue the operation of a project on its completion. I hope this might be of some use.

Cheers!

Lal Manavado

Norwegian Directorate of Health, Norway