Silva [user:field_middlename] Ferretti

Hello

I practice humility by asking myself a different question:

If people who have been working on an issue for a long time, with a much better understanding of the context did not find a good solution... how could I, an external evaluator, do so?

As an evaluator I cannot certainly find solutions but i can - with a facilitative and not an expert approach:

* help to find "missing pieces" of the puzzle, by bringing, in the same place, the views and ideas of different actors.

* help articulating and systematizing reality better, so that people can have a better map on which to find solutions

* capture ideas, lessons that too often are implicit and that - if shared - can help changing the way of working

* share some ideas about things that I had seen working elsewhere (but, watchout, I would always do this in the evidence gathering phase, as a way to get feedback on these "conversation starters". and people often find quickly a lot of things to be checked and improved)

* create spaces, within the process, for people to be exposed and react to evidence, as it is shared

* identify what seem to be the priority concerns to address - linking them to challenges, opportunities, possibilities surfaced.

This is not research. And these are not solutions.

There is a whole world of things amongst "problems" and "solutions"... it includes learnings, possibilities, systematized evidence.

And I see people really interesting and willing to engage with these... Much more than when I used to preach some simple solutions to them. :-)

Also, an evaluation does not always highlight "problems". There are often so many solutions that are just left hidden.

And evaluations have also a role in finding these and to help valuing the work done, and the many challenges solved, which should never just be given for granted.

Clarity... of course, absolutely! Elevator pitch... yes and no.

An elevator pitch is very useful as an entry point.

But there should then be a recognition that the purpose of a good evaluation is to unveil the complexity of reality (without being complicated).

It can give new elements and ideas, but not the solution.

The elevator pitch is the entry point, it highlights main areas to be addressed, and it can certainly outline some pressure points.

But I am not so sure that we can always offer a crisp idea of possible solutions.

As they say, "for each problem there is always a simple solution. And it is wrong".

Solutions are to be found, as Bob so well said - beyond the evaluation.

(or within it only if it is a participatory one, where key local actors are truly engaging in formulating findings, and truly own the process)

So the tools and messages we need are not just the elevator pitches, but these helping to convey and navigate complexity in simpler, actionable ways.

Being aware that it is not for the evaluator to hammer messages, but for the project stakeholders to own them.

Great take-away...

One point to stress.

Going beyond the report does not mean "make a visual report".

A visual report is nicer, but still a report.

"Going beyond the report" means to consider the evaluation as a process that does not end just one product - being visual or not.

Communication of findings, sharing of ideas need to happen throughout, in many forms.

A good evaluation does not need to be a "report".

I advocate for strategies, options for sharing ideas and findings with different audiences, throughout.

Which might NOT include a report. Report writing is extremely time consuming, and takes away a big % of evaluation time.

Is it the best investment? Is it needed? We are so used to think that an evaluation is a report that we do not question it.

Also... beside real time evaluations there is "real-time information sharing".

This is something too little explored. Yet it can create big changes in the way evaluation happens.

It is about sharing preliminary ideas, evidence, so that people involved in the evaluation can contribute to shape findings.

Again: we are so used to think that we share the "end products" that the possibilities of real-time information sharing are not really understood...

Thanks again for the great summary, it really help to step up discussion and to generate new ideas

(and, you know what? It is a good example of "real-time information sharing" of new ideas! :-)

Oh, well done!

Great to see that there is some recognition of the value of pictures and visuals.
The materials you shared are really helpful and inspirational, thanks.

Now... as someone who thinks visually and in pictures I consistently tried to sum up findings in a more visual way.
Graphics, drawings, multimeria are seen as "nice" and cool. Everyone likes them and feel they are useful.

But, guess what? I then have to produce a normal report, because this is what donors want.
So, visuals are to be done as an aside. Of course, for free.

Time for reporting is already usually not enough in a consultancy, so if you want to prove that visuals or other media are better, you basically need to work for free.
Because, at the end of the day, you will still have to write the proper report.

The bottom line?

Until evaluations will be mainly perceived as bureaucratic requirements and reports...we will miss out on fantastic possibilities to learn better.
And also, to involve people who might have fantastic learning, analytical, communication skills, but who are not report writers.
It is so unfortunate that we assume that "report writing" alone is the best way to capture and convey evidence and insights...

Is there any chance that we could stop thinking that an evaluation is... a report?

So many possibilities would be unlocked.

What strikes me is that we all discuss ToCs as if they were "a thing"....

Talking about a "logframe" is easy: there is standard format to it. 

It might be slightly adapted, but it is quite clear what it is, how it looks like, how it works.

The same is not true for ToCs.  What a ToC is can be vastly different.

I feel we might all use the same word, but having something vastly different in mind...

Best

Silva

It depends on what is a Theory of Change, and how it has been generated/shared.

If it remains the same as a big logframe, hidden in some proposals... it does not add much value.

If it is co-generated and owned... possibly EMERGING from the process of change, then it is an added value.

As an evaluator, I see that staff on the ground welcome discussions at theory of change level when they help to systematize experience.

But they might be clueless and confused by TOCs as proposal annexes.

So, if the Theory of Change is just bureaucracy it is actually a complication.

If it is a process of systematizing experience, owned by these involved in making change, it is super useful.

Unfortunately, the latter are very rare.