Abubakar Muhammad Moki

Good morning

I have not yet observed any shift from visiting international teams to local specialists. International teams most often have local partners so I suspect the shift may not occur in all countries.

Thanks

Abubakar Muhammad Moki

Richard

I am of the  opinion that, if the identified problem is clear and the evidence supports that it should be addressed, then the expected results are worth and interventions are properly costed with a logical flow, Government would still prioritize and allocate the required resources.  The problem is the Maputo concept of a share/fraction of GDP without knowing in  a logical flow how it is to be used. 

Abubakar Muhammad Moki

Uganda

I once again wish to state my opinion as follows:

For ones efforts to be seen, appreciated and funds allocated in the required areas, the following hold:

1.  The problem to be addressed and the expected results after addressing the problem should be clear and  backed by evidence

2.The problem to be addressed and the expected results after addressing the problem should have clear intervetions that have a logical flow and supported with evidence

3. The clear interventions should be practical for implementation in addressing the identified problem and realisation of the expeted results

4. The clear interventions should be able to be costed in a realistic manner to support allocation of the funds towards addressing the identifed problem and realisation of the expected results

5. With that logical  flow, funds should be provided for the interventions, to address the identified problem and realisation of the expected results.

6. However, what I observe in practice is that, funds in most cases are advocated for as a fraction/share of GDP without the logical flow of the problem to be addressed, expected results, clear costed intervetions to justify the fraction/share of GDP being paraded.

7. I therefore still hold the opinion that, demanding  for a fraction/share of GDP allocation should be discarded in preference of the logical flow as stated above.

Grateful

Abubakar Muhamamd Moki

Uganda.

1. Funding should be determined based on issues to be addressed, interventions identified and costed to arrive at allocations at any given allocation period. 

2. Blanket allocations should be discouraged since the issues to be addressed may worsen or even cease to exist as time goes on.

3. I therefore advocate for change of funding modalities towards assessment of issues, then interventions to address the issues, then cost to arrive at the amounts

I can provide details on how to go about the approach if agreeable

Abubakar Muhammad Moki

Uganda

To improve the quality of development projects, do the following:

1. look into the loopholes that promote corruption

2. change the mind-set of workers towards serving the citizens and not what is in for them

3. ensure sanctions are enforced when due to deter other retarding measures

4. recruit project staff based on meritocracy and not based on know-who and political patronage

5. easy information sharing

6. regular provision of feedback to the stakeholders including the beneficiaries on the performance of the project and seeking input for correction

Abubakar Muhammad Moki

Uganda

Dear Patricia R. Sfeir,

My organisation has a 'Framework for Analyzing Public Policies-A Practical Guide' that is used in evaluations generally and can as well be applied to social protection programmes and policies.  Key highlights in the framework are as follows:

1.            Each evaluation covers the duration agreed with the stakeholders.

2.            Evaluation is on three thematic areas, namely effects, implementation and cross cutting issues.

3.            Effects cover evaluation of impact, effectiveness, relevance, unintended effects and equity (including Gender and Human Rights).

4.            Implementation evaluation covers specific characteristics of social policies, processes, cost and efficiency, feasibility and adaptability. 

5.            Cross cutting issues cover sustainability/durability, partnerships and synergies.

Please let me know if this is helpful.

Grateful

Abubakar Muhammad Moki

Uganda

Dear Colleagues,

Thank you so much for the contributions that you made on the topic 'how to measure the impact of M&E work' that I posted for discussion in the forum.

I have made a compilation of the key issues that I received out of the discussions, attached to this email.

I will appreciate further guidance on how to take the subject forward.

The following are some of the key highlights:

I.             Nothing is impossible and impact evaluation of evaluations or a systematic review of M&E can be done.

II.            The impact of M&E and indicators for this “Impact" is much related to utility/ use of the outputs of M&E.

III.           Measuring the impact of M&E outcomes can be in terms of the outcome influencing decision-making, influencing project or programme or policy change/redesign, influencing more resource allocation, informing improvements in production, productivity, food security and service delivery in general, clarifying problems or challenges or issues.

IV.          It may not always be possible to measure the contribution at the impact level, but going down to outcome and output level would be easier. To give example of indicators: the number of downloads of an evaluation report or the number of reaches/ views, % of M&E reports shared on websites of related organisations.

V.            Perform some sort of counterfactual analysis

VI.          (probably using systematic reviews/mixed methods/triangulation) to determine whether having M&E systems in place transformed into some sort of policy change/ improvement or not, no matter what sector/ indicators you are taking into the consideration. The scale and breadth of such analysis would be dependent upon various factors (including but not limited to resources, expertise, priority areas, and number of interventions to account for).

VII.         Monitoring data collected at a project level can easily be aggregated at the sectoral level, getting the sectoral plan to feed back into the national strategic plan.

VIII.        The extent to which M&E actions, recommendations and decisions are implemented can be used to approximate the impact, so that if zero of those are implemented then we approximate the impact of M&E to be zero, if 50% are implemented then we approximate the impact to 50 %  etc.

IX.           The extent to which the purpose of M&E is realized can also be used, for example, better understanding of the gaps, challenges, emerging needs, changes in situations, etc.  The extent of the achievement can be rated and levels of achievement determined that can be approximated to the impacts created in that context.

X.            Adopting outcome harvesting (mixed with another approach like contribution tracing) one would be able to trace the impact of M&E system. The choice of outcome harvesting is because it does not measure progress towards a predetermined objective, but rather collects evidence of what has changed and then work backward to trace a plausible relationship between the change and an intervention contribution to this change. If government proposes an agricultural bank, this can be traced back to recommendations from some M&E effort or if agricultural insurance policy, this policy change can be traced to (i) The project Intervention;  (ii) Project M&E efforts (iii) Environment and what it means for agriculture etc.

Grateful

Abubakar Muhammad Moki

Uganda

Dear Samuel

Thanks for sharing this useful information from the ICED newsletter.

There seems to be a lot of evidence on how to establish a robust M&E system for better policy and practice.

There is need for evidence in a similar manner, on how best to determine whether the robust M&E system that has been established actually betters policy and practice.

That is how the discussion on how to measure the impact of M&E system outcomes on policy and practice comes in. The idea is that we should be able to establish the changes that M&E outcomes have contributed to. This contribution to changes of M&E could then be approximated to be its impact.

There is also need for clarity on what a robust M&E system is. The robustness of M&E system could be out of what the M&E outcomes do such as degree at which it can influence or inform better policy and practice.

Grateful

Abubakar Muhammad Moki

Uganda

Dear Isha Miranda

Thanks for the good contribution.

However, my interest is to know how to measure the impact or effectiveness of M&E outcomes. In other words, how to measure the influence that M&E evidence or outcomes have generated. How do we do that?

I believe, we should also be able to assess whether M&E outcomes are meeting their intended purposes such as to what extend do M&E outcomes influence: (i) decision and policy making, (ii) redirecting policy or project or program implementation, (iii) influencing termination of policy or project or program implementation that is not meeting its intended outcomes, (iv) influencing planning, budgeting etc.

So that M&E outcomes can be seen to have impact or not.

Grateful

Abubakar Muhammad Moki

Uganda

Dear all,

Thanks for your contributions and please see my responses below:

Dear Dr Émile Houngbo

My issue is how do I know that M&E done has added value to an organisation.

How do we measure the output and outcomes of M&E that has been carried out or done?  And not what M&E measures. M&E should not be an activity whose performance cannot be assessed.  What can be  assessed and who has the yardsticks?

Dear Zahid Shabbir,

Thanks so much for all your useful contributions. However, I have a view that the impact of M&E can be in terms of the following:

1. decisions it has influenced

2. problems or challenges or issues it has clarified

3. interventions it has redirected

4. etc.

We can recast the above and add on the list

So that an M&E system that does the above well, we say it has high impact and one that does none of the above we say has no impact and hence no value for money or value addition or contribution for its work.

Dear Mustapha Malki,

My interest is how to assess the performance of the M&E system in terms of the changes it has influenced. If for example, the purpose of M&E is for evidence informed decision making.

If an M&E output influences decision making, then that should be its impact.  If it does not influence decision-making, then it has zero impact.

Dear Dr. Tarek Sheta,

To measure the impact of M&E work we should look at:

(i)  The extent to which M&E actions, recommendations and decisions are implemented. So that if zero of those are implemented then we approximate the impact of M&E to be zero. If 50% are implemented then we approximate the impact to 50 %  etc.

(ii) The extent to which the purpose of M&E is realized for example, better understanding of the gaps, challenges, emerging needs, changes in situations, etc.  The extent of the achievement can be rated and levels of achievement determined that can be approximated to the impacts created in that context.

Let’s then refine those two above and add on the list

Dear Reagan Ronald Ojok,  

My idea is how to measure the impact of the M&E outcome in terms of the outcome influencing decision-making, influencing project change, influence more resource allocation etc.

This is because many M&E are being done at various levels and no change seems to be seen out of the M&E outcomes because of weak influence of M&E outcomes. 

So how do we measure the M&E outcomes so that we know which ones have made some influence and which ones have not made some influence as an example?

Dear Zahid Shabbir,

Thank you very much for sharing very useful information.

However, my interest is to know how monitoring being carried out informs better achievements or delivery of the broader outcomes over a period of time.  This is because when, for example, carrying monitoring of headline indicators (somewhat similar to broader outcome over the period of time), the results of the monitoring is expected to inform better realization of the broader outcomes over the period of time.  

So how do we measure the contribution of the monitoring being carried out, towards helping in informing realization of better broader outcomes over the period of time? This is because the monitoring of the broader outcomes is not done for the sake of it but with the aim to inform better realization of the broader outcomes over the period of time.

There should be therefore a way of determining whether monitoring being carried out is contributing towards better realization of the broader outcomes being monitored over time of not.

The same applies to other levels of monitoring.

That is the focus of my assumption.

Kind regards

Abubakar Muhammad Moki

Uganda

Dear Reagan Ronald Ojok,

Thank you very much for your contribution.

It is true M&E is being done at different levels being it at national, district, sub-country etc. 

Ministers, Parliamentarians, District Resident District Commissioners, their Deputies and Assistants technical officers at the Ministries, district Councilors and district technical officials, sub-county Councilors and technical officials etc all are involved in some kind of M&E work.

Resources are being utilized for M&E work at the different levels.  How do we then measure value for money of the M&E at the different levels?  We should for example be to tell that M&E work  being done at the lower levels is creating better impact than M&E being done at the national level because of lower M&E proximity to the service beneficiaries.

So what yardsticks can we use to determine which of those different players has better M&E impact and is able to influence more resources to it due to its greater impact?

I believe if we put our ideas together, we can come up with a framework of indicators that can be used to identify who does M&E better due to its better influence for better impact.  

This is yet to come.

Grateful

Abubakar Muhammad Moki

Dear Naser Qadous

Thank you for the good feedback.

As we all know, M&E is not carried out for the sake of it.  One of the reasons behind carrying out M&E is to provide evidence for informing improvements in agricultural production, productivity, food security and service delivery in general.

The issue now is, how do we determine that M&E carried out has contributed to informing improvements in agricultural production, productivity, food security and service delivery in general? 

How do we determine the contribution of the M&E carried out, so that there is value addition seen out of the investment in carrying out the M&E?

This will help in documenting value addition of each M&E being carried out.

Grateful

Abubakar Muhammad Moki

Uganda