RE: How to Measure the Impact of Monitoring and Evaluation Work | Eval Forward

Dear Abubakar,

It was nice to see this discussion on the subject. I would like to share our experience in Pakistan as we developed an M&E system for National Financial Inclusion Strategy of Pakistan (NFIS) with the assistance of World Bank Group. I would limit my discussion relevant to your queries.

First of all, we need to put quite a discern between monitoring and evaluation systems, especially the design of monitoring mechanism is very important. Out here, we developed a three tiered architecture for M&E; (i) monitoring of headline indicators (somewhat similar to broader outcome over the period of time), (ii) Activity/Action level monitoring transliterating into programs, policy or regulatory interventions, (iii) bottom tier, a project level monitoring which could be strengthened using blended tools like result based monitoring (RBM), project performance management using Earned Value Management technique (depending upon nature of project through indicators like CPI – Cost Performance Index, SPI – Schedule Performance Index, etc). Similarly, various concepts like value for money (optimizing output through minimal input) could be used to assess possible impact during the design phase of project/intervention.

With regards to evaluation specifically, I think least kind would be participatory evaluation where organization itself could be involved to evaluate the impact of interventions it has undertaken in specific sectors. However, choice remains with the organizations how to go by this endeavor, especially if its novice in the area. In such cases, donors/ multilaterals could be engaged to seek professional assistance (as is the case with many developing economies). I am sure, you must be aware of some sort of project performance reports to cater the information about viability of projects/programs like PPARs – Performance Planning and Revies, ICRs – Investor Confidence Rating, MfDRs – Managing for Development Resources, PfR-Project feasibility Reports, etc which are some of the common reporting mechanisms to assess the development effectiveness used by many multilaterals / donors.

As far as your queries regarding the monitoring of agricultural productivity/ service delivery are concerned, the technological developments have gone by far than required. For instance, you may begin a pilot with Unmanned Aerial Vehicle (UAV) technology in certain area to experiment the productivity results of specific corp/use of fertilizer etc. Similarly, Precision Agriculture is another important technological development which has yielded very good productivity results in case of India. The good thing about these systems is automated data storage and tracking capacities which can greatly assist in monitoring productivity and impact you are looking for can be deduced over the period of time and can witnessed though the ladder of Theory of Change (ToC).

I hope you would find our experience helpful to get fruitful results.

Regards,

Zahid Shabbir
Joint Director | Program Management Division | AC & MFD |State Bank of Pakistan |