Zahid [user:field_middlename] Shabbir

My contributions

• Posted on 29 Jan 2020

  How useful are theories of change in development programmes and projects?

  Discussion

  • 

    Zahid Shabbir

    Joint Director Program Management Division State Bank of Pakistan

    Pakistan

    Hi Carlos,
    
    Please find below my feedback on use of ToCs.
    
    
    Essentially, ToCs are meant to capture the changes that an intervention undergoes since its planning till the time it concludes and thereafter in the form of impact. The ToC ladder is usually characterized by inputs, activities, outputs, outcome and impact. Typically, project monitoring teams engaged in day to day management of projects can only foresee the little use of ToCs in the form of immediate output (short term), whereas the difference that project is expected to make will be assessed by an evaluation team that gauges the outcome (medium term) and impact (long-term) of the interventions.
    
    We have used this tool in designing an M&E system of our national strategy on financial inclusion since 2015 in collaboration with World Bank Group. I can say with surety, its highly effective in overall development evaluation paradigm given that proper groundwork, planning and expert choice is made before choosing relevant indicators (true and fair representation) across the ToC ladder.
    
    In brevity, few of the takeaways that I can summarize are as follows:
    
    * This is a tool mostly used by expert M&E professionals and probably not that much by project managers or staff involved with trivial monitoring assignments.

    * It is necessary to understand the gist of ToCs which tacitly works in coherence with the principles of Result- Based Monitoring & Evaluation (RBM) as compared to conventional monitoring practices.

    * Last but not the least, if you are looking to develop an M&E system that works in coherence with the results of multitude of interventions (national/global level), there is no best tool but ToC to map results across every level that can be fitted under broader and aligned evidence capturing framework.
    
    That's why, ToC has gained quite a significance for evidence-based policymaking and so much so in the design of new interventions based on prior evidence. For review of the practitioner's work, I would suggest Linda G. Morra Imas and Ray C. Rist (2009) guide, "The Road to Results". Further, I already shared my publication, "Designing Robust M&E Systems..". I hope, here you can find the examples of ToC being mapped from RBM system.
    
    Will be happy to assist further in case of any queries.
    
    Regards,
    
    Zahid Shabbir
    Joint Director
    State Bank of Pakistan

    • read on a separate page
• Posted on 15 Nov 2018

  How to Measure the Impact of Monitoring and Evaluation Work

  Discussion

  • 

    Zahid Shabbir

    Joint Director Program Management Division State Bank of Pakistan

    Pakistan

    Dear Abubakar,

    Thank you very much for your response and queries.

    I will go by Dr. Emile's viewpoint, generally it's not actually the monitoring & evaluation system itself which transforms into impact, rather the way the interventions/actions/projects are designed becomes critical to assess the impact of certain interventions.

    If I correctly understand your queries, the impact M&E can drive is simply the question of having M&E system in place or vice versa. For this, as far as my understanding and experience is concerned, you need to perform some sort of counterfactual analysis (probably using systematic reviews/mixed methods/triangulation) to determine whether having M&E systems in place transformed into some sort of policy change/ improvement or not, no matter what sector/ indicators you are taking into the consideration. The scale and breadth of such analysis would be dependent upon various factors (including but not limited to resources, expertise, priority areas, number of interventions to account for).

    Happy to discuss further in case of any queries.

    Regards,

    Zahid Shabbir

    Joint Director | Program Management Division | AC & MFD |State Bank of Pakistan |

    • read on a separate page

  • 

    Zahid Shabbir

    Joint Director Program Management Division State Bank of Pakistan

    Pakistan

    Dear Abubakar,

    It was nice to see this discussion on the subject. I would like to share our experience in Pakistan as we developed an M&E system for National Financial Inclusion Strategy of Pakistan (NFIS) with the assistance of World Bank Group. I would limit my discussion relevant to your queries.

    First of all, we need to put quite a discern between monitoring and evaluation systems, especially the design of monitoring mechanism is very important. Out here, we developed a three tiered architecture for M&E; (i) monitoring of headline indicators (somewhat similar to broader outcome over the period of time), (ii) Activity/Action level monitoring transliterating into programs, policy or regulatory interventions, (iii) bottom tier, a project level monitoring which could be strengthened using blended tools like result based monitoring (RBM), project performance management using Earned Value Management technique (depending upon nature of project through indicators like CPI – Cost Performance Index, SPI – Schedule Performance Index, etc). Similarly, various concepts like value for money (optimizing output through minimal input) could be used to assess possible impact during the design phase of project/intervention.

    With regards to evaluation specifically, I think least kind would be participatory evaluation where organization itself could be involved to evaluate the impact of interventions it has undertaken in specific sectors. However, choice remains with the organizations how to go by this endeavor, especially if its novice in the area. In such cases, donors/ multilaterals could be engaged to seek professional assistance (as is the case with many developing economies). I am sure, you must be aware of some sort of project performance reports to cater the information about viability of projects/programs like PPARs – Performance Planning and Revies, ICRs – Investor Confidence Rating, MfDRs – Managing for Development Resources, PfR-Project feasibility Reports, etc which are some of the common reporting mechanisms to assess the development effectiveness used by many multilaterals / donors.

    As far as your queries regarding the monitoring of agricultural productivity/ service delivery are concerned, the technological developments have gone by far than required. For instance, you may begin a pilot with Unmanned Aerial Vehicle (UAV) technology in certain area to experiment the productivity results of specific corp/use of fertilizer etc. Similarly, Precision Agriculture is another important technological development which has yielded very good productivity results in case of India. The good thing about these systems is automated data storage and tracking capacities which can greatly assist in monitoring productivity and impact you are looking for can be deduced over the period of time and can witnessed though the ladder of Theory of Change (ToC).

    I hope you would find our experience helpful to get fruitful results.

    Regards,

    Zahid Shabbir
    Joint Director | Program Management Division | AC & MFD |State Bank of Pakistan |

    • read on a separate page