Zahid [user:field_middlename] Shabbir

Zahid Shabbir

Joint Director Program Management Division
State Bank of Pakistan
Pakistan
  • Manage & Coordinate projects under National Financial Inclusion Strategy (NFIS) of Pakistan. NFIS is national level program to enhance access of formal financial services in the country.
  • In coordination with multilateral donor agency (WBG), developed monitoring and evaluation framework for NFIS while defining different tiers of M&E framework (RBM, Internal Monitoring etc) along with induction of EVM which is an advanced project management tool.
  • Ensure the use of adequate evaluation principles, approaches and tools (RCTs, Mixed Methods, Triangulation etc.) for conducting evaluation of projects along with the review of third party evaluation reports for SBP/donor sponsored projects.
  • Compilation of conceptual paper/policy guidelines to cover up the priority areas (students, women, especially strategic goals regarding NFIS).
  • Monitor, Financial Inclusion related program portfolio & liaise with donors on range of interventions under different components along with conducting, reviewing independent assessment of projects (PPARs, ICRs etc).

My contributions

  • How useful are theories of change in development programmes and projects?

    Discussion

    • Zahid Shabbir

      Joint Director Program Management Division State Bank of Pakistan
      Pakistan

      Salut Carlos,

      Veuillez trouver ci-dessous mes commentaires sur l'utilisation des TdC.

      Essentiellement, les TdC visent à saisir les changements qu'une intervention subit depuis sa planification jusqu'au moment où elle se termine et par la suite sous forme d'impact. L'échelle de la TdC est généralement caractérisée par des intrants, des activités, des extrants, des résultats et un impact. En règle générale, les équipes de suivi de projet engagées dans la gestion quotidienne des projets ne peuvent prévoir que la faible utilisation des TdC sous la forme de résultats immédiats (à court terme), tandis que la différence que le projet devrait faire sera évaluée par une équipe d'évaluation qui jaugera le résultat (à moyen terme) et l'impact (à long terme) des interventions.

      Nous utilisons cet outil dans la conception d'un système de S&E de notre stratégie nationale sur l'inclusion financière depuis 2015 en collaboration avec la Banque Mondiale. Je peux dire avec certitude, qu’elle est efficace dans le paradigme global de l'évaluation du développement étant donné que les bases, la planification et le choix d'experts appropriés sont faits avant de choisir des indicateurs pertinents (représentation vraie et juste) à travers l'échelle de la TdC.

      En bref, quelques-uns des points à retenir que je peux résumer sont les suivants:

      * Il s'agit d'un outil principalement utilisé par les professionnels experts du S&E et probablement pas tant que ça par les chefs de projet ou le personnel impliqué dans des missions de suivi seulement.

      * Il est nécessaire de comprendre l'essentiel des TdC, qui fonctionnent tacitement en cohérence avec les principes de suivi et d'évaluation axés sur les résultats (GAR) par rapport aux pratiques de suivi conventionnelles.

      * Dernier point mais non le moindre, si vous cherchez à développer un système de S&E qui fonctionne en cohérence avec les résultats d'une multitude d'interventions (au niveau national / mondial), il n'y a pas de meilleur outil que la ToC pour cartographier les résultats à tous les niveaux qui peuvent être dans un cadre de capture de preuves plus large et aligné.

      C'est pourquoi, la TdC a acquis une importance considérable pour l'élaboration de politiques fondées sur des données probantes et à tel point dans la conception de nouvelles interventions basées sur des preuves antérieures.

       Pour un examen du travail du praticien, je suggérerais le guide de Linda G. Morra Imas et Ray C. Rist (2009), "The Road to Results". De plus, j'ai déjà partagé avec EvalForward ma publication, "Designing Robust M&E Systems ..". J'espère que vous trouverez ici des exemples de ToC mappés à partir du système RBM.

      Je sera heureux de vous aider davantage en cas de questions.

      Cordialement,


      Zahid Shabbir
      Joint Director
      State Bank of Pakistan

  • How to Measure the Impact of Monitoring and Evaluation Work

    Discussion

    • Zahid Shabbir

      Joint Director Program Management Division State Bank of Pakistan
      Pakistan

      Dear Abubakar,

      Thank you very much for your response and queries.

      I will go by Dr. Emile's viewpoint, generally it's not actually the monitoring & evaluation system itself which transforms into impact, rather the way the interventions/actions/projects are designed becomes critical to assess the impact of certain interventions.

      If I correctly understand your queries, the impact M&E can drive is simply the question of having M&E system in place or vice versa. For this, as far as my understanding and experience is concerned, you need to perform some sort of counterfactual analysis (probably using systematic reviews/mixed methods/triangulation) to determine whether having M&E systems in place transformed into some sort of policy change/ improvement or not, no matter what sector/ indicators you are taking into the consideration. The scale and breadth of such analysis would be dependent upon various factors (including but not limited to resources, expertise, priority areas, number of interventions to account for).

      Happy to discuss further in case of any queries.

      Regards,

      Zahid Shabbir

      Joint Director | Program Management Division | AC & MFD |State Bank of Pakistan |

    • Zahid Shabbir

      Joint Director Program Management Division State Bank of Pakistan
      Pakistan

      Dear Abubakar,

      It was nice to see this discussion on the subject. I would like to share our experience in Pakistan as we developed an M&E system for National Financial Inclusion Strategy of Pakistan (NFIS) with the assistance of World Bank Group. I would limit my discussion relevant to your queries.

      First of all, we need to put quite a discern between monitoring and evaluation systems, especially the design of monitoring mechanism is very important. Out here, we developed a three tiered architecture for M&E; (i) monitoring of headline indicators (somewhat similar to broader outcome over the period of time), (ii) Activity/Action level monitoring transliterating into programs, policy or regulatory interventions, (iii) bottom tier, a project level monitoring which could be strengthened using blended tools like result based monitoring (RBM), project performance management using Earned Value Management technique (depending upon nature of project through indicators like CPI – Cost Performance Index, SPI – Schedule Performance Index, etc). Similarly, various concepts like value for money (optimizing output through minimal input) could be used to assess possible impact during the design phase of project/intervention.

      With regards to evaluation specifically, I think least kind would be participatory evaluation where organization itself could be involved to evaluate the impact of interventions it has undertaken in specific sectors. However, choice remains with the organizations how to go by this endeavor, especially if its novice in the area. In such cases, donors/ multilaterals could be engaged to seek professional assistance (as is the case with many developing economies). I am sure, you must be aware of some sort of project performance reports to cater the information about viability of projects/programs like PPARs – Performance Planning and Revies, ICRs – Investor Confidence Rating, MfDRs – Managing for Development Resources, PfR-Project feasibility Reports, etc which are some of the common reporting mechanisms to assess the development effectiveness used by many multilaterals / donors.

      As far as your queries regarding the monitoring of agricultural productivity/ service delivery are concerned, the technological developments have gone by far than required. For instance, you may begin a pilot with Unmanned Aerial Vehicle (UAV) technology in certain area to experiment the productivity results of specific corp/use of fertilizer etc. Similarly, Precision Agriculture is another important technological development which has yielded very good productivity results in case of India. The good thing about these systems is automated data storage and tracking capacities which can greatly assist in monitoring productivity and impact you are looking for can be deduced over the period of time and can witnessed though the ladder of Theory of Change (ToC).

      I hope you would find our experience helpful to get fruitful results.

      Regards,

      Zahid Shabbir
      Joint Director | Program Management Division | AC & MFD |State Bank of Pakistan |