Zahid [user:field_middlename] Shabbir

Zahid Shabbir

Joint Director Program Management Division
State Bank of Pakistan
Pakistán
  • Manage & Coordinate projects under National Financial Inclusion Strategy (NFIS) of Pakistan. NFIS is national level program to enhance access of formal financial services in the country.
  • In coordination with multilateral donor agency (WBG), developed monitoring and evaluation framework for NFIS while defining different tiers of M&E framework (RBM, Internal Monitoring etc) along with induction of EVM which is an advanced project management tool.
  • Ensure the use of adequate evaluation principles, approaches and tools (RCTs, Mixed Methods, Triangulation etc.) for conducting evaluation of projects along with the review of third party evaluation reports for SBP/donor sponsored projects.
  • Compilation of conceptual paper/policy guidelines to cover up the priority areas (students, women, especially strategic goals regarding NFIS).
  • Monitor, Financial Inclusion related program portfolio & liaise with donors on range of interventions under different components along with conducting, reviewing independent assessment of projects (PPARs, ICRs etc).

My contributions

    • Hola Carlos,

      A continuación encontrará mis comentarios sobre el uso de la TdC.

      Esencialmente, las TdC están destinadas a capturar los cambios que sufre una intervención desde su planificación hasta el momento en que concluye y, posteriormente, en forma de impacto. La escala de la TdC generalmente se caracteriza por insumos, actividades, productos, resultados e impacto. Por lo general, los equipos de monitoreo de proyectos que participan en la gestión diaria de los proyectos solo pueden prever el poco uso de TdC en forma de resultados inmediatos (a corto plazo), mientras que la diferencia que se espera que haga el proyecto será evaluada por un equipo de evaluación que evalúa el resultado (mediano plazo) y el impacto (largo plazo) de las intervenciones.

      Hemos utilizado esta herramienta para diseñar un sistema de M&E de nuestra estrategia nacional sobre inclusión financiera desde 2015 en colaboración con el Banco Mundial. Puedo decir con certeza que es muy eficaz en el paradigma general de evaluación del desarrollo dado que se realiza un trabajo preliminar, planificación y elección de expertos adecuados antes de elegir indicadores relevantes (representación verdadera y justa) en toda la escala de la TdC.

      En resumen, algunas de las conclusiones que puedo resumir son las siguientes:

      * Esta es una herramienta utilizada principalmente por profesionales expertos en S&E y probablemente no tanto por los gerentes de proyectos o el personal que solo participen en las tareas de monitoreo.

      * Es necesario comprender la esencia de las TdC, que funcionan tácitamente en coherencia con los principios de Monitoreo y Evaluación Basados ​​en Resultados (RBM) en comparación con las prácticas de monitoreo convencionales.

      * Por último, pero no menos importante, si está buscando desarrollar un sistema de S&E que funcione en coherencia con los resultados de multitud de intervenciones (nivel nacional / global), no hay mejor herramienta que TdC para mapear los resultados en todos los niveles que puedan ser ajustado bajo un marco de captura de evidencia más amplio y alineado.

      Es por eso que la TdC ha ganado una gran importancia para la formulación de políticas basadas en evidencia y tanto en el diseño de nuevas intervenciones basadas en evidencia previa.

       Para una revisión del trabajo del profesional, sugeriría la guía de Linda G. Morra Imas y Ray C. Rist (2009), "El camino hacia los resultados". Además, ya compartí con EvalForward mi publicación, "Designing Robust M&E Systems ..". Espero que aquí pueda encontrar los ejemplos de TdC que se están mapeando desde el sistema RBM.

      Estaré encantado de ayudarlo en caso de cualquier consulta.

      Saludos,

      Zahid Shabbir
      Joint Director
      State Bank of Pakistan

    • Dear Abubakar,

      Thank you very much for your response and queries.

      I will go by Dr. Emile's viewpoint, generally it's not actually the monitoring & evaluation system itself which transforms into impact, rather the way the interventions/actions/projects are designed becomes critical to assess the impact of certain interventions.

      If I correctly understand your queries, the impact M&E can drive is simply the question of having M&E system in place or vice versa. For this, as far as my understanding and experience is concerned, you need to perform some sort of counterfactual analysis (probably using systematic reviews/mixed methods/triangulation) to determine whether having M&E systems in place transformed into some sort of policy change/ improvement or not, no matter what sector/ indicators you are taking into the consideration. The scale and breadth of such analysis would be dependent upon various factors (including but not limited to resources, expertise, priority areas, number of interventions to account for).

      Happy to discuss further in case of any queries.

      Regards,

      Zahid Shabbir

      Joint Director | Program Management Division | AC & MFD |State Bank of Pakistan |

    • Dear Abubakar,

      It was nice to see this discussion on the subject. I would like to share our experience in Pakistan as we developed an M&E system for National Financial Inclusion Strategy of Pakistan (NFIS) with the assistance of World Bank Group. I would limit my discussion relevant to your queries.

      First of all, we need to put quite a discern between monitoring and evaluation systems, especially the design of monitoring mechanism is very important. Out here, we developed a three tiered architecture for M&E; (i) monitoring of headline indicators (somewhat similar to broader outcome over the period of time), (ii) Activity/Action level monitoring transliterating into programs, policy or regulatory interventions, (iii) bottom tier, a project level monitoring which could be strengthened using blended tools like result based monitoring (RBM), project performance management using Earned Value Management technique (depending upon nature of project through indicators like CPI – Cost Performance Index, SPI – Schedule Performance Index, etc). Similarly, various concepts like value for money (optimizing output through minimal input) could be used to assess possible impact during the design phase of project/intervention.

      With regards to evaluation specifically, I think least kind would be participatory evaluation where organization itself could be involved to evaluate the impact of interventions it has undertaken in specific sectors. However, choice remains with the organizations how to go by this endeavor, especially if its novice in the area. In such cases, donors/ multilaterals could be engaged to seek professional assistance (as is the case with many developing economies). I am sure, you must be aware of some sort of project performance reports to cater the information about viability of projects/programs like PPARs – Performance Planning and Revies, ICRs – Investor Confidence Rating, MfDRs – Managing for Development Resources, PfR-Project feasibility Reports, etc which are some of the common reporting mechanisms to assess the development effectiveness used by many multilaterals / donors.

      As far as your queries regarding the monitoring of agricultural productivity/ service delivery are concerned, the technological developments have gone by far than required. For instance, you may begin a pilot with Unmanned Aerial Vehicle (UAV) technology in certain area to experiment the productivity results of specific corp/use of fertilizer etc. Similarly, Precision Agriculture is another important technological development which has yielded very good productivity results in case of India. The good thing about these systems is automated data storage and tracking capacities which can greatly assist in monitoring productivity and impact you are looking for can be deduced over the period of time and can witnessed though the ladder of Theory of Change (ToC).

      I hope you would find our experience helpful to get fruitful results.

      Regards,

      Zahid Shabbir
      Joint Director | Program Management Division | AC & MFD |State Bank of Pakistan |