Isha [user:field_middlename] Miranda

Isha Miranda

Visiting Lecturer and Independent Evaluator
Independent Consultant
Sri Lanka

Isha, As address myself an independent elevator working for various organizations nearly 20 years and presently appointed to the Board of Directors  and CEO of Agromart outreach foundation (establish 1989).  well experiences with fully pledge NGO and programme management and evaluations with a strong management skills , monitoring skills, also as a visiting lecture,  trainer and facilitator  working in the field of INGO, Government , private sector, development and in the humanitarian filed. I have been trained and worked as team leader many years exclusively selected lies in the conceptions the most difficult situations and embrace complexity process, has an ability, stability and control, and instinctively resolve problems rapidly sometimes before they fully understand a problem’s significance. An effective trainer bring many experienced to the table also willing to share the knowledge. Notable expertise are Gender and Women empowerment, SDGs and CSO, Public Private Partnership and Governance enhancements.

Member of Associations and Professional Bodies;

    Sri Lanka: Current

  • Member SDG’s People’s Platform Volunteer People’s Review- Sri Lanka 
  • Visiting  lecture , Trainer and Facilitator: Sri Lanka Institute of Administration and Development
  • (Government Civil Service) 
  • Visiting lecture University of Kotalawala  Defence  Academy of Sri Lanka - Post graduate studies 
  • Member Government National Evaluation Policy Development Technical Committee-DPMM of Ministry of Finance   of Sri Lanka
  • National SDG’s VNR Review committee –Ministry of Sustainable Development  of Sri Lanka
  • Member of the Board - Sri Lanka Evaluation Association(SLEvA)             
  • National Committee Member - Sri Lanka Micro Finance Forum 
  • Member of the Board – Agro Mart Outreach Foundation
  • Member – Gender Women Empowerment  Sri Lankan Major Group – SDG 5 and 10
  • Adviser on Gender and women empowerment – NGO consortium of CHA 

Internationally:

  • Member world forum of SDG 16 , 5 10
  • Former Board Secretary - Asia Pacific Evaluation Association.(APEA)
  • Member – American Evaluation Association –(AEA)
  • Member – Community of Evaluators –(COE-South Asia)
  • Member -Technical Advisory Evaluation Committee –UNFPA-New York

EvalPartners: Member of the Eva partners working sub-group: Eval SDGs

My contributions

  • Can visual tools help evaluators communicate and engage better?

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Son muchos los beneficios de la utilización de herramientas visuales

      Los elementos más cruciales son 

      1. Visualización de datos. En términos de recopilación de datos cualitativos utilizando procedimientos de recopilación de datos. 
      2. Visuales basados en pruebas para presentar los hallazgos en términos de actividad y resultados concretos, así como de consecuencia e impacto. Por ejemplo, el impacto de una pandemia o el control de una enfermedad específica en los sectores agrícola y sanitario, o los esfuerzos de consolidación de la paz.  
      3. Diagramas, detección de plagio, retroalimentación y evaluación por pares, encuestas (TV, medios sociales y grabación de entrevistas a informantes clave) y sondeos en el aula en términos del contexto de la educación.
      4. Informe visual - Resultados - Positivos y negativos. Por ejemplo, la mayoría de los informes de la JICA (Agencia de Cooperación Internacional de Japón) se combinan con imágenes. 
      5. También proporcionan a los usuarios flexibilidad y diversidad al permitirles seleccionar entre una amplia gama de elementos gráficos, formatos y estilos y adaptarlos a sus gustos y objetivos.

       

      Los retos son: Las encuestas en línea necesitan la rendición de cuentas de los datos y la identificación del encuestador y el encuestado. 

  • Neutrality-impartiality-independence. At which stage of the evaluation is each concept important?  

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Estimada Mallika

      Estoy muy orgullosa de que hayas sacado este tema a la luz. Nosotros, como evaluadores, siempre actuamos con neutralidad e imparcialidad. Pero muchas organizaciones como la Naciones Unidas, el Banco Mundial, el Banco Asiático de Desarrollo, etc. (como dice Abubakar: "La mayoría de las veces los evaluadores se seleccionan debido a algunas conexiones y, una vez seleccionados, esperan ser seleccionados de nuevo en el futuro") se sienten cómodos con algunos grupos de evaluadores o empresas de evaluación una y otra vez. 

      He revisado muchos informes de evaluación y he encontrado un deterioro de la profesionalidad en los informes de evaluación, ya sea porque: a) son muy parciales, b) carecen de sinergias entre los hallazgos y las recomendaciones y las conclusiones, c) son informes extensos de casi 100 páginas, sin análisis productivos, falta de recogida de datos productiva, menos profesionalidad en la recogida de datos y en los cuestionarios. Estas son algunas de mis observaciones.

      Creo que ha llegado el momento de alzar la voz en este sentido, presionando también por la profesionalidad de la evaluación, dadas las prioridades anteriores, para salvar esta profesión. 

  • Are you considering social and environmental criteria in your evaluations?

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Los problemas son los siguientes 

      Falta de evaluación horizontal: el problema más común en los TdR es la falta de evaluación horizontal. Hay varias lagunas en el proceso de evaluación que podrían llenarse pero, en cambio, los TdR a menudo piden centrarse en su tema de forma vertical y no horizontal.
      Nosotros, como evaluadores, estamos obligados a ejecutar los TdR debidamente y esto provoca:


      - Falta de supervisión: deberíamos poder utilizar en nuestro beneficio los elementos externos importantes que puedan haber influido en la intervención.

      - Carencia en las herramientas de observación, como las herramientas de comportamiento de las partes interesadas

      - Carencia en los debates de los grupos focales: es decir, falta de tiempo y preparación

      - Lagunas en los factores externos al programa.

      También suele haber un conocimiento inadecuado de la comunidad, los lugares o la intervención por parte del equipo de evaluación: la mayoría de los evaluadores llegan al tema con nociones preconcebidas basadas en sus experiencias anteriores. Este es uno de los errores más graves que se cometen. Creo firmemente que los programas comparables pueden enfrentarse a diferentes obstáculos, metodologías, perfiles de beneficiarios y comportamientos, y que hay que tener en cuenta las variables temporales. Hay que pensar en nuevas dificultades. No utilizar el mismo equipo una y otra vez.

      Estos son algunos de los aspectos sociales y medioambientales que deben incluirse en la evaluación:
       

      Los medios de vida deben evaluarse en función del tema y del contexto local, por ejemplo, la producción de cultivos: la evaluación debe examinar la colaboración y la cohesión del gobierno o de cualquier otra organización, la capacidad (por ejemplo, la logística del gobierno y los servicios a la comunidad: instalaciones sanitarias, educación, centros agrícolas para la promoción, la capacidad de los beneficiarios de la recogida de productos, el desarrollo de la capacidad de colaboración) y los nuevos conocimientos sobre los cambios climáticos, la mitigación de los riesgos, las subvenciones del gobierno (consejos, fertilizantes, semillas, tecnología, etc.).
       

      Contribución de las partes interesadas: Evaluación de los conocimientos de la comunidad y de las acciones relacionadas con sus medios de vida. Lagunas de coherencia, conocimientos técnicos, logística (localización y nuevas tecnologías de desarrollo de cultivos), conocimiento de los productos, información sobre el mercado, variables de precios y contribuciones de los intermediarios del mercado. Además, hay interferencias e impactos políticos locales.
       

      Actores (comportamientos de los beneficiarios) - Consistencia en los campos de producción: aspectos de salud (riesgos de bienestar, deterioro de la salud por epidemia, enfermedades infecciosas, abusos familiares y externos, niveles de educación (formal, informal y subjetiva), nivel nutricional de la familia (adultos y niños).
       

      Gestión financiera - Pobreza: Reducción Trampas, ganancias y pérdidas, las mujeres tienen mayor acceso a la microfinanciación dentro y entre las familias. El crédito debe dirigirse a los hogares de bajos ingresos, especialmente a las mujeres.

       

  • Reporting and supporting evaluation use and influence

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Estoy de acuerdo con los comentarios anteriores de John y Silva. 

      La responsabilidad de los evaluadores es dar recomendaciones, no soluciones. Pero las recomendaciones ayudarán a las "soluciones". 

      Lo que falta en la práctica de la evaluación es  

      a. La mayoría de las recomendaciones son poco realistas y no son realizables. 

      b. Es hora de que los evaluadores cultiven una visión de futuro. 

      c. Los resultados y las recomendaciones deben tener en cuenta el programa (de forma holística).

      d. Los evaluadores deben diseñar sus propios indicadores de evaluación y esto debe incluirse en los términos de referencia. 

       

       

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Estimado Mauro,

      Confío en que este correo te encuentre bien. Ciertamente, como dice Dorothy, la evaluación tiene un componente sobre la presentación de los hallazgos y la retroalimentación antes del informe final.

      Para mí, esa es la mejor parte del ejercicio de evaluación. En algunos casos, solicité a las organizaciones que brindaran una buena representación durante la presentación de los hallazgos y la retroalimentación, incluidos diferentes niveles, como subnacionales, nivel central, a veces funcionarios de campo. Los beneficios de presentar los hallazgos de la evaluación son que podrá participar en la verificación de datos y también recopilar más información cualitativa, así como superar el malentendido de que la evaluación es un ejercicio de detección de fallas, etc.

      También podría tener una discusión previa con las partes interesadas sobre el proceso de evaluación.

      Isha

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Estimados todos,

      Estoy de acuerdo con Silva, los informes de evaluación son muy burocráticos. Aunque la gente no lea, tener un gran informe parece todo un "trabajo bien hecho". 

      Pero muy pocos lo leerán todo, puede que no lo lean en absoluto, puede que sólo lean la página del resumen ejecutivo. 

      Los requisitos para los informes de evaluación deberían venir con los términos de referencia, pero lamentablemente los términos de referencia son muy poco informativos y carecen de innovación o visión. En su mayor parte, debería decir que son de cortar y pegar. 

      La evaluación no ha cambiado mucho en los aspectos mencionados. 

      Saludos cordiales 

      Isha

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Estimados Jean, Gordon y otros,

      Gracias por un tema tan bueno para la discusión. Lamentablemente, tenéis razón al plantear estas cuestiones.
      Muchas veces he visto informes de evaluación que son voluminosos, con demasiadas cosas que leer y también un gran desfase entre las conclusiones y las recomendaciones.

      Escribir demasiadas páginas hará que el lector se aburra, leyendo algunas cosas que ya conocía o buscando ir al grano.

      Mi recomendación es que sea sencillo.

      1. hacer una página ejecutiva de menos de 4 páginas (escribiendo en ambos lados), destacando los resultados y la conclusión y las recomendaciones basadas en los resultados.

      2. Hacer un resumen de menos de 10 páginas, con más tablas, diagramas y resultados en viñetas.

      3. El informe completo debe tener 50 páginas.

      Saludos cordiales
      Isha

       

  • Making more use of local institutions in evaluation

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Con Covid-19 veo a los consultores nacionales de mi país más implicados en la evaluación. Aquí algunas respuestas a las preguntas propuestas:

      1. ¿Cómo se gestiona este cambio de responsabilidad? ¿Qué instituciones están implicadas: gobierno, universidades, ONG, empresas de consultoría privadas o particulares? 

      En mi contexto, veo tres entidades mayoritariamente implicadas en la evaluación: las consultoras privadas (en su mayoría), las universidades y las ONG. Las universidades son muy teóricas y buscan detalles que no pueden encontrar sobre el terreno; suelen redactar informes largos y exigentes. 

      2. ¿En qué medida se está asumiendo esta responsabilidad? ¿Sigue limitándose a la recogida y el análisis de datos, o incluye una mayor responsabilidad en la gestión de la evaluación?

      Veo ambas cosas. Algunas empresas de evaluación se encargan de toda la organización, coordinación y colaboración del proceso de evaluación en nombre de la unidad de evaluación. Otras sólo se encargan de la recogida y el análisis de datos. 

      Ambas son designadas como Consultores Nacionales o consultores nacionales de evaluación. 

      3. ¿Cómo se financia este trabajo?

      Creo que siguen prevaleciendo las contribuciones de los donantes y que los presupuestos de la mayoría de los programas incluyen el presupuesto de la evaluación.

      Creo que debido a Covid19 no hay muchos efectos de costes debido a la menor participación de las consultorías internacionales. En mi caso, la mayoría de las veces me contratan por días o con una cifra fija con perdiem.

      4. ¿Cuáles son las dificultades encontradas? 

      Sobre la formación: los recolectores de datos necesitan ser formados para la tarea aunque ya estén formados.

      En cuanto a la elaboración de informes: suele ser la tarea del consultor nacional. Es un gran reto y es donde la experiencia entra en juego. Si tienes experiencia en trabajar en la evaluación de programas nacionales, haces las cosas correctamente.  La mayoría de las veces hay que identificar el proceso, seleccionar los documentos e identificar a las partes interesadas.  En la mayoría de los casos hay que rediseñar las preguntas y los métodos para adaptarlos al contexto local. 

      Entre los retos a los que se enfrentan los evaluadores debido a Covid 19 también se encuentran

      más revisión y zoom de la entrevista en función de las preguntas establecidas de antemano. Sin embargo, ambas partes deben conocer bien la tarea y el programa en detalle. 
      reuniones de entrevista: muchos no se sienten cómodos (sectores gubernamentales y de base) con las entrevistas de zoom. 
      Si no se puede organizar bien la plantilla, será un poco difícil informar. 

       

  • How do we move forward on Evaluation Systems in the Agriculture Sector?

    Discussion

  • Racism in the field of evaluation

    Discussion

  • How do we adapt our evaluation approach to the impact of the Covid-19 pandemic?

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Hoy en día la evaluación ya no puede ser un ejercicio independiente. En el contexto de esta crisis, es necesario que se convierta aún más en un proceso continuo y orientado a la acción a lo largo de la intervención.

      Tiene que contribuir concretamente al bienestar social, ambiental y económico de las personas, así como a la paz a nivel local y regional, así como a nivel mundial.

      Cito a Michael Quin Patton: “Todos los evaluadores deben convertirse ahora en evaluadores de desarrollo, capaces de adaptarse a sistemas de dinámica complejos, prepararse para lo desconocido, para incertidumbres, turbulencias, falta de control, no linealidades, y para el surgimiento de la Inesperado. Este es el contexto actual en todo el mundo en general y este es el mundo en el que la evaluación existirá en el futuro previsible” y recomendaría leer su blog: https://bluemarbleeval.org/latest/evaluation-implications-coronavirus-global-health-pandemic-emergency

      Isha Wedasinghe Miranda
      Independent Evaluator and Programme Management Consultant

      Sri Lanka 

      Isha Miranda.ppt

  • Youth in agriculture: what lessons can we draw from evaluations?

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dorothy : So my real question is .... are evaluations making a difference or not? If not, how does that happen to greater effect?

      Answer: If the evaluations are focus on “why? Factor” and “how actor”? on youth instead of the Agriculture only, the   following needs to emphasis on findings and recommendation;

      1. Addressing on what? less negatives more positives

               Gaps and lesson learned – focus on the opportunities within and human development and          technology

               Opportunities- Focus on next generation as well as  transformation from traditional        farming to  entrepreneurs - Horizontal analysis

      2.     in the process of Evaluation  can  focus to attract youth to get involve in the process of “evaluation” where participatory and visibly youth can see the evidence that can be produce for a behavior changes on occupations on agriculture among youth.  (Messengers)

      3. Recommendations: achievable and magnetisms

        • Clearly clarify the sustainability of the agriculture short term and long term  
        • from traditional industry to  technical transformation in  agriculture
        • Answers to the risk and assumptions in youth prespectives
        • Society in terms of accepting they are farmers. (Social Acceptance) as a profession
        • Linkage with other professions  which can enhance/impact and sustainable  agriculture industry   
        • Recommendation on professionalism in capacity development
        • taboo Traditionalism in land ownership - specially South Asian culture - Gender discrimination issues. 

       

       

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      There is no concept called Youth agriculture evaluation. I believe evaluation does not have barriers, the profession comes with complete packages including technical experts.  

      Overall youth in evaluation are very limited around the world. Most youth have not been exposed to the profession due to many veterans still engaged and dominated in this profession.Most  either who had been ex -members of the UN organizations or associates who are either by retiree of international organizations. Also organizations too looking comfort zone to hire same professionals.

      i.e. Youth programme evaluation conducted by seniors, aging retiree professionals who are unable to understand the minds of youth. 

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear Dorothy

      On engaging younger people in agriculture, I would like to share this article “Why are our youth not interested in agriculture?” by David Felix https://searchlight.vc/searchlight/our-readers-opinions/2006/08/25/why-are-our-youth-not-interested-in-agriculture/

      Regards

      isha

  • Recurring errors in public policies and major projects: contributions and solutions from evaluation

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear All, 

      I am sharing the Sri Lankan experience as a contribution to this debate. I am currently participating in the development of the Policy framework of M&E and one of the guiding principles we are using is the SDG concept of "Leaving no one behind".

      What are the most common mistakes that you face in your country?

      1. Common mistakes are linked to the fact that politicians make election promises that are not achievable and deliver election winning economic analysis.
      2. The biggest issue is that the country holds to outdated policies and laws, which remain unattended. For instance, it is common that countries that have been under a colonial era still have policies drafted in that era being used in government economic and social governance as well as laws in the country, which are ineffective and irrelevant.
      3. Too many governing structures: national, provincial and local authorities govern in parallel but they have underneath different political agendas which can be seen mishandling some policies, misinterpreting national priorities, mismanaging national interventions and resources etc.
      4. Politicization of governance structure and bureaucracy: everybody wants to remain in power and to remain in the position, which paralyzes the governance.
      5. Corruption at all levels, and hidden corruption even more dangerous than the visible one.

      Here are my suggestions to improve the use of evaluation in policies and programmes:

      1. Key to the any country economics planning is setting up a separate independent unit for monitoring and evaluation under the Act or Policy of government constitution/legislation to safeguard taxpayers money, accountability and transparency of the government programmes and projects/ interventions. This unit can be established under either the Planning ministry or the Ministry of finance but needs to maintain independence.
      2. The National Auditor general should focus performance audit function, operation or the management systems and procedures of a government entities to assess whether the entity is achieving economy, efficiency and effectiveness in the welfare and employment of available resources. This is a qualitative method.
      3. The Ministry of policy planning or another responsible entity should undertake in-depth analysis of all policies and trade agreements, government circulars and amendments with policy activities, to identify the gaps and lesson learned
      4. Develop a National M&E policy and Policy framework (the Sri Lankan government is in the progress of developing the framework). The M&E Policy can be mandatory. Set up expert and technical committee of National M&E / reviews and assessments committee. This will be the backbone of the planning and finance entities. The committee responds to the ministry of finance and Ministry of policy planning and economic reform.
      5. Promote joint evaluation with other funding and donor agencies and public participation in order to strengthen ownership.

  • What can we do to improve food security data?

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear Emile

      Answering to your questions is highly complex in terms of collecting data and collecting quality data.

      See below my answers, which I hope will somewhat help you or that we can come together to improve the data collection.

      • How can we monitor and evaluate efficiently progress towards the SDG2 – End Hunger if we cannot count on reliable data and consequent statistics and indicators?

      All SDGs are complex, unable to be determined through direct data and most data vary from community to community.

      • Do you think there are also weakness and  challenges in data collection in your country?

      Yes we do have same issues. Departments of censers and statistics are not able to identify the data collection methodologies for SDGs. Practically it has been a nightmare to them.

      Another issue is that we could be able to gather many quality data could through CSO's. But those data are not being sharing at national level and within the organization. Therefore, we are losing quality data and useful data.

      Therefore, how do you address this issues?

      Then it comes to "Big Data". How accurate and viable when you know that we are missing some data?

       

  • What can evaluations do in terms of capacity development?

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear All,

      This is an interesting discussion, please see below my comments.
      Evaluation is not just about doing and writing a report. It also includes competencies in effectively designing, managing, implementing and using M&E. It includes strengthening a culture of valuing evidence, valuing questioning, and valuing evaluative thinking.  In terms of "Evaluative Thinking" it is not just looking at the programme, looking at the data analysis and giving the conclusions and recommendations, but also forward thinking beyond the programme. Looking for an unexpected theory of change rather than the expected one, it is visionary thinking, horizontal broad approaches. These are not capacities that can be learned by in training classes or in workshops only, but also need to develop a curriculum in the field. Joint and participatory evaluations are one of the methods to gain these experiences.

      Isha

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear Luisa and Lavinia,

      Thanks for sharing your work and thinking on Evaluation of Capacity Development and the framework you are working on.

      Unfortunately, many of these frameworks are useless if we do not develop the capacities of evaluators in the first place. Most frameworks are uninformative and lack of room for subjective and suitable adjustments.

      For instance, you ask about participatory evaluations. Most TORs are participatory in general (I think it is always cut and paste) and indicate methods that are generic and not up to evaluation expectations. In agriculture very often evaluators are either agriculture specialists or researchers; they are not evaluators and are not able to capture the specific needs of farmers that differ one from the other despite being all farmers. Often they use blanket commonly known questions.

      Participatory approaches are certainly a way to carry out a meaningful evaluation and to develop capacities of evaluands and beneficiaries, but to get there we have to make sure participation is effective and not a token. 

      Here some recommendations:

      1. We need to develop capacities of evaluators on the use of participatory approaches so that they are able to understand farmers’ points of view.
      2. Develop a tool / guideline for participants on participatory sessions.
      3. Elaborate soft skills as well as on the job skills on evaluation for young and emerging evaluators
      4. Develop capacities of trainers and facilitators on how to address marginalized people and indigenous communities in line with the focus on equity and gender.
      5. Guidelines: competences on selecting evaluation organization and external evaluators (individuals). 

      Isha Miranda

      Independent consultant and trainer, Sri Lanka

  • Is this really an output? Addressing terminology differences between evaluators and project managers

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dearest Natalia,

      I am not surprised either and I have similar experiences with many organizations.

      This is the one of the reasons why I decided to lobby for “freedom of speech for evaluators”. Let me explain what I mean.

      First, we cannot be independent and bound to be under ToRs instructions; this is not how evaluation should be conducted. Evaluation is a lesson learning, gap-finding mission to eliminate obstacles and prepare to become a visionary leader, seeing things early, logically and to respond and share with rest of the stakeholders and guide them to take things forward.

      I have done some fact-findings on this subject:

      1. In many organizations, the M&E entity or unit lacks evaluation knowledge at the field or ground level. They are leaning the art during the evaluation period from consultants.
      2. Most evaluators are bound to work as per the “Handbook” given by the organization and are unable to attend to any changes if required. Very uninformative.
      3. Most handbooks are more similar to curriculum for evaluation studies than “guidelines for evaluation”.
      4. A very few “Handbooks” are evaluation oriented, and instead focus on research approaches. Even the Kellogg foundation evaluation handbook does not differentiate the Researcher and Evaluator in https://cyc.brandeis.edu/pdfs/reports/EvaluationHandbook.pdf (see page 7).
      5. In some cases the definition of the Evaluation is questionable in both documents, Handbook and TORs. i.e. TOR and handbook are not compatible. The Handbooks give guidance on major programmes evaluations or end/post evaluations but lack guidance and examples in the field and ground challenges.
      6. Very few give templates for a TORs Terms of Reference
      7. Many ToRs are cut and paste
      8. Most manuals include standards for all evaluations including the questionnaires as well as target groups and instruction on methods of conducting the interviewers and on selection of target group identified by the programmes. Most of the time they are very biased.
      9. No questions are suggested to address the indirect outcome or impact of the programme but only direct expect-able answerable questions.

      Overall sometimes I have asked the organization "why do you hire me, you can do it yourself" when they give and indicate everything on the methodology to conduct the evaluations.

      Best

      Isha

      Independent evaluator

      Sri Lanka

  • What about stakeholders? Evaluation of social protection programs.

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear Patricia,

      see below my comments on your questions 1 and 2.

      1.      Have you ever been involved in the evaluation of social protection programs? What is the approach to assessing such programs?

      Yes, nationally and sub-nationally as well as internationally. At national level in my country the Government has created a program called "Samurdhi", in English "Prosperity". It is a social protection program for citizens living well below the poverty line. The vision is "To make a Poverty Free Empowered and Prosperous Sri Lanka by 2030”. The mission is “Contributing to economic development through the building up of a poverty free prosperous country by empowering disadvantaged people (economically, socially, politically, physically, psychologically, legally and environmentally) and minimizing regional disparity through delivering effective, efficient, speedy and productive solutions in a people-friendly manner through the satisfactory contribution of the network of Departmental, Community Based Organizations and Micro-Finance Institutions and professionals with the collaboration of the private, public, people and political sectors and local and global agencies“. 

      At sub-national level, micro finance institutions, NGOs and other government sub-national entities have taken over most of the targets from economic to social achievements. Therefore, most programs circle it around this objective. However, there are gaps showing in this program due to various social and political environment embedded into the systems.     

      Initially the program adopted the longstanding welfare approach using both monetary approach and non- monetary approach.

      Then it expanded to address the multidimensional aspects of poverty such as economy (consumption and assets), human development (education, health, safe sanitation, safe drinking water, electricity), socio-cultural dimension (dignity and network), political dimensions (power and voice) and protective aspects (conflict, natural disasters, risk of eviction).

      For more information:

      http://www.samurdhi.gov.lk/web/images/stories/publications/english/samurdhi_programme.pdf

      http://repository.kln.ac.lk/xmlui/bitstream/handle/123456789/5305/MK%20Nadeeka-43-64.pdf?sequence=1

      2.      What are the key elements that any expert would be looking for in social protection activities/programs?

      The evaluation questions should address both economic and social aspects of the social protection programme,to assess its contribution to community development in rural areas: 

      •             Is the programme sustainable, contributing to a stable community rather than creating dependency?

      •             Are the program elements linked with national priorities in terms of livelihood development? F.i. are there links with agriculture/non agriculture sectors, government Agribase/non Agriculture trade subsides/welfare programmes, and activities relating alternative and product development  non traditional agribase products?  

      •             Is the program considering land Management, Agriculture land distribution and harvesting technics?

      •             Are there activities supporting livelihoods such as market development activities relating to the local areas, market expandable beyond the local area, usable technology and introducing new methods, product development?

      •             Financial inclusion, control over income over expenditure management, loan management.

      •             Family management and prevention of addiction: such as alcoholism, drugs (locally and internally made stuff) local gambling, abuses and harassment on women and children.   

       

  • How to Measure the Impact of Monitoring and Evaluation Work

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear Abubakar Muhammad Moki,

      I would like to share the Sri Lankan experience. In Sri Lanka the Evaluation Policy reached the legislation after 16 years and was launched in Sep 19th 2018.  The Policy framework is now under construction.  I have been through this process all these years and now are a member of the National Policy Framework Technical committee.

      First and foremost, it is important to create a culture of evaluation and key activities can fall under three broad areas:  

      1. Enabling environment,
      2. Institutional Capacity,
      3. Individual capacity in the country at all levels from national to sub national government.

      If the country does not have a Policy on Evaluation it is necessary to develop it in order to create the legally bonded “must” culture. However, policy alone is not enough to create a culture, which needs the following activities:

      1. Enabling environment, creating a demand before the supply
      • Do not wait for the whole government but can do it organization by organization.
      • If the government has a central agency who already does monitoring you could add or transform the mandate to Monitoring and Evaluation.
      • Setting up national policy combined with sub-national.
      • All the international donor funded intervention (INGO’s, Finance organizations etc): at the time of the contracting government should/could ask for a joint evaluation activity (i.e. evaluation team will comprise with Government and funding organization members) it is very much a lesson learning exercise that will help in creating ownership too.
      • Set up a “must” activities and embed the evaluation into that with inclusiveness in every government own program/intervention/project either nationally or sub nationally.
      • Lobbying the legislation either upper house/lower house/senate on policy on evaluation through members who are champion of good governance. Even few members can have a large impact.
      • Create more awareness by utilizing your VOPEs and other Professional Associations.

      2. Institutional Capacity: policy and frameworks take a long time to materialize. Therefore creating awareness on “what is evaluation” and “how” will benefit to their work and knowledge among the government institution and can make a tremendous impact. Take this message and promote the upcoming policy and framework even it is not yet ready.

      Target institutional capacity for:

      • Government and Sub –National institutes
      • Academia – Setting up workshops and other possible certificate or diploma programmes by practitioners of the field of evaluation.
      • Institutes like Project Management, Government Administration training institutes, universities etc.
      • Private sector institutes under Project Management etc.

      3. Individual capacity: this is important to create the culture.

      However, it is better that above centres enroll all the members of the working world to adopt as part of their job as learning and implementing evaluation “Must”

      Challenges:

      • Resources and Practitioners in the country also need international support in order to upgrade the skills and methods. i.e. evaluation is not a uniform effort it’s change all the time, methods, approaches norms and standards etc.
      • Make it important and vital also how to create demanding culture,
      • Evaluation is very costly and how do you show the authorities that it is Value for Money.
      • Raise awareness to all via language they understand. i.e. legislation, individuals and institutional level too.

      If you need any clarification or any other assistance, please ask me I am happy to help.

  • Challenges of evaluation

    Discussion

  • The issues facing global agriculture

    Discussion

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear Bintou,

      Many thanks for the response. This is what I like about EvalForward, it gives us an open platform to talk about many things.

      I guess that I only somewhat agree with what you say, because my thinking is that the future evaluations should take a broader approach than what we normally do.

      I would like to quote my friend Zenda Ofir focusing on DAC criteria, as we both honestly believe that DAC criteria need a face lift and argue that time is up for changes of the old with new thinking. 

      Zenda says "yes, we can have ‘top-down’, ‘bottom-up’ or ‘up-and-down’ interventions and strategies. But we need to be much more aware of the realities within which we should aim to make a difference".

      Also she says that "We are working ourselves into a technocratic, simplistic notion of development, humanitarian work and evaluation that makes us increasingly irrelevant for that which matters now.

      Yes, it is in part the result of the political economy in which we work. But we are not that powerless to change key aspects of our work. It is a matter of will and conviction".

      http://zendaofir.com/updating-the-DAC-criteria-part-10/.

      Best regards

      Isha

       

    • Isha Miranda

      Visiting Lecturer and Independent Evaluator Independent Consultant
      Sri Lanka

      Dear EVAL-ForwARD members,

      I was delighted to see a lively debate raising from my discussion question! 

      To my query about the need to address the burning issues I mentioned when evaluating agriculture-based interventions, the majority of your responses drew the attention on the scope of the evaluations, which are bound by theories of change. 

      I continue to think that the commissioner/s of evaluation must take these concerns seriously when it comes to either policy level evaluation or to any activity related to agricultura-based programme evaluation in their future assessments. I strongly believe that as evaluators we should find a way to incorporate these burning issues in the TORs of evaluation for the benefit of programme implementers, and entities, so that they come to think more about productive and sustainable manners to lead there interventions in future.

      I thank you all for your insightful contributions.

      Isha Wedasinghe Miranda

      Sri Lanka

  • What works in improving food security and nutrition in very poor communities?

    Discussion