Serdar [user:field_middlename] Bayryyev

Serdar Bayryyev

Senior Evaluation Officer
FAO
Italy

My contributions

  • Evaluating Scaling Efforts: Measuring What Matters

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear Hezekiah,

      Elaborating further on my responses, and answering your questions, the following considerations are quite important to note. The Food and Agriculture Organization (FAO) employs several context-sensitive evaluation approaches to assess the success of its initiatives effectively. Here are some specific strategies that FAO has used:

      1. Stakeholder Engagement: FAO actively involves stakeholders at various levels, including local communities, government bodies, and NGOs, in the evaluation process. This participatory approach helps to gather diverse perspectives, ensuring that evaluations are grounded in the realities and needs of those directly affected.
      2. Contextual Analysis: Prior to evaluations, FAO conducts thorough contextual analyses to understand the socio-economic, environmental, and cultural factors influencing agricultural practices and food security in specific regions. This helps in tailoring evaluations that reflect local conditions and challenges.
      3. Utilizing Mixed Methods: FAO employs a combination of qualitative and quantitative methods in its evaluations. By integrating surveys, case studies, focus group discussions, and interviews, the organization can capture a holistic view of outcomes and impacts, taking into account the complexities of local contexts.

      And ensuring the success of sustainability in scaling agricultural projects is critical for long-term success and impact. Several indicators can help assess the sustainability of scaling efforts in such initiatives. Here are some key indicators to consider:

      1. Economic Viability:
        - Cost-Benefit Ratio: Measure the economic returns from the project relative to the costs incurred.
        - Profit Margin: Assess the profitability of scaled operations.
        - Market Access and Value Chain Integration: Evaluate the ability of farmers to access markets, as well as their participation in value chains.

      2. Environmental Impact:
        - Soil Health: Monitor soil fertility, erosion rates, and organic matter content.
        - Biodiversity: Measure the diversity of crops and wildlife in the area influenced by the project.
        - Water Usage: Assess water efficiency and the sustainability of water resources used in agricultural practices.

      3. Social Equity:
        - Community Involvement: Evaluate the level of local stakeholder engagement and participation in decision-making.
        - Access to Resources: Monitor equitable access to land, credit, and inputs for all farmers, especially marginalized groups.
        - Gender Inclusion: Assess the involvement of women and marginalized groups in farming, decision-making, and benefits from the project.

      4. Resilience and Adaptability:
        - Risk Management Practices: Evaluate the adoption of practices that enhance resilience to climate change, pests, and market fluctuations.
        - Diversity of Income Sources: Analyze the diversification of income streams for farmers involved in the project.

      5. Technological Adoption and Innovation:
        - Use of Sustainable Practices: Measure the extent to which sustainable agricultural practices are adopted, such as organic farming, conservation agriculture, and agroforestry.
        - Innovation Transfer: Assess the effectiveness of knowledge and technology transfer mechanisms.

      6. Capacity Building and Education:
        - Training and Support Programs: Evaluate the availability and effectiveness of training programs for farmers.
        - Access to Information: Monitor the availability and accessibility of agricultural research and extension services.

      7. Impact on Local Communities:
        - Food Security: Measure changes in household food security levels in the target area.
        - Nutrition Improvement: Assess improvements in dietary diversity and nutrition among local populations.

      Hope this is useful and kindest regards,

      Serdar Bayryyev, Senior Evaluation Officer, FAO

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear Hezekiah,

      We have assessed various programs and interventions of FAO and its partners, aimed at enhancing food security and sustainable agriculture. These reports are available from our website: https://www.fao.org/evaluation/en/

      In many instances, our evaluations informed stakeholders on whether efforts were made for scaling up the programs further, and whether scaling-up has worked or not. We have leveraged evaluation findings to provide actionable recommendations for scaling, highlighting key success factors which led to broader adoption strategies at larger scale or in other locations.
       

      Here are a few illustrative cases:
       

      1. Integrated Pest Management (IPM): An evaluation of the FAO's implementation of IPM in various regions showed significant improvements in pest control and crop yields. The evaluation highlighted how local farmer training and the adaptation of IPM strategies led to successful outcomes. This informed stakeholders about the potential benefits of scaling up IPM programs in other agricultural zones, particularly in countries facing similar pest challenges. The key success factors identified included participatory approaches involving local communities and leveraging traditional knowledge.
       

      2. The Improved Global Governance for Food Security initiative: This program aimed to enhance cooperation among stakeholders in food security. Evaluations revealed that enhanced governance structures were instrumental in enabling better stakeholder engagement and resource allocation. The findings recommended scaling up governance training at local levels as a best practice that could be adopted in other regions. The evaluation emphasized the importance of establishing clear communication channels and collaboration frameworks, which were pivotal in achieving positive food security outcomes.
       

      3. Climate-Smart Agriculture (CSA): Evaluations of FAO's CSA programs demonstrated how integrating climate resilience into farming practices benefited farmers in various countries. The assessments pointed out that scaling these interventions led to improved productivity and sustainability. Key success factors included the provision of technical assistance, access to climate information, and the establishment of demonstration plots. As a result, recommendations were made to expand similar CSA initiatives to other vulnerable agricultural areas.
       

      4. Food and Nutrition Security Program in Ethiopia: The evaluation of this multifaceted program indicated significant improvements in dietary diversity and access to food among targeted households. It highlighted the importance of community participation and localized content in training modules, which were critical for the successful implementation of the initiative. The recommendations called for expanding the program further into other regions of Ethiopia and beyond, based on the positive outcomes observed, thereby underscoring the crucial role of context-specific interventions.
       

      5. Support for Smallholder Farmers: An evaluation focused on FAO's initiatives to support smallholder farmers through access to resources and markets found that these programs had a high success rate in improving food security and income levels. The successful models were identified for potential scaling up in different contexts across Africa and Asia. The report emphasized the importance of building local partnerships and ongoing support mechanisms to ensure long-term sustainability and effectiveness.
       

      These evaluations, among others, illustrate how FAO and its partners use assessments to derive actionable insights that facilitate the scaling up of programs aimed at enhancing food security and promoting sustainable agriculture practices globally. The findings provide a roadmap for replication and adaptation in various contexts, ultimately contributing to broader efforts in achieving food security objectives.


      Lessons Learned:
      One key lesson has been the importance of context-sensitive evaluations that consider local conditions and stakeholder dynamics. Furthermore, engaging with local communities throughout the evaluation process enhances ownership and relevance of scaling initiatives. The criteria that could be used for assessing successful scaling could include the following: 
      - Evidence of increased access or adoption of innovations.
      - Positive changes in target outcomes (e.g., improved livelihoods, enhanced food security).
      - Sustainability of scaling efforts over time.


      I look forward to further discussions and shared learning on this important topic!
       

  • Responsive Evaluation: A joint reflection from Rome-based agencies on the 2024 National Evaluation Capacities Conference held in Beijing, PRC

    Blog

    The landscape of governance and development is ever-evolving, necessitating robust mechanisms for evaluation to ensure that programs and policies are responsive to the unprecedented global challenges. The 2024 National Evaluation Capacities (NEC) Conference held in Beijing on October 14-18, served as a pivotal platform for stakeholders to come together and enhance their understanding of evaluation as a tool for accountability and informing timely and effective responses to the global challenges. 

    The NEC Conference brought together over 500 participants from more than 100 countries representing a diverse group of evaluators, policymakers, civil society representatives, and international organizations. Key sessions and panels

    Posted by

  • Evaluating sustainable pathways to climate-resilient agriculture: recent experiences from an IFAD evaluation

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear Nanthi,

      Thank you for sharing your insights on evaluating sustainable pathways to climate-resilient agriculture. The examples you presented underscore important lessons vital for tackling the intersection of agri-food systems and climate change.

      1. The Role of Evaluation in Sustainable Agriculture  

      I appreciate your emphasis on the significance of evaluation in gauging the overarching context. Instead of concentrating solely on immediate project impacts, evaluations should reflect on the broader objectives of sustainable development. It is essential to analyze the interconnections and synergies among human, socio-economic, and ecological systems, alongside potential trade-offs.

      2. Innovative Methodologies  
      The introduction of a new rubric methodology to examine the relationship between agricultural interventions and ecosystem health is a noteworthy advancement. This innovative approach provides a nuanced perspective on how agricultural practices influence natural resources, facilitating more informed decision-making. Categorizing impacts by typologies will offer greater clarity for stakeholders and enhance their understanding of the associated risks and benefits.

      3. Agile and Responsive Evaluations  
      I completely concur that the unprecedented complexity of today's development challenges necessitates more responsive and proactive evaluation methods. Given the intensifying climate-related challenges, it is crucial to shift away from traditional, lengthy evaluation cycles in favor of more rapid and real-time assessments. Evaluations should deliver timely feedback that supports adaptive management, as such flexibility is vital in the face of rapid changes in both climate and agricultural environments.

      4. Integrating Cross-Cutting Issues 
      Integrating social impact assessments—including considerations for gender equality and women's empowerment—into the evaluation process demonstrates a comprehensive approach to development. By addressing social dimensions alongside environmental factors, a more holistic evaluation can guide decision-makers toward inclusive and sustainable agricultural practices, ensuring that solutions are tailored to respect local ecosystems and meet community needs.

      Best regards,  


      Serdar Bayryyev

      Senior Evaluation Officer

      Food and Agriculture Organization 
       

  • Developing effective, inclusive and gender responsive MEAL systems

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear Eriasafu,

      Thank you for raising these important questions and emphasizing the need for integrating the evaluation function with those of monitoring and learning. All these important functions are inter-connected and as such are integral parts of the effective project management cycle. It would, indeed, be the most effective approach if an evaluation could be integrated in the overall results-focused management system from the start, rather than be carried out as a “one-off” exercise occurring only at certain point of the project/programme cycle.

      I would like to highlight a few points that may stimulate further discussion on the questions that you have posted.

      Purpose and utility

      Before any monitoring and evaluation system is established or developed, its main users/stakeholders and developers need to be clear on the purpose and utility of the system. For example, the system that is mainly generating data for higher-level (corporate) reporting is different from the system that focuses on measuring benefits to the local community or beneficiary-level impact monitoring.  Any system developed need also be pragmatic, and take into account available capacities and resources, without creating or adding levels of complexity or functions that may not be used subsequently.

      Human rights and gender-related considerations.

      The monitoring and evaluation systems should adhere to the rights-based approaches, potentially identifying the effect of programmes on people realizing their human rights and identifying potential best practices of ensuring respect for human rights while implementing operations and programmes. The monitoring and evaluation processes and activities should duly integrate gender-related considerations, measuring any effect of programmes on women, girls, men and boys, and assessing benefits and deprivations. These can be achieved by integrating gender-sensitive indicators of performance in projects’ results and resources frameworks, identifying potential sources of gender-disaggregated information, and ensuring that gender-disaggregated data is collected and used for monitoring and evaluation, to the extent possible. 

      Addressing constraints

      Monitoring and evaluation activities should take due consideration of the projects and programmes’ operating environment and in particular potential constraints and risks. In emergencies and crisis-affected settings, there are multiple constraints and limitations, including issues of secure access to project sites and intended informants, availability and reliability of data, possible biases of key informants in crisis-affected settings due to their location or affiliation. These constraints can be addressed by using local partners/experts, using third party monitoring, and/or use of modern technology that allows remote monitoring and data collection (e.g. remote sensing via satellites, geospatial data available, digital data collection, mobile phone-based data platforms, remote sensing with satellites, etc.)

      Do No Harm

      In all contexts, and particularly in crises-affected settings, “Do No Harm” principles should be applied in planning and undertaking the monitoring and evaluation activities, bearing in mind potential sensitivities and tensions. The proposed M&E approaches should be planned in a manner that do not affect providers of monitoring and evaluation data, and do not exacerbate existing tensions, and/or worsen relationships between the informants and the other local actors/communities. 

      Kindest regards,

      Serdar Bayryyev (FAO)

  • Principles for a healthy relationship between evaluators and experts

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear Ibtissem,

      Thank you for posting a blog on this important issue. The complexity of development interventions requires that evaluation teams have various types of expertise. Yet, it is often quite difficult to secure right mix of expertise, and it is even more challenging to ensure that this expertise is effectively utilized during a short span of the evaluation process.

      Evaluation managers/leaders (hereinafter referred as Evaluation Team Leaders) need to make their thoughtful decisions on what expertise they need to obtain for the evaluation team, and what combination of expert knowledge and experience would work best for their evaluation.  In making these decisions and in leading the evaluation processes, Evaluation Leaders may indeed be guided by the five competency domains highlighted by Ibtissem  (or the Competencies for Canadian Evaluation Practice), which resonate with the 2016 UNEG Evaluation Competency Framework (http://www.unevaluation.org/document/detail/1915 ).  

      The role of the evaluation leader in managing the teams of experts, is indeed strategic and challenging, and I would like to add just a few considerations in this regard, to stimulate further discussion on this important topic:

      Establishing strong leadership

      Evaluation Team Leaders need to effectively lead the evaluation team, coordinating effective engagement of each expert in the team, validating their individual inputs and undertaking quality controls at key milestones. Continuous communication with each team member throughout the process is essential to ensure that any questions are potential risks of drawbacks are timely and effectively addressed.

      Ensuring compliance with organizational standards

      The Evaluation Team leaders need to require compliance of each team members with applicable rules and regulations, and standards of conduct established by the organization, and establish quality parameters of expected deliverables. In collaboration with each team member, Evaluation Team Leader should ensure that there is a shared understanding of what is to be evaluated and how, clarifying how organizational standards and processes are to be applied, and what is expected from each individual team member.

      Focus, focus, focus

      Biases are unavoidable, yet their influence can be minimized by focusing on evaluation objective and clearly establishing the scope of evaluation analysis. Furthermore, it should be made clear that any expert inputs and viewpoints need to be validated and supported with solid and relevant evidence.

      Building relationships

      Teambuilding skills are essential, to ensure that team members support each other and works towards achieving common objective.

      Engaging external expert panels

      If at all possible, external panels of experts may be engaged to provide feedback on key thematic areas of your evaluation analysis. Recognized experts from counterpart organizations may assist by serving as high-level expert panel members, to review findings produced.

      These are just a few examples, and I anticipate continued discussion on this topic. 

      Kindest regards,


      Serdar Bayryyev

       

  • How to evaluate science, technology and innovation in a development context?

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      The need to evaluate science, technology and innovation in a development context

      The significance of science, innovation and research in supporting global efforts towards more sustainable and climate-friendly development is growing. There is an urgent need for relevant science, quality research and innovations that are ground-breaking, as the world is experiencing new and unprecedented challenges and crises. Evaluation of the quality of evaluation and research is essential in determining the usefulness of and effectiveness of science, innovation and research activities.  Evaluation findings should help decision-makers in determining important priority areas for further investigation and facilitate the decision-making on allocating resources for future research activities.

      Key limitations

      The evaluation of science and research is, however, quite complicated, and facing numerous methodological challenges. For example, assessment of the relevance and significance of scientific or research products is mostly based on the use of bibliometric methods. This is a quantitative method that may in fact produce solid evidence-based findings, yet its use is constrained by major limitations.  For example, not all science, innovation and research products are included and properly recorded in the bibliographic databases, or not even published, hence not all products can be assessed.

      The bibliometric methods often are based on calculating average number of citations, which also presents the basis for some biases. For example, there is sometimes an overly exaggerated attention to a specific author, who is known for previous work in a specific field or is affiliated with institutions that have strong political or financial support. In terms of citations, some authors may also deliberately exclude certain reference materials from their publications. Henceforth, whenever the bibliometric data analysis is used, it should be used with caution and should be combined with the use of other methods for validity purposes.

      The other major limitation is that in today’s complex world of science and innovation, there are various standards or criteria of assessing quality of research, science and innovation in various parts of the world, and various parts of science and innovation.

      Assessment of science and research products may also be biased due to the differences in political affiliation, beliefs, culturally or religiously-based perceptions of those who undertake these assessments or evaluations. 

      Key considerations

      As this stream of evaluation function is still evolving, there are few key considerations that need to be taken into account in undertaking relevant evaluation, or in developing appropriate evaluation tools and methods.

      Assessing relevance/significance of science and research.

      Assessment of relevance or significance of science, innovation and research products need to take due consideration to the context in which these products are to be used. What works in one context may not be suited for the other, and what constitutes innovation and ground-breaking science varies substantially depending on the intended use or users.

      Assessing effectiveness (or quality)

      In assessing the effectiveness of the research and scientific analysis, the key is to assess “influence” of these activities, or extent to which the science, innovation and research products have influenced the policies, approaches or processes.   

      Assessing the degree of “networking”, i.e. the degree to which the researchers and scientific institutions have interacted with all relevant stakeholders, including those that may have had a “negative” or opposing stance to the subject research theme/topic.

      Assessing Transformational nature

      In today’s world, perhaps, the most important criteria for assessing the relevance, use and efficiency of science, innovation and research activities, is whether these activities cause truly transformational change, or at least trigger important policy discourse on moving towards such transformational change.

      The above-written are suggestions for consideration, that are aimed at stimulating further feedback into this important discussion.

      Kindest regards,


      Serdar Bayryyev

      Senior Evaluation Officer

       

       

  • Can we use an evidence-based, evolving Theory of Change to achieve "local learning” during project design?

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear John,

      In my view, the Theory of Change, or logical framework, or any other method used to guide the design of the development intervention (project) is critical. These methods should be based on comprehensive analysis of the development context and the critical issues to be addressed to meet the needs and desires of local communities.

      Multilateral organizations have some examples of "community-driven development" projects. For example, the International Fund for Agricultural Development (IFAD) has recently published an evaluation synthesis of "Community-driven development in IFAD-supported projects" (IFAD, April 2020), which is based on the review of case studies of community-driven development projects. The theory of change used for this synthesis was based on the assumption that social capital and empowerment are at the center of the community-driven development approach. This theory of change assumes that participatory implementation process "...is expected to achieve a truly sustainable transformation of rural livelihoods by building poor peoples' capacities to make use of a wider range of livelihood options and by transforming community-government relations to better support people-centred development processes". This theory of change is illustrated in Figure 2 on page 5 of the synthesis paper accessible via the following link:

      https://www.ifad.org/documents/38714182/41898849/ESR+CDD+-+final+with+c…

      Kindest regards,

      Serdar Bayryyev

      Food and Agriculture Organization

       

  • Cohérence des interventions dans le secteur agricole

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear Nabyouré Jean Stanislas OUEDRAOGO,

      Thank you very much for raising this strategic question, which is at the heart of the discussions held today in the context of best practices and development pathways towards achieving the Agenda 2030. All countries, all developments stakeholders, communities and individuals should act in collaborative and productive partnerships to make progress towards achieving the Sustainable Development Goals. 

      The proliferation of development interventions could indeed become a negative factor, if their universe is composed of projects that do not exploit potential synergies and apparent complementarities, and are not based on partnerships based on solid analysis of mutual benefits generated from joining forces, capacities and resources toward common goals.

      We live in communities and environments that are affected by a multitude of factors that are interconnected, broadly defined as social, economic, environmental, health-related and other development factors. Accordingly, the development interventions should be developed with due attention top and in full consideration of these inter-linkages, inter-connections and trade-offs. There are examples of good practices used by development organizations to coordinate and consolidate the universe of development interventions to take full account of development context and exploit potential complementarities in addressing the inherent interlinked development issues and challenged. Some of these examples are show below: 

      • The United Nations Development Assistance Framework, if planned with due consideration of local context and with robust analysis of development challenges, would guide UN entities and other development actors  in producing coherent and well-coordinated package of development support towards achieving national development goals and objectives.
      • United Nations Global Compact that helped creating multi-stakeholder initiatives, supported by the UN, international financial institutions, private businesses and small and medium-sized enterprises, to address development challenges in a more coordinated manner.
      • The United Nations pooled funding mechanisms, which serve as channels for directing flows of development and humanitarian assistance from diverse groups of external factors through national budgeting and financing schemes, helped to improve effectiveness, reduce duplication and promote alignment among a wide range of actors. 

      These are just a few examples from the recent past. The new generation of the United Nations Sustainable Development Cooperation Frameworks (https://unsdg.un.org/sites/default/files/2019-11/UNSDG-SDG-Primer-Repor…), are being developed to move the development aid paradigm from assistance to cooperation, and from individual contributions by development agencies to a collective and coherent response to countries' opportunities, gaps and challenges. 

      The new Cooperation Frameworks will consider development priorities from multiple perspectives of the diverse groups of stakeholders, taking their views as the basis for developing coherent development support package. In doing so, the Cooperation Frameworks will aim at developing interventions that take full consideration of potential effects among different sectors. If done and implemented right, these Cooperation Frameworks will guide the transformation of development projects into a coherent and well-coordinated package of development assistance, aligned with the national plans towards achievement of Sustainable Development Goals.

      Just a reflection.

      Kind regards,

      Serdar Bayryyev

      Food and Agriculture Organization

       

       

       

       

       

       

  • How to use Knowledge Management to strengthen the impact of Evaluation on smallholder agriculture development?

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear Paul,

      Thank you for raising this important issue, frequently faced by development practitioners. While both functions, the M&E on one hand, and the knowledge management, on the other, contribute to organizational learning and effective programming towards generating intended benefits to the people and communities, these functions quite often are managed by different units and follow different organizational practices and cycles. To ensure that these functions complement each other, these have to be planned and synced, preferably at the design stage of the project. For example, the results from periodic monitoring conducted at quarterly intervals, may be used to produce knowledge products, such as newsletters and case study brochures to raise awareness on the results achieved. Annual monitoring exercises could be informing another type of knowledge products - such as the lessons learned. Utility of evaluations can be enhanced by wider dissemination and broadcasting (e.g. via social media, TV, radio), with support from knowledge management professionals.

      In a effort to ensure that both M&E and KM teams work collaboratively together, the following initial steps could be considered and applied: 1) discussing with programme management and beneficiaries their needs in M&E and KM products; 2) Agreeing on a joint plan of M&E and KM activities, focusing on complementarities and sequencing; 3) Developing a plan for effective utilization and dissemination of M&E and KM products.

      Kindest regards,

      Serdar Bayryyev

      Senior Evaluation Officer (FAO)

       

  • Evaluation to support the “leaving no one behind” agenda: insights from the UNDP NEC Conference 2019

    Blog

    Over 500 participants from 100 countries joined this 6th NEC Conference to exchange and learn how to advance national evaluation agendas and to discuss the use of evaluation to accelerate the implementation of the SDGs.

    Leaving no one behind amid the rise in inequalities in society is a mounting concern in both the developed and developing world. Captured in Sustainable Development Goal 10, addressing inequalities needs to go beyond looking at the increasing gaps between higher and lower levels of income and wealth. Inequalities, in fact, touch on a multidimensional set of social, cultural, geographical, ethnic and other drivers that

    Posted by

  • Challenges of evaluation

    Discussion

    • Serdar Bayryyev

      Senior Evaluation Officer FAO
      Italy

      Dear Hynda,

      You have raised a very important question, which affects the quality of evaluation work. Evaluations of development programmes in the broadly defined areas of rural development, agriculture and food security are inherently complex. The assessments of results in these areas are affected by a multiplicity of biophysical, economic, and social systems and factors. There are different types of constraints and challenges in evaluation work that depend mostly on the context of the programmes or policy work being evaluated.  For example, accurate and timely assessments of potential impact and development change may be affected by the remote location of project sites, social stratification of rural communities, time required to produce productivity gains, adoption capacities of local communities, and many other factors.

      Evaluators often encounter issues with availability of baseline data, or information on the prevailing conditions of the development situation at the start of the projects or programmes addressing food security and agriculture development. This issue could be addressed by reconstructing baselines, for example, using ‘recall’ technique, i.e. requesting key beneficiaries or stakeholders to recollect information about these conditions in the past.

      Security situation in the country may also have a huge impact on the access to data and methods we chose for evaluation. The choice of evaluators could also be highly limited, as not all may have necessary clearance to visit high-risk areas, or experience in working in similar situations. 

      Accessibility of project sites may also be restricted or banned. To address these constraints, local consultants with access to restricted zones may provide support in data collection, and potential alternative evaluation methods could be also considered. In recent FAO’s evaluation of the large irrigation rehabilitation programme in Afghanistan, evaluation team faced a constraint of accessing some of the programme sites. The team opted for alternative method by using the open-source data from Google to assess the potential impact of the programme on the livelihoods in those specific sites. Google Earth maps were utilized to measure the expansion of the irrigated area and the vegetative cover along different sections of the rehabilitated canals. The methodology for measuring these areas was also using preliminary information from enumerators in the field who had access to the restricted zones, and were engaged in supporting collection of necessary data and information for the evaluation (e.g. the GPS coordinates of the irrigated areas in the vicinity of the irrigation canals). Then this information was analyzed based on historic data available from Google Earth on before- and after-project conditions and the changes based on vegetative cover at different periods during a year.

      These are just a few highlights of the constraints and challenges that evaluators may encounter in their work and an example of possible ways to address those. The range of such constraints is quite broad, and we encourage all members of this community to share their experiences in addressing different types of constraints and limitations.

      Kind regards,

      Serdar Bayryyev,

      Evaluation Officer

      Food and Agriculture Organization (FAO)