How to evaluate science, technology and innovation in a development context

©FAO/IAEA

How to evaluate science, technology and innovation in a development context

4 min.

Science, technology and innovation (STI) underpin development and can help us achieve all Sustainable Development Goals. Development organizations use STI to accelerate transformational change towards more sustainable and inclusive growth.

However, evaluating the quality of STI is not straightforward for development organizations. Here, I would like to share my thoughts on how development organizations could meaningfully evaluate STI in their work.

The challenge

Many development organizations do not generate STI but gather and share STI knowledge. They provide platforms for knowledge exchange and disseminate the knowledge through knowledge products, such as publications and databases. Evaluating the technical rigour of headquarters-based knowledge products is relatively straightforward. However, focusing on the technical rigor of the headquarter-based knowledge products alone is not adequate in evaluating the STI in the development organizations. This is because development organizations also apply STI in their field activities, and the optimal STI in the field is not always the most cost effective or cutting edge.

I work for the Food and Agriculture Organization (FAO) of the United Nations, which leads international efforts to end hunger and achieve food security for all. The Organization undertakes normative knowledge work at headquarters and provides technical assistance in the field. Here, I will share my thoughts on how I would evaluate STI at FAO.

Impact pathway

FAO’s headquarters-based technical units pool and add value to global knowledge on STI. FAO adjusts the global STI knowledge into the given local context when using it in the field activities to enhance the resilience, environmental sustainability and economic viability of agri-food systems. Lessons learned in the field, in turn, feed into FAO’s headquarters-based global knowledge pool. This mechanism enables FAO to continue updating its knowledge base and sharing it as public goods.

FAO applies three types of modalities in implementing the work on STI: (1) providing intergovernmental / technical platforms, (2) pooling global knowledge/data and publishing research findings and (3) applying the global knowledge through technical assistance, policy advisory services and capacity-development activities.

Optimal STI in the field is context specific as each country has a unique development path suited to its socio-economic needs. Therefore, the Organization needs to select the appropriate STI taking into consideration the needs of the country, and transforms it to fit to the local context.

Scope and purpose of an STI evaluation

An STI evaluation should cover both normative knowledge work at headquarters and operational work in the field. It should also examine the interactions between the two.

The evaluation can aim to inform how STI can enhance the work of the Organization. To do so, it needs come up with concrete recommendations, including (1) how to improve the STI knowledge base, (2) how to enhance the utility of the STI and (3) how to improve institutional mechanisms.

In the context of FAO, the specific objectives of the evaluation could include:

  • an assessment of the relevance and scientific rigour of the Organization's knowledge products on STI in key thematic areas under its mandate;
  • an examination of the effectiveness of the Organization’s institutional arrangements to generate/add-value and disseminate work on STI;
  • an examination of the utility and an assessment of the relevance and effectiveness of the Organization’s work on STI in the field;
  • an assessment of the coherence and synergy of the Organization’s work on STI with its other work, as well as with work on STI outside the Organization;
  • an examination of the partnership aspects of the Organization’s work on STI; and
  • identification of challenges and opportunities, enabling and hindering factors and good practices.

Approach and methodology

To achieve these objectives, the evaluation could undertake:

Stocktaking and sampling: Take stock of the Organization’s key STI themes, flagship activities and products, institutional settings, platforms and implementation modalities, then undertake sampling using the criteria to be established.

Synthesis: Review past evaluations to gather and synthesize evidence of STI in the work of the Organization.

Bibliometric analysis: Conduct a bibliometric analysis of the Organization’s flagship publications and knowledge products to assess their relevance and technical rigour against criteria to be established.

Interviews: Conduct semi-structured interviews with internal (both headquarters and decentralized offices) and external stakeholders.

Surveys: Carry out online surveys to systematically gather feedback from stakeholders at a country level on the relevance, impact and utility of the work on STI in the field.

Case studies: Conduct case studies on selected field activities to identify enabling environments in which STI can accelerate the development process.

Conclusion

Development agencies have increasingly emphasized the role of STI in development. Evaluations of STI, if done properly, can help development organizations to enhance the benefit of the STI‒development interface in their work. I hope there will be more discussions on this important topic.