Can Artificial Intelligence Accelerate Theory-Driven Complex Program Evaluation?

Can Artificial Intelligence Accelerate Theory-Driven Complex Program Evaluation?

This paper was prepared by the Independent Evaluation Group of the World Bank for international development specialists, with both evaluation professionals and nutrition sector practitioners in mind. It presents the methodology and results used to pilot and test the applicability, usefulness, and added value of using artificial intelligence for advanced theory-based content analysis. Traditionally, qualitative synthesis would be used to perform a theory-driven structured analysis of project reports. This pilot sought to assess the efficiency gains generated by artificial intelligence–assisted content analysis in labeling and classifying text according to an outcome-based conceptual framework. The approach used a set of interventions associated with the World Bank’s stunted growth and chronic malnutrition evaluation portfolio, consisting of 392 unique project reports from 64 countries.