The evolution and proliferation of Information and Communication Technologies, as well as their increasing accessibility and affordability, is having a great impact on lives, work and opportunities. This trend is being analysed in a broad range of fields, including development, where it has led to a well-established body of knowledge and research called ICT4Dev.
What about evaluation? Is it truly reaping the benefits of ICTs or merely looking at potential applications with caution? Should evaluators embrace the ICT revolution or shy away from engaging more closely? What about big data, their challenges and risks?
Information and Communication Technologies for Development Evaluation, a recently-released book edited by Oscar Garcia and Prashanth Kotturi, sheds light onto the relationship between ICTs and evaluation, discussing ways of harnessing the rapid evolution of technologies and the internet for development evaluation work, and includes examples of frontrunner applications.
ICTs can certainly go a long way to improving the efficiency, timeliness and rigor of evaluations. At the same time, ICTs do have their limitations and bear risks that have to be considered with a critical perspective.
ICTs for use in evaluation include a wide range of tools, which go from simple mobile-based tools and the ubiquitous apps with which we are all familiar, such as Skype and Google Maps, to cutting-edge neural networks for deep learning, artificial intelligence and advanced remote sensing algorithms.
ICTs are already informing data collection, analysis and dissemination and contributing to seamless integration of each one of these stages:
- Data collection: Data collection is benefitting, in particular, from evolution in remote sensing, wireless connection and cloud computing. The increasing accessibility and accuracy of remote sensing images, both those which are publicly available and those which can be collected in a targeted way through drones, offer accurate geospatial information that can be triangulated with other data and information. Evaluations of climate change, forestry, infrastructure and agriculture projects are particularly suited to the application of remote sensing tools. Wireless connection allows for the collection of data during field surveys, which can then be stored and processed in real time through the use of cloud services. This helps to reduce the time gap between the collection and analysis of data.
- Data analysis: Computer-assisted data analysis has seen a surge in efficacy and potential over the past few years. Thanks to the incorporation of machine learning and language processing, vast amounts of qualitative data from unlimited sources and in different formats can be interpreted. Analysis of quantitative data is also being profoundly impacted by machine learning, with applications now able to process data from multiple sources and build predictive models (ex with software such as R and Python).
- Dissemination and learning: Enhancing the contribution that evaluation can provide to evidence-based decision-making will largely depend on how evaluation is able to reach the public domain and inform the public debate. However, the issues of how to disseminate evaluation results more broadly, and to whom these results should be disseminated, have no blueprint solutions. There is certainly room to use ICTs to segment and reach different audiences through targeted channels and by diversifying communication products, languages and messages. Beneficiaries, for example, are often left behind in the return of evaluation results, leading to criticism that evaluation is a purely “extractive” exercise. The easier and faster data collection and analysis mentioned above can offer opportunities to provide feedback in real time to beneficiary groups involved in an evaluation.
Evaluation is today called on to conduct more complex analyses of complex realities, where linear, result-chain thinking is replaced by interactive-results thinking and may involve many more variables than those which are strictly related to the project. This reflects the challenges involved in achieving the Sustainable Development Goals, many of whose indicators are interconnected. In this sense, ICTs such as big data are needed more than ever to help understand the big picture and the more advanced ICT tools required for interaction, learning and interpretation of meaning.
Having discussed the benefits of adopting ICTs, however, should not overshadow the tradeoffs of a more extensive use, such as upfront monetary costs, which include both sunk costs and recurring ones, and the specific skills and knowledge required to use ICTs.
Also, mainstreaming ICTs in evaluation will need to go along with a broader process of adopting them at the organizational levels and across the organizations’ operations.
The book dedicates specific attention to “big data” with a chapter by Michael Bamberger, touching on one of the underlying themes of the entire publication. It explores the reasons why this evolution should not be overlooked by evaluators. Big data provides information and analytical capacities that would have been unimaginable until a few years ago. It is sufficient to note that 90% of all the data in existence today has been generated over the past 2 years! This exponential growth is the result of advances in ICTs and of the Internet of Things (the ways in which ICT tools connect and interact) and is attracting the interest of the business community, the media and the international development community alike. Use of big data can help to address several evaluation challenges but also raises concerns about the commercial, ethical and political implications of the ways in which data are collected, controlled and used. Furthermore, handling big data requires specific data-analysis skills which many evaluators may not have. These two challenges could explain the still-limited interaction between data scientists and evaluators and the slow uptake of big data in evaluation.
The discourse on ICTs and evaluation continues to evolve and this book provides a substantive contribution to the development of the ICT4Eval realm of knowledge and practice.