Will artificial intelligence replace us as evaluators?

Will artificial intelligence replace us as evaluators?

5 min.

Technological progress can make jobs obsolete, will evaluators follow suit?

Artificial intelligence: A threat or a game-changer?

ChatGPT is taking the world by storm. It is a revolutionary technology with the ability to provide human-quality responses while interacting in conversational dialogue.[1] But let’s not focus on this particular tool; let’s talk about technology in general and artificial intelligence (AI) more specifically. I am more concerned about the exponential growth of technological invention and how far it can go in replacing us humans in the evaluation profession.

Let’s talk hypothetically for a moment. Imagine freezing a person in the seventh century and waking them 500 years later. What would have changed? Not all that much, really … a few empires, clothing, language, perhaps, but a person could easily adapt to his/her new surroundings (at least in theory). Now imagine you freeze a person in the 1980s and wake them in 2023. Would that person be able to understand all the changes that had happened in that short timeframe? All the progress of the past 30 or so years? This is the exponential growth I am referring to.

Will evaluation professionals slowly vanish from the market?
photo of aircraft listener and knocker-upper

 

The above pictures are testament to the fact that technological progress can sometimes make other jobs obsolete, as was the case with “aircraft listeners” and “knocker-uppers”, long since replaced by radars and alarm clocks, respectively. Will evaluators follow suit?

Our work depends on technology. Writing a report or collecting and analysing quantitative data, for example, are two tasks that are quite difficult to accomplish without the use of a computer. Developing a theory of change, monitoring key performance indicators, comparing studies, analysing a portfolio … the list goes on; are all tasks that we are accomplishing and taking to the next level courtesy of the newly available tools at our disposal. We humans provide the guidance, the thinking and the analysis. We design, collect and write, assisted by technology. But what if technology is soon able to replace all of these tasks?

This question occurs to me as I analyse a database in our beloved software, Excel. I decide what to analyse, then select, design, collect, visualize and interpret. The hardest parts, the calculations and the graphics, are done by the tool. The thinking here is that in a constantly evolving world, technology is at the forefront of progress. It provides solutions to any simple or complex environment and makes us more dependent on new tools as the days pass. We evaluators are no exception to the rule.

Another example I could give is NVivo software ‒ a qualitative data analysis software that can organize and manage large volumes of interview transcripts, survey responses and other qualitative data sources. It’s not quite the ultimate tool, but a valuable one that helps to extract information, with ample room for progression.

Technological tools are already helping us extensively in our day-to-day work and are growing in importance as the years go by. Are they likely to grow in tandem with AI until they reach a point of no return, where we become obsolete and they take over the entirety of our function? Are they likely to be able to take over the guidance, thinking, analysis, design and other tasks we usually do?

Technology and AI will find their limits

In response to these questions comes the latest technological invention ‒ artificial intelligence, the ability of machines to perform tasks that would normally require human intelligence, such as visual perception, speech recognition and natural language processing.

Even though it has made significant strides in automating various tasks and functions, I believe AI will reach its limits and not be able to fully replace the essential role of human evaluators.

First, AI lacks the ability to understand context, which is necessary in our field, especially in complex settings where understanding the unique and specific challenges we may face is key. It also lacks human judgment and subjectivity, and may not be able to consider certain relevant factors or bring subjective insights and personal experience to the evaluation process. These are often very important in complex or nuanced evaluations.

Second, AI relies on data and algorithms to function and make decisions, but these may be incomplete or inaccurate. Moreover, there is a limit to algorithms, as they can in no way replace the human interaction, empathy or emotional intelligence required in all evaluations. Evaluation requires understanding and managing emotions, as well as building relationships.

Third, evaluation jobs often involve ethical considerations and adaptability, which cannot be fully addressed by AI, at least for the time being. In an ever-changing world, it is hard to see automatons and algorithms discerning ethical values or principles or making ethical judgments on their own.

The way forward

Evaluation professionals must use technology and artificial intelligence as a supporting tool to enhance the accuracy, efficiency and consistency of their evaluations. For example, AI can help evaluators analyse large amounts of data quickly and identify patterns that may not be immediately visible to the human eye. In addition, evaluators can use technology to automate certain aspects of the evaluation process, such as data entry or report generation, which can save time and reduce errors. However, it is important for evaluators to recognize the limitations of technology and AI and to continue to rely on their own judgment and expertise in making final decisions. Ultimately, technology and AI should be viewed as tools that can augment, but not replace, the essential role of human evaluators in making informed decisions. Human judgment, subjectivity, interaction, emotions and ethics are a step too far for automated decision-making.

Reflections

I would like to end this article by asking my fellow evaluation professionals for their thoughts and reflections on the exponential growth of technology and the introduction of artificial intelligence into our world and profession. What do you think about it? Do you think it will replace us one day or are you of the same opinion as me, that it is an essential tool to accompany us on our professional journey?

  • Hi Vicente,

    Thank you for your comment. Agreed! IT is a fantastic tool which we have to learn how to get the best of it.

    Indeed, AI development poses us the challenge to be technically updated to get the best of it. We must remain humble, keep on learning and embrace the change otherwise, we might be 'left behind'.

  • Hi Rami,

    I fully agree with you and with Jorge. I think that AI is a fantastic tool which we have to learn how to get the best of it in order to leave her do the "awkward work" and we concentrate on the interesting part of analysis, interpretation and development of conclusions and recomendations.

    We must always bear in mind that AI depends on the information it was feed with. So, the evaluator expert should always keep a close and critical eye on the results of AI work.

    These AI developments poses us the challenge to be technically updated to get the best of them, to understand how they work, and to really "add value" to the evaluation work. 

    Thanks for posing here such interesting topic, which most of us are thinking about!

  • Hi Jorge,

    Thank you for your comment. I do agree with your reflection and the unpredictability of the future. And yes, we are already using AI so much even without noticing while discovering new tools on a daily basis, some which are extremely beneficial to our profession.

    I also do think that sooner or later, algorithms will become more accurate, complete and able to solve more complex issues. However, they remain in a way a 'mathematical equation', thus unable to perform certain human tasks like judgement which takes me to the context point. It may be able to analyze 'basic' context in the future but some variables would also require human interaction. But as you said, no one knows what will happen and AI can always surprise us.

    Fully agree on the evaluation journey and ethical consideration. As you said 'Is this going to change in the (near) future?', only time will tell. This invention is here to stay and will definitely have an impact (or is already having) on the professional journey itself be it positive or negative.

  • Hi Rami, 

    These are interesting questions. I find it very difficult to predict and imagine how things will be in the future when the changes we see these days are so many, and coming so fast. We are already using so much AI, even without noticing, and there seem to be new tools almost every day. I am not so sure that AI will lack the ability to analyse context in the future. Why not? And I guess that the incomplete or inaccurate algorithms may be fixed. But whether we will be replaced? I guess this will depend very much on the professional journey itself. If we think of ethical values and principles, as you mention, then we'd need to consider what is it that we are evaluating, and why are we doing it. Is this going to change in the (near) future?