Presenter: Benedetto Lepori, Università della Svizzera italiana

Discussant: Mike Thelwall, The University of Sheffield

Abstract

It has recently been argued that ChatGPT might become a serious competitor to quantitative approaches to identify similar institutions for the purposes of comparing research performance. However, it is largely unknown how ChatGPT finds out ‘similar’ universities and whether results depend on the information provided by the user, and are stable over different queries. To address these questions, in this paper we resort to a sample of more than 1,000 universities included in the European Tertiary Education Register (ETER). We test different queries providing different levels of information on the focal university and we compare results we similarities computed from quantitative data in ETER. Preliminary results suggest that, indeed, ChatGPT is able to identify some good peers, but that results strongly depend on the querying strategy and on the universities’ characteristics.