A Comparative Study of the Pragmatic Capabilities of ChatGPT and Humans in Compliment Responses in English

ABSTRACT

Generative artificial intelligence (GenAI), exemplified by AI‐driven chatbots like ChatGPT, has stimulated considerable interest in the field of applied linguistics. However, important questions regarding ChatGPT's affordances for second language (L2) learning and teaching remain under‐investigated. For instance, what are ChatGPT's inherent linguistic attributes (e.g., pragmatic capabilities)? Furthermore, to what extent does ChatGPT reflect human pragmatic competence so that this AI‐driven chatbot can be leveraged as a viable tool for L2 pragmatics acquisition? To fill this gap, the current study compared ChatGPT's pragmatic capability in responding to compliments in English with that of humans. Compliment responses were collected from ChatGPT‐3.5 and 230 human participants (129 first‐language English speakers, 101 advanced L2 English speakers) through synchronous text‐based conversation tasks. The results revealed that both ChatGPT and the human participants tended to use the pragmatically conventional strategy of acceptance (e.g., “Thank you.”) when responding to compliments in English. Nevertheless, nuanced distinctions emerged in the diversity of sub‐strategies employed within the main strategy of acceptance, with ChatGPT's compliment responses demonstrating less pragmatic diversity than those of the human participants. ChatGPT's pragmatically conventional compliment responses in English indicate its affordances for L2 pragmatics acquisition, whereas the observed distinctions suggest the need to critically integrate ChatGPT into L2 pragmatics learning and teaching. The implications of this study are discussed for language instructors and AI engineers.