Effect of Using Generative Artificial Intelligence in Different Roles in a Course on Computer Programming

ABSTRACT

Generative artificial intelligence tools have become widely available in the past 3 years. Generative artificial intelligence is increasingly being adopted across diverse domains including the teaching of computer programming. In this study, four experiments were conducted to examine the effectiveness of generative AI in different roles within a course on programming. The students who used AI as a tutor reported higher motivation, programming self‐efficacy and computational thinking of students, but the use of AI does not improve examination performance. When used as a problem solver, AI improves performance on programming tasks, but this improvement does not transfer to subsequent tasks completed without AI assistance. Additionally, AI can be utilized to evaluate and grade programming assignments. Although AI‐generated grades are largely consistent with those assigned by a human expert, students exhibit low levels of trust in their accuracy. Furthermore, AI serves effectively as a design partner in complex programming tasks by proposing alternative solution strategies and facilitating discussion of trade‐offs. In conclusion, generative artificial intelligence represents a promising technology with significant potential to enrich the teaching‐learning process in programming education. However, its benefits can be maximized when applied to open‐ended problems supported by well‐structured and descriptive prompts.