DOI: 10.3390/jintelligence14100232 ISSN: 2079-3200

Examining Computer-Science Students’ Metacognitive Beliefs and Reported Practices in Verifying AI-Generated Code: A Concurrent Mixed-Methods Study

Shanshan Li, Jingqi Tang, Zhen Qiang, Zhuo Wang

Generative artificial intelligence (GenAI) accelerates programming, but its educational value depends on whether students check generated code before accepting it. This concurrent mixed-methods study examined which beliefs accompany such checking among computer-science students at one Chinese university. Survey responses from 294 students were analyzed with an exploratory structural equation model treating three practices as separate outcomes: checking code line by line before use, practicing without AI, and declining in advance to delegate. Robustness checks included ordinal estimation, acquiescence modeling, split-half replication, and multiverse analysis; eleven students were interviewed. Verification-dispensability belief, endorsing reasons for treating further checking as unnecessary, was the only construct negatively associated with all three practices in every robustness check. Other beliefs were narrower or inert: perceived AI literacy accompanied line-by-line checking alone, generic metacognitive self-regulation none. Delegation breadth, the number of activities handed to GenAI, was unrelated to all three: volume of delegation said little about checking. Interviews exposed a mismatch between self-reported categories and actual workflows: students allocated checking by task purpose, accepted process-level understanding, and delegated verification to other AI models, a practice unmeasured by the survey. The data are cross-sectional and self-reported, so these are associations requiring behavioral and longitudinal confirmation.