Doing Digital Multimodal Composing with Generative Artificial Intelligence: Towards an Integrated Critical Digital Literacies Model

The widespread availability of generative artificial intelligence tools has had a dramatic effect on language education, transforming how we think about and go about writing. This includes ‘multimodal composing’, which combines moving image, soundtrack, speech, and on-screen writing in digital communication. In this article, we focus on the potentials and pitfalls of generative artificial intelligence in a pedagogy of digital multimodal composing. The article aims to explore how the affordances and constraints of generative artificial intelligence tools powerfully shape the multimodal composing process in a way that enables or constrains individual agency, creativity, and criticality; and conceptualize a model of best practice for the use of generative artificial intelligence in digital multimodal composing. As language and literacy educators, we report the findings of a practitioner inquiry study into our own pedagogically motivated digital multimodal composing practices, using generative artificial intelligence tools to collaboratively complete a digital multimodal composing project (a digital video scientific documentary) designed for second language learners of English for science. We recorded this process by keeping copies of input and output as well as maintaining regular research journals. The analysis identified five themes relevant to agency, creativity and criticality in digital multimodal composing, namely: (a) feelings; (b) relating to generative artificial intelligence: the iterative, collaborative loop; (c) ideological positioning; (d) modal interactions: process and product; and (e) voice and authenticity. We suggest an integrated critical digital literacies model to inform digital multimodal composing pedagogy that engages with generative artificial intelligence at three key levels, extending previous work by highlighting the role of affect: a macro level focusing on feelings, materialities, and ideology; a meso level focused on the design of a collaborative, iterative loop in generative artificial intelligence digital multimodal composing workflow; and a micro level focused on interaction with generative artificial intelligence.