DOI: 10.12688/f1000research.190843.1 ISSN: 2046-1402

Assessing the accuracy and patient acceptability of AI vs human translation of patient-facing clinical trial documentation: A mixed methods comparison study

Liz Allaway, Haimeng Ren, Renata Tagg, Kate Borthwick, Chris Wignall, Danielle Jeffrey, Daniel Griffiths, Gareth Griffiths
Show PDF Cite
Background Diversity in research participation is essential to understand the generalisability of trial findings and reduce health inequalities. Inability to understand English is an exclusion criterion for many trials, where recruitment and consent processes rely on written materials. Professional translation is resource heavy while AI offers quick, free translation but evidence for its accuracy and accessibility is lacking, particularly where complex information needs to be conveyed clearly. We compared AI to professional human translations to assess whether these could be a viable alternative in clinical research. Methods A selection of trial documents were translated into three of the most common non-English or Welsh languages in the UK (Polish, Punjabi, Chinese) by professional human translators and widely available AI applications, and reverse translated back to English. We conducted a dual-layered evaluation: i) End-User Acceptability: Native speakers evaluated translated documents, providing feedback through surveys and interviews; ii) Linguistic Coding Analysis: Independent reviewers compared reverse-translations against original English texts with findings validated by a sixth reviewer. Reviewers were blinded to which were AI or professional translations. Results While human translation is generally more accurate than AI, error margins are small and AI translations were found to be an acceptable alternative by native language reviewers. AI applications varied in performance, with Claude outperforming other applications. Translations into European language (Polish) were generally more accurate, though there was little variation across document type. Errors in medical and regulatory terminology showed the need for human validation across all translation pathways. Conclusions Translations by AI applications are generally accurate enough as a support mechanism to clinical trial recruitment, but careful human oversight is required, particularly for consent processes. Further evaluation of AI translations in real-world trial settings is needed to provide robust evidence of their acceptability to trial participants and develop best practice guidance for trial teams.

More from our Archive