Can machines write opinion pieces like humans? A corpus-based analysis of modal verbs and personal pronouns in author-written and GPT4 generated op-eds

Abstract

The rapid development of AI-generated text, in common usage and model performance, raises concerns about how convincingly it can mimic human writing, making it increasingly difficult to distinguish AI text. This study addresses the issue by examining opinion editorials (op-eds), a first-person genre that allows authors to express opinions and persuade readers. Two corpora were compiled: 100 human-written op-eds from UK newspapers and 100 GPT4 op-eds, generated for this study, with texts of 500 to 1000 words. Corpus-assisted discourse analysis was employed, guided by Systemic Functional Linguistics, in the selection of analysis features. Frequency and concordance searches of personal pronouns and modal verbs were conducted and followed by discourse analysis on common pronoun + modal verb bigrams. The analysis revealed systematic differences. Human authors used I and you more than twice as often as GPT4, creating a more personal and narrative style. Whilst GPT4 relied heavily on inclusive first-person plural, especially we in repetitive, rhetorical patterns. Modal verbs also diverged: humans favoured will and would, whereas GPT4 more frequently employed can, might, and must. A novel finding was GPT4’s standardised sequencing of bigrams such as can + we interrogatives and repeated end-loaded we must, indicating standardised logogenetic structures which were absent in the human texts. In the human corpus, idiolectal patterns were revealed, as well as a greater register variation of op-ed writing. Overall, the study highlights the importance of register and genre awareness in distinguishing AI from human-authored texts and reaffirms the value of function words (personal pronouns and modal verbs) in authorship analysis and AI classification.