Paper and Digital Reading Assessments. Exploring aspects of validity using PIRLS and ePIRLS

Abstract

The shift from paper-based to digital assessment formats in education raises fundamental questions about the comparability of reading scores across modes and over time. International large-scale assessments increasingly rely on digital delivery while continuing to report trends on a common scale, making it essential to establish whether paper-based and digital reading assessments support equivalent score interpretations and valid trend comparisons across cycles. This dissertation examines the comparability and validity of paper-based and digital reading assessments using data from PIRLS 2016 and its digital extension, ePIRLS. The thesis consists of an integrative essay and three empirical studies, each addressing a distinct aspect of comparability and validity: score comparability across modes, subgroup differences, and the predictive value of reading scores for later educational outcomes. Results show that paper-based and digital reading assessments share a strong common core of reading comprehension, demonstrating substantial construct overlap across modes. At the same time, systematic mode-related variation is observed, indicating that the two assessment formats are not fully equivalent despite their strong association. Longitudinal analyses further demonstrate that both paper-based and digital reading scores meaningfully predict later academic outcomes, supporting their predictive validity, while small but systematic differences in predictive strength suggest that digital reading captures partly distinct aspects of educationally relevant skills. Taken together, the findings show that paper-based and digital reading assessment scores are similar but not interchangeable. Validity in digital reading assessments should therefore be understood as conditional on assessment mode, student population, and context, rather than assumed a priori. The dissertation concludes that ongoing empirical validation is essential to ensure the quality and interpretability of trend comparisons in international reading assessments as formats and literacy practices continue to evolve.