Linking Recent and Older IEA Studies on Mathematics and Science

Abstract

The purpose of this thesis was to develop procedures that allow researchers to make reasonable comparisons of grade-eight mathematics and science achievement and motivation scales over a long time period, despite changes to the instruments, populations, and procedures between administrations. The data were selected from international large-scale assessments administered by the International Association for the Evaluation of Educational Achievement. Student data were used from the Trends in International Mathematics and Science Study (TIMSS) and its four predecessors conducted before 1995. The assessments have targeted slightly different populations (13-year-olds, 14-year-olds, and eighth-grade students), and the constructs changed somewhat across administrations. The thesis, therefore, aimed to: 1) evaluate the degree of comparability across these assessments; 2) link the cognitive test results onto the TIMSS reporting scale with the use of item response theory (IRT) modeling; 3) explore the feasibility of linking the motivational scales in these assessments with different approaches in the IRT and structural equation modeling frameworks. Despite the assessments being carried out since the 1960s, a high level of stability in the inferences and cognitive constructs of the assessments was found. The measurement of the motivational constructs was found to be less stable. Overall, the results indicated that the comparability of the scales has improved over time. Different linking approaches were explored, and they yielded similar results in the country-level trend descriptions of achievement and motivation. The linking techniques applied in this thesis may be beneficial for linking other large-scale assessments, in which changes have occurred between administrations. Moreover, with the scales established in this thesis, it is possible to examine long-term changes in educational systems. Powerful statistical approaches may be applied to these system-level longitudinal data to address causal research questions.