The project "Development and implementation of a school-to-work transition concept for schools serving disadvantaged communities" implemented an intervention in schools in Bremen and Bremerhaven to reduce social and ethnic disparities in academic achievement. Several institutions cooperated in the scientific research of the intervention: the Institute for Educational Quality Improvement (IQB) at the Humboldt-Universität zu Berlin, the Friedrich-Alexander-Universität Erlangen-Nürnberg and the Max Planck Institute for Human Development in Berlin, as well as more recently the Leibniz Institute for Science and Mathematics Education in Kiel (IPN). Using this data, we address substantive and methodological issues in educational psychology and educational research.
The study employed a multi-cohort-multi-occasion design. All participating students attended lower secondary school. From 2004 to 2008, data from three grade-based groups of students were collected bi-yearly. Specifically, in each of the three waves, all students in grades 5, 7, and 9 were surveyed. Thus, a new cohort of fifth grade students was included in waves 2 and 3; the design comprises different cohorts which were surveyed one, two, or three times. Change over 4 years can be predicted for Cohort 1 (cf. Figure 1).
Curricularly valid standardized tests assessed achievement in mathematics, reading, and spelling. Anchor-items linked the different versions of the tests within domains. Therefore, longitudinal analyses can be conducted with test scores sharing a common metric. A notable feature of the study is its comprehensive consideration of social behavior: In addition to students' self-reports and teachers' and parents' other-reports, students' social behavior was assessed by a sociometric method, that is, by peer ratings within classes. These different perspectives on students' social behavior contribute to a valid description of prosocial as well as deviant behavior in lower secondary school. Moreover, the student questionnaires contained items relating to students' family background and their experiences with and attitudes toward school. A large proportion of students had an immigration background and a substantial number of students came from families with a relatively low socioeconomic status. Analyses can therefore systematically consider both these student characteristics.
Selected research questions
- Reciprocal effects between different abilities
Academic achievement is usually at least moderately correlated across different areas. This may be explained by shared underlying abilities such as intelligence or by transfer effects from one achievement area to the other. However, there is only little empirical evidence about the extent to which this relation is also reflected in the development of academic achievement. The design of EIKA makes it possible to conduct longitudinal analyses of these relations: specifically, reciprocal effects between achievement in different domains (German and mathematics) as well as reciprocal effects between achievement within a domain (reading comprehension and spelling) can be investigated.
- How important are cognitive and motivational student characteristics for achievement growth in lower secondary school?
Motivation is assumed to promote academic achievement beyond cognitive characteristics like psychometric intelligence. Controlling for psychometric intelligence, the empirical evidence for an incremental positive effect of motivation on academic achievement is, however, weak. The present design makes it possible to elaborately model the impact of motivational and cognitive student characteristics on achievement growth in lower secondary school.
- How does externalizing problem behavior impact student achievement growth?
Externalizing problem behavior may contribute to low academic achievement by reducing the time spent on academically-oriented behavior. In addition, disruptive student behavior in class may substantially impede effective teaching and may impair teacher-student relationships. This might be reflected in students' grades beyond their objectively lower achievement. Moreover, previous research has demonstrated that low academic achievement contributes to problem behavior. The present design enables longitudinal analyses of the complex interplay of externalizing problem behavior and different measures of academic achievement to be conducted.
- Effects of repeated test-taking on estimating growth in student achievement
A defining characteristic of longitudinal studies is the repeated administration of identical or equivalent measures of the constructs to be investigated. Research indicates that such repeated measurement of achievement may result in practice effects. Practice effects would systematically bias estimates of achievement growth and thus pose a threat to validity. In EIKA, identical tests were administered to cohorts that differ with regard to their previous exposure to such tests. Given the absence of differences in other important achievement-related characteristics, this design makes it possible to gauge the effects of repeated test-taking on estimates of achievement growth in different academic domains.
Prof. Dr. Olaf Köller
Prof. Dr. Gabriel Nagy
Prof. Dr. Jan Retelsdorf
Dr. Kerstin Schütte
Dr. Päivi Taskinen
Institute of Psychology, Kiel University:
Prof. Dr. Friederike Zimmermann