Achievement Growth at the end of lower secondary school: What do we learn from PISA?
A repeated measures study was implemented within the scope of PISA 2012. Students that were in 9th grade in 2012 and tested in mathematics, science and reading comprehension were tested again one year later (10th grade) using PISA items. The changes in achievement provide information on whether competencies measured by the PISA tests increase over the last year of secondary school year. Overall, the findings are disappointing.
PISA in Germany
Since 2002 the Programme for International Student Assessment (PISA) has been testing academic competencies in mathematics, science and reading of 15 year old students in many countries. PISA compares achievement among educational systems on an international level and informs participating countries how well prepared students are regarding the transition to vocational education and social integration. The IPN led the national project management of PISA in Germany in 2003 and 2006. Since 2010 the IPN has been collaborating with the Technical University of Munich (TUM), the German Institute for International Educational Research in Frankfurt (DIPF) and the Center for International Student Assessment (ZIB). The ZIB was responsible for PISA 2012 and 2015 and is currently preparing PISA 2018. As was the case in 2003, the ZIB had the opportunity to test 9th graders in 2012 and again in 2013 as 10th graders. This PISA 2012 Plus Study is one of the ZIB cooperative core projects. The IPN was responsible for analyzing the data from these repeated measurement study and was able to find out very interesting results regarding school effectiveness research. Among other things, school effectiveness research questions learning rates of one school year in core school subjects. US studies in mathematics and reading show achievement gains around 0 towards the end of secondary school.
How much do reading, mathematics and science competencies improve in the 10th grade?
PISA 2003 could provide an answer for mathematics and science: on the PISA scale which has an international average of 500 points, German students on average gained 25 points in mathematics, and 21 points in science. This corresponded to the so-called effect size of d = 0.33 in mathematics and d = 0.24 in science. The increases were statistically significant and substantial, but could be found only after students showing implausibly large negative achievement changes in the 10th grade had been excluded from the analysis. PISA 2012 Plus applied more sophisticates statistical methods than were used in 2003. No schools or students were excluded when achievement gains in mathematics, science and reading were negative during grade 10. Furthermore, differences between school types (academic vs. non-academic secondary school) were also of interest. In the analysis for mathematics 6,358 students were taken into account. For reading there were 4,954 and 4,930 for sciences. Figure 1 shows changes from 9th to 10th grade in relationship to the tested domain and the school type attended. The findings are indeed sobering. Students in non-academic tracks show no achievement gains and a substantial loss can be observed in science. The situation is very different for academic track students where a significant growth can be observed in mathematics and science. Admittedly, PISA tests do not claim curriculum validity or that learning opportunities are offered in 10th grade to further improve student competencies. At the same time many of the tested students in non-academic track schools are about to transition from a school setting into a vocational training setting. If one accepts that the competencies tested by PISA are essential for a successful career, it only stands to reason that the last year of non-academic track schooling shows no specific effect regarding these competencies. The ZIB plans further analyses to investigate which schools, individual, curricular and academic conditions result in students experiencing competency increases or losses. We hope to gain insights on how to optimize completion of lower secondary school in a way that further facilitates the transition to vocational training.