You are here: Home / The IPN / Archive / Scientific Literacy of German Adults

Scientific Literacy of German Adults

November 26th, 2018

About the Conception and Application of Scientific Standards for the Adult Cohort

Dr. Linda I. Haschke

Science competencies are considered part of general basic education - not least because, in a modern industrialized society, they contribute to being able to live and act in a self-determined and reflective way. It is therefore important that adults are scientifically literate, both on an individual, social and global level.

Adults, for example, need science abilities and skills to deal with information disseminated through the media on an individual level. Studies show that 4.5% of global Internet searches include health-related aspects and that the information presented is taken to evaluate or even replace medical advice. Being scientifically literate can help select and evaluate qualitatively different information in order to make reasoned decisions.

To date, no science performance standards exist to determine what knowledge and skills adults must have to be considered scientifically literate.

The National Education Panel (NEPS) is the first longitudinal study of its kind in Germany, providing data on the development of competencies over the lifespan. From the data, empirical educational research hopes to conclude which educational decisions are related to educational success. As part of NEPS, we assessed the scientific literacy of German adults and developed science performance standards for adults linked to the NEPS Science test.

The NEPS Science Test for Adults

A science test was developed within the framework of the National Education Panel to assess the scientific literacy of adults.

- The quality of the items was tested in two pilot studies (in 2011 and 2012).

- In total, the main test consisted of 22 multiple-choice or multiple-true false items.

- The main survey took place between autumn 2012 and spring 2013.

- A total of 6,657 adults from all over Germany participated in the test.

- Almost half (50.5 %) of the participants were male and between the ages of 27 and 69.

- The study was conducted as a single test in a domestic environment.

- The selected test mode was the paper-pencil test.

- The test time was 25 minutes.

- The adults were then asked about their individual biographies in a computer-based interview.

Instead of presenting a continuous scale of performances, we defined specific performance levels. The boundaries between those levels are cut scores set by a committee of experts in a standard setting workshop.

The Item Descriptor (ID) Matching method was used to determine the science performance standards for adults and the associated cut scores within the NEPS Science Test. Within the ID Matching Method the panel experts matched the required knowledge and skills for each item to the knowledge and skills defined by specific performance level descriptors (PLDs). For example, if a person can solve an item requiring performance level 3 skills, the person will be located at that level. The item descriptor matching method allows determining which items are representative of the performance levels.

External experts prepared preliminary PLDs in preparation for the actual standard setting workshop. These preliminary PLDs were finalized by the panelists in the standard setting. The panelists worked through the Ordered Item Booklet (OIB) and matched the items to the PLDs. Each expert first made this assignment individually and received various feedback at the end of each round. This iterative procedure was repeated in three subsequent rounds. After completing round 3, the panelists set the final cut score to distinguish the performance levels Basic and Advanced.

In an additional fourth round, the experts decided how many items in total an adult had to master successfully to be considered scientifically literate. The corresponding theta-value was the cut score between the performance levels Below Basic and Basic. Consequently, when a person’s performance score was below this cut score and therefore did not reach the Basic level, the participant was labeled as scientifically illiterate according to the standard setting results.

As result of the IRT scaling, the wright map depicts the personal abilities and item difficulties that are located on the same scale (joint scale) are adapted to the PISA metric. The items above and including the 16th item require advanced skills. Since item 16 has a difficulty of 619 points, the participants had to achieve 619 points to reach the Advanced level. A total of 11.4 % of the sample achieved these results and can be labeled as advanced scientifically literate.

Adults with a performance score below 619 points were assigned to the Basic level. In the course of the standard setting, the experts decided that in order to be labeled scientifically literate, an adult has to master 11 out of 16 items. The corresponding theta-value is 457 points. Almost half of all participants (49.1% of the sample) reached the Basic level. In summary, 39.5% of German adults did not reach the Basic level and must thus be labeled scientifically illiterate.

Adults represent a particularly heterogeneous cohort because the adults already left institutional education and therefore have varying access to science learning opportunities. Therefore, we divided the sample into subgroups according to gender, age, migration background, and level of education.

On average, the sample achieved a personal ability score of 487 points with a standard deviation of 105 points. There were no significant differences in the means with regard to gender and migration background. However, larger differences can be observed for the characteristics age and level of education. For example, almost 75% of young adults ( aged 27-40) reach at least Basic level, while 60% of older adults ( aged 64-69) do not. We found the greatest differences within the level of educations. Almost 90% of adults with education on the tertiary level (e.g. university degree) reach Basic level, while around 60% of adults with education on primary level scored below Basic level.

The results of a logistic regression suggest that the level of education and the age of participants are the major predictors for being scientifically literate. Although gender and migration background are also significant determinants, their influence on a person’s scientific literacy is smaller.