To the top

Page Manager: Webmaster
Last update: 9/11/2012 3:13 PM

Tell a friend about this page
Print version

Exploring rater variabili… - University of Gothenburg, Sweden Till startsida
To content Read more about how we use cookies on

Exploring rater variability in a foreign language speaking test

Conference contribution
Authors Linda Borger
Published in The twelfth Annual Conference of EALTA, Copenhagen, Denmark 28th - 31st of May, 2015
Publication year 2015
Published at Department of Education and Special Education
Language en
Subject categories Languages and Literature, English language, Didactics, Pedagogy


This exploratory study aims to provide insight into the issue of rater variability in a paired speaking test, part of a mandatory Swedish national test of English. Six authentic conversations were rated by (1) a group of Swedish teachers of English (n = 17), using national performance standards, and (2) a group of external CEFR raters (n = 14), using scales from the CEFR, the latter to enable a tentative comparison between the Swedish language syllabus and the CEFR. Quantitative and qualitative methods were employed to analyse scores and raters’ justifications of these scores in the form of concurrent written comments. As expected, findings showed some variability of ratings as well as differences in consistency. Analyses of the written comments, using NVivo 10 software, indicate a wide array of features taken into account in raters’ holistic rating decisions, however with test takers’ linguistic and pragmatic competences, and interaction strategies the most salient. Raters also seemed to heed the same features, indicating considerable agreement regarding the construct. Furthermore, a tentative comparison between written comments and scores shows that raters notice fairly similar features across proficiency levels, but in some cases evaluate them differently. The results will be discussed in relation to issues of design, testing model and participating rater groups. Finally, the results will be considered with regard to the current policy of national assessment in Sweden, where teachers are responsible for rating their own students’ tests as well as for awarding their final subject grades.

Page Manager: Webmaster|Last update: 9/11/2012

The University of Gothenburg uses cookies to provide you with the best possible user experience. By continuing on this website, you approve of our use of cookies.  What are cookies?