Till sidans topp

Sidansvarig: Webbredaktion
Sidan uppdaterades: 2012-09-11 15:12

Tipsa en vän
Utskriftsversion

IRT Models for Expert-Cod… - Göteborgs universitet Till startsida
Webbkarta
Till innehåll Läs mer om hur kakor används på gu.se

IRT Models for Expert-Coded Panel Data

Artikel i vetenskaplig tidskrift
Författare Kyle L. Marquardt
D. Pemstein
Publicerad i Political Analysis
Volym 26
Nummer/häfte 4
Sidor 431-456
ISSN 1047-1987
Publiceringsår 2018
Publicerad vid Statsvetenskapliga institutionen
Sidor 431-456
Språk en
Länkar dx.doi.org/10.1017/pan.2018.28
Ämnesord Bayesian methods, expert opinion, latent variables, IRT models, cross-national data, anchoring vignettes, elections, reliability, agreement, democracy, quality, opinion, Government & Law
Ämneskategorier Statsvetenskap

Sammanfattning

Data sets quantifying phenomena of social-scientific interest often use multiple experts to code latent concepts. While it remains standard practice to report the average score across experts, experts likely vary in both their expertise and their interpretation of question scales. As a result, the mean may be an inaccurate statistic. Item-response theory (IRT) models provide an intuitive method for taking these forms of expert disagreement into account when aggregating ordinal ratings produced by experts, but they have rarely been applied to cross-national expert-coded panel data. We investigate the utility of IRT models for aggregating expert-coded data by comparing the performance of various IRT models to the standard practice of reporting average expert codes, using both data from the V-Dem data set and ecologically motivated simulated data. We find that IRT approaches outperform simple averages when experts vary in reliability and exhibit differential item functioning (DIF). IRT models are also generally robust even in the absence of simulated DIF or varying expert reliability. Our findings suggest that producers of cross-national data sets should adopt IRT techniques to aggregate expert-coded data measuring latent concepts.

Sidansvarig: Webbredaktion|Sidan uppdaterades: 2012-09-11
Dela:

På Göteborgs universitet använder vi kakor (cookies) för att webbplatsen ska fungera på ett bra sätt för dig. Genom att surfa vidare godkänner du att vi använder kakor.  Vad är kakor?