QRM Conference 2022
The school of Quantitative Research Methods in Education held its annual conference on June 13-14 in Gothenburg
Keynote presentation
Methodological issues in research on effects of schools and teachers on student outcomes
Prof. Dr. Sigrid Blömeke, Director and Professor, University of Oslo, Centre for Educational Measurement (CEMO)
This talk will draw on three recent studies on effects of schools and teachers on student outcomes conducted with national and international data. The focus will be on the respective methodological challenges involved, in particular the theoretical modelling of relations and its transformation into measurement models. Each topic will be assigned 30–40 minutes including questions.
The first topic is the modelling of mediation processes and is based on Blömeke et al. (2022a). Opening up the black box: Teacher competence, instructional quality, and students’ learning progress (in Learning and Instruction).
The second topic is the modelling of moderation processes and is based on Blömeke et al. (2022b). The role of intelligence and self-concept for teachers’ competence (in Journal of Intelligence).
The third topic is the generalizability of results, for example in case of studies that include several countries, and is based on Blömeke et al. (2021). School innovativeness is associated with enhanced teacher collaboration, innovative classroom practices, and job satisfaction (in Journal of Educational Psychology)
Workshops
Analyzing large-scale assessment data using R–package RALSA
Dr. Plamen Mirazchiyski, International Educational Research and Evaluation Institute (INERI), Slovenia
June 14
The workshop will be divided into two parts. The first part will make a short introduction to ILSAs' complex sampling and assessment designs using TIMSS as an example. A specific stress will be put on the consequences of these designs on the analysis of studies' data using examples.
The second part will introduce the analysis software (RALSA) for analyzing data from ILSAs. The data preparation and analysis features will be explained. The overview will also include demonstrations on using both the command line and the graphical user interface. The remaining time will be devoted to guided hands-on training using the graphical user interface of the package. Through these guided exercises all features will be demonstrated. If the time permits, at the end of the workshop assignments will be given to the participants to complete on their own with assistance from the instructor.
The workshop will use data from Nordic countries participating in the IEA’s Trends in International Mathematics and Science study (TIMSS) 2019 (grades 4 and 9). Regardless of the study and the cycle, RALSA always applies the correct estimation techniques, given the study design and implementation.
By the end of the workshop the participants are expected to have gained the following:
- Knowledge, understanding and appreciation on the ILSAs design and methodology, as well as the statistical complexities and issues for analyzing their data;
- Knowledge and understanding on the computational routines used in ILSAs; and
- Skills to analyze data using an R package, tailored for ILSAs design.
Target group and previous knowledge requirements
The workshop is intended both for analysts who do not have yet the knowledge and experience using ILSAs’ data, as well as for researchers with more experience.
Previous experience with R is welcome, but not a prerequisite, RALSA has intuitive and easy to use syntax, as well as graphical user interface. Working knowledge on basic statistics is required. All materials for the workshop, including the software, will be provided free of charge. The participants will need a computer to install the software and perform the sample analyses.
An Introduction to Propensity Score Matching for Causal Inference
Dr. Isa Steinmann, Centre for Educational Measurement at the University of Oslo (CEMO), Norway
15 June 2022
This workshop will introduce how propensity score matching methods can be used to address cause and effect questions in empirical research.
Causal research questions aim to isolate effects of a treatment, independent of other effects and differences between the treated and untreated. The most straightforward way to answer causal research questions is to conduct randomized trials like in pharmaceutical studies, for instance. In many fields including education, randomized trials are however very difficult to conduct, due to practical, ethical, or financial reasons, among others. Therefore, educational research often has to rely on observational data, that is, information that stems from pure observations of educational processes and outcomes without an interference of the researchers. Under specific circumstances, it is however possible to isolate causal effects anyway. This workshop will focus on propensity score matching methods both from a theoretical and applied perspective. Example studies will help to understand the assumptions and prerequisites behind this method and to evaluate its scope critically.
This workshop will first revisit Rubin’s potential outcome framework, a model that formalizes cause and effect questions, and the issue of selection bias. Second, different propensity score matching and balance check methods will be introduced. Lastly, the estimation of effects and the central advantages of propensity score matching over other methods will be discussed.
Target group and previous knowledge requirements
“The workshop is an introduction to the topic of propensity score matching and is primarily targeted at researchers who have not yet worked with the method themselves. For more experienced participants, further literature will be provided. Basic knowledge of causal inference is welcome but not a prerequisite.
The R demonstrations are for illustrative purposes and will be designed so that participants without R experience can follow as well. Laptops and software are not necessary.”
