Understanding the impact of observer effects on student affect
Date
2024
Authors
Liu, X.
Gurung, A.
Baker, R.S.
Barany, A.
Editors
Kim, A.
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Book chapter
Citation
Event/exhibition information: 6th International Conference, ICQE 2024, Philadelphia, US, 03/11/2024-07/11/2024
Source details - Title: Advances in Quantitative Ethnography, 2024 / Kim, A. (ed./s), vol.2279 CCIS, pp.79-94
Statement of Responsibility
Conference Name
Abstract
The measurement of affect presents a challenge for researchers in quantitative ethnography and related communities, as each of the several possible methods for obtaining ground truth have downsides and often disagree with each other. One common method is field observations; some accounts have raised concerns about observer effects, but further research is needed to understand how much observer effect is present and exactly how observer effects manifest. In this study, we attempt to quantify and detail observer effects in classroom data collected using the Baker Rodrigo Ocumpaugh Monitoring Protocol (BROMP). We leveraged prediction models developed in prior work to assess students’ affect and disengaged behavior, during periods where observers were present or not present. Statistical analyses revealed differences in both affective states and behaviors on days when field observations were conducted. When observers were present, students showed increases in concentration and decreases in frustration, off-task behavior, and gaming the system. Findings suggest that the collection of observation data changes the proportion of behavior and affect observed, which should be taken into account when designing, implementing, and analyzing observation-based research. We discuss implications of this finding for quantitative ethnography, when observation data is systematized, visualized, and compared. Our findings do not suggest that the individual observations of specific affective states or behaviors are invalid but offer insights for QE scholars to consider when collecting and analyzing observation data.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
Copyright 2024 Springer