Current studentsExisting staffContact us
 

A common misapplication of statistical inference: nuisance control with null-hypothesis significance tests

Simple item page
dc.contributor.authorSassenhagen, J.
dc.contributor.authorAlday, P.M.
dc.date.issued2016
dc.description.abstractExperimental research on behavior and cognition frequently rests on stimulus or subject selection where not all characteristics can be fully controlled, even when attempting strict matching. For example, when contrasting patients to controls, variables such as intelligence or socioeconomic status are often correlated with patient status. Similarly, when presenting word stimuli, variables such as word frequency are often correlated with primary variables of interest. One procedure very commonly employed to control for such nuisance effects is conducting inferential tests on confounding stimulus or subject characteristics. For example, if word length is not significantly different for two stimulus sets, they are considered as matched for word length. Such a test has high error rates and is conceptually misguided. It reflects a common misunderstanding of statistical tests: interpreting significance not to refer to inference about a particular population parameter, but about 1. the sample in question, 2. the practical relevance of a sample difference (so that a nonsignificant test is taken to indicate evidence for the absence of relevant differences). We show inferential testing for assessing nuisance effects to be inappropriate both pragmatically and philosophically, present a survey showing its high prevalence, and briefly discuss an alternative in the form of regression including nuisance variables.
dc.identifier.citationBrain and Language, 2016; 162:42-45
dc.identifier.doi10.1016/j.bandl.2016.08.001
dc.identifier.issn1090-2155
dc.identifier.urihttps://hdl.handle.net/11541.2/121730
dc.language.isoen
dc.publisherElsevier
dc.relation.fundingGerman Research Foundation BO 2471/3-2
dc.relation.fundingEuropean Research Council 617891
dc.rightsCopyright 2016 Elsevier Inc. All rights reserved. Access Condition Notes: Postprint available 1 December 2017
dc.source.urihttps://doi.org/10.1016/j.bandl.2016.08.001
dc.subjectcontrolled study
dc.subjecthuman
dc.subjecthuman experiment
dc.subjectnull hypothesis
dc.subjectprevalence
dc.subjectstatistical model
dc.subjectstatistical significance
dc.subjectstimulus
dc.titleA common misapplication of statistical inference: nuisance control with null-hypothesis significance tests
dc.typeJournal article
pubs.publication-statusPublished
ror.mmsid9916083996501831

Files

Collections