The Cognitive Reflection Test: familiarity and predictive power in professionals

Abstract

The CRT is an increasingly well-known and used test of bias susceptibility. While alternatives are being developed, the original remains in widespread use and this has led to its becoming increasingly familiar to psychology students (Stieger & Reips, 2016), resulting in inflated scores. Extending this work, we measure the effect of prior exposure to the CRT in a sample of oil industry professionals. These engineers and geoscientists completed the CRT, seven bias tasks and rated their familiarity with all of these. Key results were that: familiarity increased CRT scores but tended not to reduce bias susceptibility; and industry personnel, even without prior CRT exposure, scored very highly on the CRT - greatly reducing its predictive power. Conclusions are that the standard CRT is not a useful tool for assessing bias susceptibility in highly numerate professionals – and doubly so when they have previously been exposed.