“Whither, badges?” or “wither, badges!”: a metastudy of badges in Computer Science education to clarify effects, significance and influence

Abstract

Gamification is becoming a widely used technique in an at- tempt to improve engagement and effectiveness of learning, with virtual rewards (often in the form of badges) frequently forming a core part of this structure. But how do badges actually work? Recent studies in gamification have clearly identified that while we can say this approach works, any analysis of the success stories quickly confirms that context is essential in both the system we gamify and the users that take part. In the absence of context, a badge is obviously meaning and valueless, but studies in badge use generally refer to success without focusing on the underlying process and, in some cases, report significance without effect size, report the badge without the process or do not sufficiently isolate the variables to allow us to address what the influence of the badge is. In this paper, we classify the existing work to identify the existing research where badges have been evaluated in a way that appears significant, with properly established control groups, isolation of factors and longitudi- nal considerations, and with an exposure of the underlying process to determine if what we are seeing is a successful process in action that has a badge at the end, rather than a process that has been made successful through badge use. A badge system can be a proxy for scaffolding and an individ- ual badge can give focus much in the way we would expect in cognitive apprenticeship but both of these outcomes have strong implications that there is a definite limit to how many badges we award, the way in which they are contextualised and the training framework in which they are presented.