Using comparison judgments to study representations

Abstract

Three projects are presented, all using comparison data to investigate representations. Processes of comparison are the focus here because of the strong links they create between the abstract representations much of cognition aims to study and unambiguous choice outcomes. The superficial similarities between these projects, that they all use browser based studies to reach relatively large numbers of people, and apply quantitative models to summarize and interpret the results, derive from two things: a common set of concerns with representation structure, and the use of comparison tasks to contrive situations where different representations predict different task behaviors. These basic ideas are applied across different domains to address current questions of representation and measurement in similarity and language. The first section compares two prominent theories of similarity judgment, transformational similarity and structural alignment, across three studies. The first of these constructs triad stimuli such that the two approaches make opposite predictions, the second measures similarity using an alternative measure of same-different discrimination speed, and a third applies both tasks to a common set of stimuli to clearly resolve their similarities and differences. The results show evidence of a misspecification in the apply rule of the transformational account current for geometric shapes, and also show that while same-different discrimination and deliberative comparison measures of similarity judgment are largely consistent, there are differences which appear to arise due to the different time constraints of the two tasks. The second section investigates a paradigm for testing the impact of transformation learning on similarity and categorization judgments. In this paradigm, a common set of test items follows two different training conditions, such that no test item is present in any training, and the status of each test item as a match, near match, or non-match to the training varies by condition. Responses to identical test items are compared across training conditions to expose the impact of transformation training on similarity and categorization judgment. Across multiple iterations of this basic design I show that the transformations are learned, and that transformation learning does impact similarity and categorization judgment. Change in similarity and categorization ratings due to training are largest in the easiest training conditions where transformations are presented explicitly to participants during training, and less pronounced when transformations are presented implicitly. Some generalization of learning is shown across related transformations, suggesting some similarity structure among transformations. The third section moves into empirical studies of syntax, comparing different ways of measuring sentence acceptability, the degree to which a sentence appears well-formed to a speaker of that language. This is related to the similarity work in the first and second sections through its use of Thurstonian modeling for structure discovery, which is capable of inferring acceptability scores for each sentence while also avoiding the need to present a rating scale of any kind to participants. This study complements existing work on the Type 1 and Type 2 error rates of the most common measurement techniques with its investigation of within and between participant test-retest reliability. The Likert task is found to be particularly effective. The results presented here show it has particularly good reliability properties and help empirically validate the common practice of interpreting averaged Likert ratings as a fine-grained measure of gradient acceptability.