What Bayesian modelling can tell us about statistical learning: what it requires and why it works

Abstract

This paper explores the why and what of statistical learning from a computational modelling perspective. We suggest that Bayesian techniques can be useful for understanding what kinds of learners and assumptions are necessary for successful statistical learning. The inferences that can be made by a learner are driven by both the units that such learning operates over and the levels of abstraction it includes. Other assumptions made by the learner have non-trivial affects as well, including assumptions about the process in the world generating the data, as well as whether it is more reasonable to make inferences on the basis of types, tokens, or a mixture of the two. Finally, of course, any learner must incorporate –whether explicitly or implicitly – certain assumptions in the form of their prior biases and the nature of the hypotheses they can represent and consider. We discuss the ways in which these assumptions might drive what is learned, and how Bayesian modelling can be a useful way of exploring these issues.