Valid measurement of laboratory learning experience quality from the student perspective

Abstract

Since the early 2000s, improvement of the student learning experience in university level laboratory activities in Australia has been sought by the Advancing Science by Enhancing Learning in the Laboratory (ASELL) project. The nation-wide project has made use of the ASELL Student Learning Experience (ASLE) survey to gather data and draw conclusions regarding student perspectives of their learning experiences, using trends observed in the data to inform pedagogy. Analyses of rating scale response format items on the ASLE survey have typically involved an integer value scoring system applied to the response categories. The appropriateness of such integer scoring techniques and the subsequent application of parametric statistical methods to ordered categorical data in this way is contested in statistical literature, which raises questions regarding the validity of ASELL project conclusions drawn in the past. In this thesis, Rasch measurement is applied to a data set of ASLE survey responses, using the true interval scale measures gained to test the validity of the scoring techniques and parametric methods more typically applied to ASLE data. The role of student biases in survey response and ‘objectivity’ of any measures associated with learning experience quality are explored, yielding quantitative models of the student perception of laboratory learning experiences. The thesis culminates in the use of factor analysis to develop a Linear Logistic Test Model for a data set of over 9000 completed ASLE surveys, explaining the responses received as linear combinations of a small number of major factors in the student laboratory learning experience. The model is used to draw pedagogical conclusions from the ASLE survey data set uninfluenced by limitations of the integer scoring techniques usually applied. The work has major implications for valid interpretation of ASLE survey data received both in the past and in future, suggesting that whilst integer scoring methods may be amenable to parametric statistics, the conflation of student dependent and student independent factors limits the generality of any conclusions drawn. Student independent measures obtained from Rasch analysis, however, reveal that the perceived relative quality of a laboratory exercise is largely consistent through the student population sampled. The Linear Logistic Test Model generated reveals a wide range of connections between different facets of the laboratory learning experience and this general perceived learning experience quality, informing effective science pedagogy. Pedagogical conclusions include strong connections between group work and understanding of theoretical content, the advantages of data analysis and individual work in development of more technical or practical skills, evidence for the importance of structuring activities appropriate to the ability level of the students, as well as ways to generate student interest and foster perceptions of a positive overall laboratory learning experience. A need for compromise between teaching objectives and learner preferences is highlighted, noting that the “best” way to design a laboratory activity largely depends on the intended purpose of the exercise.