Mapping employable skills in higher education curriculum using LLMs
Date
2024
Authors
Zamecnik, A.
Barthakur, A.
Wang, H.
Dawson, S.
Editors
Mello, R.F.
Rummel, N.
Jivet, I.
Pishtari, G.
Valiente, J.A.R.
Rummel, N.
Jivet, I.
Pishtari, G.
Valiente, J.A.R.
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Book chapter
Citation
Event/exhibition information: 19th European Conference on Technology Enhanced Learning, EC-TEL, Krems, 16/09/2024-20/09/2024
Source details - Title: Technology Enhanced Learning for Inclusive and Equitable Quality Education, 2024 / Mello, R.F., Rummel, N., Jivet, I., Pishtari, G., Valiente, J.A.R. (ed./s), vol.15160 LNCS, pp.18-32
Statement of Responsibility
Conference Name
Abstract
The rapid advancement in large language models (LLMs) is reshaping education globally. LLMs, with their natural language processing capabilities, offer opportunities to analyse curriculum mapping and can aid instructional designers with structuring courses to ensure that the curriculum adheres to government requirements and that learners are offered employable skills. Despite the potential of LLM to serve as an agent for quality assurance, its capability to check if learners will receive the expected skills from a higher education context based on the curriculum description is yet to be explored. This paper proposes a novel intervention, leveraging LLMs to initiate and accelerate the curriculum mapping process within higher education. More specifically, our study compared expert reviewers’ assessments with annotations from LLMs for an initial teacher education program in higher education. We found that LLMs provide comparable evaluations to experts regarding the skills learners are expected to acquire from each of the twenty six courses. The subsequent analysis revealed mixed Kappa scores for the skills under review, indicating strengths and limitations in LLM application for curriculum quality assurance. By exploring the potential synergies between experts’ mappings and LLM annotations, this work highlights exciting avenues for leveraging AI in curriculum mapping and quality assurance practices for researchers and practitioners alike.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
Copyright 2024 The Author(s), under exclusive license to Springer Nature