Let the Data Inspire the Students

Project introduction and background information

As educators and scholars note, a considerable amount of self-regulation is required from students to be successful in a blended learning environment (Montgomery et al., 2019). Self-regulated learning (SRL) involves thoughts, feelings, actions, and adaptations systematically generated to attain a self-set goal in the process of learning (Zimmerman, 2000). Feedback from prior performances plays an important role in SRL by allowing students to reflect and make adjustments in their goal-oriented actions (Zimmerman, 2000). However, in traditional online learning environments, very few cues are available for students to make judgments on their learning and performance, placing a higher cognitive burden on students than face-to-face learning environments (Viberg et al., 2020).

In recent years, student-facing learning analytics-based dashboards (LADs) have emerged as a tool to intervene and support students, especially those with low SRL skills. LADs are potentially interactive, personalised, and monitoring displays of students’ learning data (e.g., learning management system logs), aiming to provide insight into their learning patterns and performance (Park & Jo, 2015). However, despite their potential, LAD design and evaluation still require improvement (Kaliisa et al., 2024). This project aims to improve our understanding and the efficacy of LADs.

Objective and expected outcomes

The primary aim of the project will be to improve student learning outcomes through the implementation of personalised, data-driven dashboards. By applying educational theories and leveraging Learning Analytics, the project will seek to address key learning challenges such as motivation, self-regulation, and time management. The dashboards will be designed to offer students actionable feedback, track their learning activities, and help them adjust their strategies for better outcomes.

The project will aim to achieve several objectives: first, to understand which elements to include in a dashboard from a user-centered and theoretical point of view; second, to design and test dashboards that are both theoretically sound and practically effective; and third, to evaluate their impact on cognitive, affective, and behavioural learning outcomes at the course level. The expected outcomes will include improved student motivation, better self-regulation, and enhanced learning performance, ultimately contributing to the TU/e 2030 educational vision of more personalised and ICT-supported learning.

Results and learnings

In the first study of this project, we tested student preferences for feedback elements and reference frames (used for comparison) in dashboards for different learning outcomes. We found that, on average, students perceived feedback that showed them how many modules and exam quizzes they completed as most useful, and comparisons to others as least useful. More importantly, our results emphasise that individual learning differences, especially in SRL skills and social comparison orientation, and the educational purpose of the dashboard can influence the usefulness that students perceive from LAD items, highlighting a clear need for personalised dashboards.

Future studies are still ongoing, and results will be announced in the future.

Recommendations

Based on the results of the first study, we recommend that feedback provided to students, especially via an LAD, should be personalised such that students with higher SRL skills are given more process-learning feedback, while students with lower SRL skills are given more product-learning feedback, and students with higher social comparison orientation are shown comparisons to external references (at least to improve their motivation and performance), while students with lower social comparison orientations are only shown comparisons to internal references (i.e., themselves).

Further studies to corroborate these results in real classroom environments are still ongoing, and further recommendations will be announced in the future.