Temporal analytics for understanding students’ study behaviours in digital educational environments

Abstract

The growth of using technologies in education has motivated the development of research to study and promote students’ academic success in the educational environments applying them. Digital educational settings provide a high volume of data for this analytical purpose and enable researches to easily collect and analyse data from students’ interactions within the system (audit trails) as they proceed towards their study goals. This has motivated the development of Learning Analytics (LA) approaches in these educational settings that offer innovative applications of analytics methods to understand and promote students’ study behaviours. One of the main challenges of LA approaches is making connections between students’ data traces and educational assumptions which is necessary to ensure the improvement of education and needs interdisciplinary knowledge. In addition, in digital environments, students’ data are available at different levels (i.e., fine-grain to coarse-grain) and can be structured in varied ways (e.g., aggregated, temporal). This data requires appropriate formulation to be able to reveal information regarding specific aspects of students’ study behaviours. In this matter, the main focus of LA research is on representing students’ study behaviour based on the aggregated measures of their task level interactions within digital environments that helped to identify various patterns of learning processes associated with learning outcomes. However, this suffers from some limitations, mainly due to neglecting the time dimension that could better reveal the effect of processes students used during studying. In addition, investigation of students’ behaviour at specific context levels such as session is understudied by research that may reveal novel insight into particular aspects of students’ study processes. This thesis provides an understanding of students’ study behaviours by considering specific levels of conceptualizing students’ data and the level at which their behaviours can be structured.

In the first part of this thesis, a temporal analysis based on clustering and statistical tests is performed in the context of a Massive Open Online Course (MOOCs) where students’ study behaviour is investigated at session level; that is, dedicated blocks of time in which learners complete single or multiple contiguous learning tasks without interruption. The concept of “session” has rarely been explicitly examined in relation to learning outcome in online learning. Creating and managing sessions when learning online has been associated with an important factor that is associated with students’ time management strategy that subsequently can impact their academic outcome. The result of this study provides insight into varied ways that students organize and prioritise their time in terms of sessions when learning in a MOOC and how these behaviours impact students’ academic outcome.

In the second part, a study is conducted in the context of two offerings of a MOOC, where the impact of sequential representations of students’ task level behaviour on their learning outcome is investigated. This study considers assessment task outcomes as a proxy for learning outcome rather than students’ final achievement that could provide more insight regarding students’ progress over time. For this purpose, temporal and non-temporal prediction models are used to show how the sequential nature of learners’ task level behaviour in a MOOC is more informative (predictive) of their assessment outcome rather than aggregated measures examined in most studies. Additionally, it provides insight into variations in behavioural sequences of high and low achieving students when preparing for assessments using a sequential pattern mining approach. The results show that it is possible to successfully predict students’ readiness for assessment tasks, particularly if the sequential aspects of students’ behaviour are represented in the model. Moreover, the results reveal some behavioural patterns reflecting specific learning strategies that may be more effective in promoting learning.

In the third part of this thesis, a study is performed in the context of a digital word processing software to examine the importance of the temporal nature of students’ writing behaviour on their writing outcome. It helps to understand how particular aspects of the writing process at specific moments of writing influence the writing outcome. This view is understudied in writing research using students’ audit trails (i.e., keystrokes). For this purpose, a temporal approach is proposed combining classification and local feature interpretation as methods. The results reveal the importance of temporal analysis when studying students’ writing behaviour. Findings also reveal that the influence of specific writing behaviours on writing quality is likely determined in combination with other writing characteristics, that emphasise the necessity of using models that capture and take the interrelationship between features into account.

In summary, this work contributes to the learning analytics research by raising awareness regarding the need to account for various levels of conceptualizing data and different dimensions when studying learners’ behaviour. Various stakeholders could take benefits from the knowledge discovered in this research to improve learners’ study behaviours. In particular, educators can identify which study behaviours require support - and (most importantly) when – so they can select relevant interventions to include in their courses.