The availability of data coming from digital learning environments is creating the possibility to measure learning like never before. Analytics for Learning (A4L) represents a network of researchers exploring the measurement of learning strategies and behaviors within digital learning environments by supporting (1) the development of design patterns and worked examples of measures and (2) organizing capacity building activities on data intensive research techniques and operationalizing behavioral measures.
Cognitive abilities and skills, e.g., students’ content knowledge, represent an important but limited window into how and why students succeed in learning environments. A4L works with cutting-edge researchers using data from digital environments to provide new insights into measuring learning processes. Learning strategies and behaviors can be measured in digital learning environments through a variety of machine learning, statistical, and data management techniques. Along with diverse analytical techniques, effectively measuring learning behaviors requires attention to learning theory and merging multiple types of data (e.g., discourse data and click-stream data).
What is a Design Pattern?
Design patterns are narrative tools that can support the development and implementation of measures for learning strategies and behaviors. They present components of an argument needed to assess the validity of inferences made about students.
What is a worked example?
Worked examples illustrate how particular measures were developed and implemented. They illustrate elements of design patterns and describe the decision-making that went into the measurement development process. Thus, worked examples showcase the often implicit side of learning analytics and educational data mining. However, instead of simply providing tutorials on how to run particular statistical and machine learning models, worked examples are aligned with key components of design patterns, including task features, potential rubrics, and measurement models. Different online learning environments and digital technologies generate large amounts of data on students’ interactions within the learning system, making it possible to track and identify an array of student behaviors and constructs. The goal of the worked example is to make the process of using these data transparent and shareable.