The learning analytics tool pilots are a key component of UBC’s Learning Analytics Project. The pilots are aimed at identifying faculty members’ questions, exploring actionable sources of data, and trialing tools that empower students, faculty members, and departments with analytics to address the questions they have identified.
Are you a UBC instructor? Several of the tools outlined below can be used in your courses now. You can also learn more about other ways you can get involved with learning analytics at UBC.
Pilot Topic: Analytics for Teaching
An instructor-facing dashboard for Canvas that displays student activity
Pilot description
UBC instructors who teach large blended and online courses have requested tools that will help them keep better track of students’ online activity and performance. This pilot seeks to identify possible enhancements to Canvas’s activity dashboard that could provide instructors with improved insight into student activity.
Pilot status: In progress
A customized instructor-facing dashboard for Canvas is currently under development. The project team is currently offering Canvas Analytics workshops, which provide an introduction to Canvas’s built-in learning analytics and quiz statistics tools and provide an overview of ways of using these tools to encourage student engagement, support struggling students, and refine teaching practices.
VizIT: An instructor-facing dashboard for edX that shows student activity
VizIT helps instructors stay informed about learner activity in edX. This chart highlights the problems students in a course had the most difficulty with.
Pilot description
VizIT is a tool for edX that provides instructors with a comprehensive view of students’ online course activity data. This tool can be used with both edX Edge-based UBC courses and edX-based UBC MOOCs (massive open online courses). In this pilot, UBC faculty members using edX and the Learning Analytics Working Group are seeking to evaluate the effectiveness of the VizIT course dashboard by providing instructors with data about the types and levels of student engagement with course material.
Pilot status: Completed
VizIT has been pilot tested in 36 courses as part of the Learning Analytics Project. The project team's evaluation report for VizIT will be posted here.
OnTask: Providing timely, personalized, and actionable feedback to learners
Pilot description
Giving timely, relevant feedback to students is a critical aspect of good teaching and supporting student success. This pilot gives instructors the opportunity to try OnTask, a web-based tool that uses data from online learning tools such as Canvas to allow instructors to define progress indicators, track students’ progress, and give targeted feedback based on the metrics they set for their course. OnTask includes a dashboard showing individual and aggregated student progress metrics and allows instructors to easily send a custom message to subsets of students who have yet to meet performance indicators, giving them a personalized reminder of what they need to do to catch up.
Pilot status: In progress
OnTask has been piloted by UBC instructors in six courses, and in general, faculty and students have provided very positive feedback about the impact of OnTask on teaching and learning. The project team continues to offer OnTask workshops for faculty and to contribute improvements and bug reports to this open source project.
Please contact the Learning Technology Hub if you would like to use OnTask in your course.
Strategies and data requirements for effective student team formation
Pilot description
This project investigates ways of visualizing student group characteristics and supporting team formation based on student profile information. Ultimately, investigators are interested in understanding how team composition affects team performance and how to optimize group formation for different learning outcomes. This project may also lead to the development of a tool that suggests teams based on instructors’ preferred team composition strategies.
Pilot status: In progress
The primary area of interest for the instructors involved with this pilot — research into effective grouping strategies — was determined to be out of scope for the Learning Analytics Project.
Threadz: Network analysis and visualization of Canvas discussions
Pilot description
This network visualization shows a typical line/node graph that connects users together. It can be used to quickly discern individuals or groups that are isolated or that are a driving hub in the discussion forum.
It can be a challenge for instructors to effectively monitor online class discussions. This pilot explores the Threadz network tool, which integrates with Canvas to provide instructors with visualizations of what's happening in course discussions and reliable metrics for measuring student engagement. Developed by Eastern Washington University, this tool enables instructors to easily access data about who is engaged, whose responses are going unanswered, and whether students are achieving the engagement goals that were set for the course. Threadz can also alert instructors to potential issues early on, such as learner isolation and instructor-centric discussions, which can provide insights that help make discussions more helpful for all learners.
Pilot status: In progress
Threadz has been piloted by eight UBC instructors in twelve courses. In two of those courses, the instructor requested that Threadz be set up to also allow students to access the tool's visualizations. The project team continues to offer Threadz workshops for instructors.
To use Threadz in your course, please submit a request to the Learning Technology Hub that includes your course number.
Pilot Topic: Analytics for Learning
A student-facing dashboard for Canvas that displays anonymous, comparative activity data
Pilot description
Providing students with data about their own online course activity and engagement with learning materials, compared with the activity of anonymized peers, has been demonstrated to support self-monitoring and self-directed learning. The goal of this pilot is to field test a student-facing activity tool embedded in Canvas that provides students with up-to-date data on their own activity versus that of their peers.
Pilot status: In progress
The project team is actively collaborating with the University of Michigan on this pilot and is targeting a pilot launch in one UBC course in summer 2019.
A student-facing dashboard for edX that provides personalised feedback based on previously successful learners' activity
Pilot description
Compared with students who are taking an in-person course, students in a fully online course lack information about how and when their peers who are successful in that course are engaging with learning materials. To bridge this gap, this pilot seeks to explore a personalized feedback system for students in edX that facilitates anonymized social comparison with previously successful learners. The goal is to offer an interactive, student-facing dashboard that displays visualizations of multiple behavioral indicators. This tool could potentially be implemented for all UBC’s edX MOOCs.
Pilot status: Discontinued
This pilot has been discontinued due to low student interest in edX dashboards. The project team is focusing efforts on areas with higher impact, including a student-facing Canvas dashboard.
Pilot Topic: Analytics for Program Planning and Advising
Data-driven curriculum analysis
Pilot description
This pilot explores statistical and visual techniques for analysing and displaying historic student performance data. Areas of focus for this pilot include course difficulty, the impact of a particular course on overall academic performance, curriculum coherence, dropout paths, and the impact of course load on student performance. This project could lead to the development of models that aid curriculum committees and department heads in decision making regarding course and curriculum offerings. Student advisors could use findings obtained through these analyses when advising students about course difficulty and course load. This information could also be shared with course instructors to provide insight into students’ experiences in their courses or programs.
Pilot status: In progress
Currently developing course content sequencing visualizations.
Predicting success in UBC program specializations to inform admissions criteria
Pilot description
Due to high demand, student admissions into specializations at UBC can be competitive and are often based on factors such as students’ overall GPA. This pilot will explore methods for determining the predictors of success in program specializations. The findings from this pilot could lead to new knowledge that supports better, evidence-based admissions standards that take into account relevant criteria that more effectively predict student success. This pilot may also explore predictors of success in specific courses—for example, by considering helpful or unnecessary prerequisites.
Pilot status: Reassigned to Planning and Institutional Research Office (PAIR)
In consultation with the UBC Planning and Institutional Research Office (PAIR), it was determined this pilot more closely aligns with their operational mandate and that PAIR will take on this project.
Registration dashboard showing real-time enrolments
Pilot description
Departments and programs face a serious challenge when planning how many course sections to offer and how many instructors to appoint or assign, especially for required courses. Patterns of student registration are complex and highly dynamic, and it’s difficult for departments to obtain useful data or to project future enrolments. Underestimation results in overfull classes—and students being unable to register in necessary courses—while overestimation can result in half-full sections and inefficient use of resources. This pilot seeks to develop predictive models, based on past enrolment, with the goal of projecting final enrolment numbers, including final student count after add/drop cut-off dates. Ultimately, this project could help programs and departments allocate resources more effectively and support better course planning, contributing to student satisfaction and success by ensuring more students are able to get into required and preferred courses.
Pilot status: Discontinued
This pilot has been determined to be out of scope for the Learning Analytics Project.
Student success dashboards for Canvas
Pilot description
At UBC, the Student Information System (SIS) collects data on enrolment pathways, grades, failure rates, graduation rates, and other variables. This pilot explores ways of using this data to develop faculty- or department-level diagnostic dashboards that provide an overview of whether and how students may be struggling. These types of data could ultimately be made available to all department heads and senior leadership, supporting evidence-based decision making that improves student learning and success.
Pilot status: On hold
The Student Success Dashboards pilot is currently on hold, pending staff capacity and discussion with the UBC Planning and Institutional Research Office (PAIR).
Visualizing and reporting on student enrolment pathways to inform curriculum review
Pilot description
Departments and curriculum committees have increasingly asked for help in understanding how students move through their courses and programs. Requests have been made for overviews that will give departments a ‘big picture’ understanding of common enrolment pathways and sequences or pathway differences between groups (e.g., majors vs. non-majors). Other requests are focused and seek to answer specific questions about the impact of a particular course or the effect of changing program requirements or course offerings. This pilot could lead to insights into actual patterns of student enrolments, which could be beneficial for instructors, departments, and curriculum committees. These insights could also be used to advise students who are considering various possible paths through their degrees.
Pilot status: In progress
A solution for the Faculty of Science is currently in development.
Call for Participation
Each of the pilots listed above contributes to the Learning Analytics Project's iterative exploration process. Our understanding of the UBC community's interests and the ongoing development of UBC's data policies and procedures continue to inform the Project's focus and the progress on each pilot.
Faculty members who are interested in participating in the pilots outlined above or who would like more information are invited to meet with the project team and to check out the list of learning analytics pilots tools that are currently available to UBC instructors.
About the Learning Analytics Pilots
In September 2017, UBC launched twelve exploratory learning analytics pilots as part of the Learning Analytics Project. The primary goals for the pilots are to expand on the UBC community's existing expertise and interest, build traction in learning analytics, facilitate the development of prototypes, and gain insight into uncharted areas to guide UBC’s overall approach to learning analytics.
The pilots were chosen based on faculty submissions to a call for proposals issued to faculty in June 2017. Factors taken into account when reviewing submissions included the potential benefits to students; feasibility in terms of data availability and project complexity; and achieving a balanced mix of pilots meeting different needs for students, faculty, and administrators. To support the highest impact on teaching and learning, the Project prioritized proposals that sought answers to questions that are answerable, actionable, and common.