I came across these quality metrics from Coursera in some reading today and thought it was interesting.
Components of quality metrics definitions
Engagement (Completion rate): The proportion of completion-eligible learners who complete courses and items
Satisfaction (Rate of 5-star reviews): The proportion of star ratings – given by course completers – that are a perfect five stars. This metric captures more variability than average star ratings.
Skill development (Average score Delta): The average increase in skill scores, demonstrated in graded assignments, and projects in the course.
Career outcomes (Rate of career benefit): The proportion of completers, responding to our survey, who report receiving career benefits from the course.
Coursera
In the paper Great online learning outcomes happen by design, Coursera states that ‘completion rates among most populations of learners are substantially higher than 50% and can be far higher in courses that adhere to Coursera pedagogy best practises‘. For student satisfaction, they state, ‘The average star rating across courses on Coursera is 4.7 out of 5 stars’. Career outcomes are at 73% of the responding students claiming a positive job-related outcome.
The engagement and satisfaction numbers are far lower than what we would see in a good university course. This is before you take into acocunt that satisfaction scores at universities are usually taken mid-year and include students that will not complete, whereas Coursera only asks completers. However, it is worth noting that these numbers are a dramatic improvement from the early days of MOOCs when 15% completion rates were not unheard of and where satisfaction was low. What is more impressive is that Coursera is operating at scale, with 70 million students, making up nearly 200 million online enrollments on over 4000 courses provided by around 200 different universities.
These improvements are down to investment from Coursera and other MOOCs in delivery models and quality. It is worth watching these large providers with their massive data sets, intense focus on the student experience, and lower costs. Perhaps the MOOC might deliver the promise that people initially hoped they would provide.
Read the full paper, Great online learning outcomes happen by design, on Coursera’s website.
SamuelJTanner 