Why do you hate learning? Why do you hate the future?

Trying not to use the term “drop out” in the MOOC world is now a thing, with the publication of proper actual research about it.

Well, up to a point. Said proper, actual, research was presented in a session on “The Future” at the WWW14 conference in Seoul, Korea. I’ve had a hunt around the usual places and am unable to offer you a link to Ashton Anderson’s slides, or to any blog or twitter commentary on the presentation.

One of the most amazing parts of the paper is the acknowledgements – it gives one a real sense of how valuable this kind of research currently is, and how much is invested in it. I make no apologies for quoting it in full below.

“We thank Andrew Ng, Daphne Koller, Pamela Fox and Norian Caporale-Berkowitz at Coursera for their help with implementing the badge system and for sharing the data with. Supported in part by a Google PhD Fellowship, a Simons Investigator Award, and Alfred P Sloan Fellowship, a Google Research Grant, ARO MURI, DARPA SMISC, PayPal, Docomo, Boeing, Allyes, Volkswagen, Intel and NSF grants […]”

That’s some serious research funding firepower, not to mention the near-priceless opportunity to experiment “live” with Coursera learners.

Man reviews academic paper

I'm not going to look here at the paper in any great depth - suffice it to say that I am not convinced that the engagement styles section really adds anything to our understanding - the overwhelming majority of "learners" in all courses do not participate or participate at very low levels, and that very high (>80% of marks) achievers actually watch less lecturers than is average.  For those that have read MOOC participation research such as the work of Weller and Jordan, these findings should come as little surprise.

The second section, around adding a system of "badges",  looks at the use of various configurations of badge incentive systems to get "learners" to make more forum posts. This is another one of those interventions hampered by experimental design - there was the chance to do a nice randomised controlled trial, but the investigators chose to use an earlier iteration of the course as a control rather than using a null group of students in the same iteration. Because the course itself may have changed, and the cohort for the last iteration would have different characteristics and is itself twice the size of earlier runs (tables 1 and 2) , it's not really safe to make comparisons or to base changes on one particular intervention.

So anyway - the focus is on different ways of presenting the badge system (basically, you get a shiny object for key learning activity like making a post or starting a thread, based primarily on measures of "interest" from other students. As you get badges for expressing "interest" as well, this is likely a gameable measure as, if badges are desirable to students it is in everyone's, er, interests to vote as many posts as "interesting" as possible.

Ashton et al decided to look at the total number of forum actions (not meaningful engagement, per se) as a measure of success, which is a teeny bit disengenous. Apparently if you tell "learners" how to get a badge by clicking on things, they are more likely to click on the things than if you don't. Because SCIENCE.

So, you are probably asking yourselves, what has this not especially interesting but well-funded bit of research got to do with drop-outs? Absolutely zero, it would appear to me – but this is the finding (in the loosest possible sense) that the Chronicle takes from the paper.

Coursera (and FutureLearn, and EdX…) have spent the last year or so trying to downplay the importance of the average 85% drop-out rates that Katy Jordan so expertly curates, and do so in concert with the “student experience” wing of academia which assert that someone who has done none of the work associated with learning can’t be said not to have done any learning.  Liyanagunawardena [2014] characterises this latter tendency in asking academics what they thought “drop out” meant and then asking them to reflect on it in a semi-structured interview untill they behaved like academics and saw both sides of the argument.

For everyone else, who thinks that perhaps all of these claims of enhanced learner experience and the attractiveness of video&quiz&forum pedagogy need to be backed up by evidence of improved engagement, this pincer movement yields a very uncomfortable experience.

Either we talk about drop-outs because we hate learners, or because we hate the future.

The most common metaphor I hear is this one:

@dkernohan MOOC like a newspaper – read what interests you. No need to read the lot.

— Alastair Creelman (@alacre) June 23, 2014

This is a great argument to advocate not reading the entire internet.  However, a MOOC is nothing like a newspaper. It is a curated and structured set of materials leading towards a particular goal. To use my favourite metaphor, it is like a menu at a restaurant.

Even with the price set at zero, the majority of users are not merely trying the odd forkful or just eating the tiramisu. They are turning their nose up at the whole meal. The overwhelming weight of published research supports this.

So why would anyone be arguing otherwise? Let’s don our tinfoil hats and speculate!

Maybe MOOCs are like newspapers, but in another way. Maybe content creation costs are going up but there is no sign of a sustainable business model.  So like the rest of the rapidly collapsing internet content industry, they turn to the the one group of people more starry-eyed and unrealistic than people who think they can make money out of internet content – advertisers.

To persuade advertisers to give you money in return for putting flashing things on your pages you need to give them class A controlled substances evidence that your pages are being looked at by people. Ideally those who have money, the coveted demographics that overwhelmingly sign up for MOOCs.

So it is in the interest of the content creator to emphasise the level of engagement that it has, and their likelihood of engagement with messages from their platform of choice. And clearly “drop out” is not an attractive term here. Drop-out implies that someone is paying no attention to any part of your website or mailings. “Drop out” does not delight advertisers.

If you’ve got big numbers like the number of people who sign up for a MOOC then you want to use them. Stands to reason. Headline writers like them, advertisers LOVE them.

Any redefinition you can do to get rid of those words has the potential to be immensely valuable. So that is why, if I had a MOOC platform or an interest in cheap online education, I’d totally be funding and supporting research that redefined “drop-out”.