More on Online Education

by on November 21, 2012 at 12:46 pm in Economics, Education, Web/Tech | Permalink

At Cato Unbound I respond to some of the critics of my article Why Online Education Works. Here is one bit:

We do need more studies of offline, online, and blended education models, but the evidence that we do have is supportive of the online model. In 2009, The Department of Education conducted a meta-analysis and review of online learning studies and found:

  • Students in online conditions performed modestly better, on average, than those learning the same material through traditional face-to-face instruction.
  • Instruction combining online and face-to-face elements had a larger advantage relative to purely face-to-face instruction than did purely online instruction.
  • Effect sizes were larger for studies in which the online instruction was collaborative or instructor-directed than in those studies where online learners worked independently.
  • The effectiveness of online learning approaches appears quite broad across different content and learner types. Online learning appeared to be an effective option for both undergraduates (mean effect of +0.30, p < .001) and for graduate students and professionals (+0.10, p < .05) in a wide range of academic and professional studies.

Nicoli November 21, 2012 at 2:18 pm

“The effectiveness of online learning approaches appears quite broad across different content and learner types.”

Have any studies found that certain learning types do much better in an online environment? For example visual learners? Also, it seems like the quality and nature of the course might matter a lot too.

gwern November 21, 2012 at 11:47 pm

There’s some evidence, yes. My particular interest is Big Five personality traits, and there it’s suggested Conscientiousness affects the usefulness of education. There is some modest study support that offline classes require less Conscientiousness for a given level of performance than fully online classes; see my citations in http://www.gwern.net/Conscientiousness%20and%20online%20education (and the general discussion might be interesting).

Interestingly, the summary in OP is consistent with Conscientiousness as a factor: if we accept that online education should be better then a small increase in average scores is consistent with some students benefitting much more and some losing a little; face-to-face elements compensate for lack of Conscientiousness, giving you the best of both worlds for those low & high; and independent study hurts results for the exact same reason.

(I’m not sure what ‘content and learner types’ means, I haven’t read the paper yet. It could be that personality traits are included there.)

Bluto November 21, 2012 at 3:04 pm

An interesting control would be to measure face-to-face vs. online vs. mixed vs. student teaching self through reading books.

JP November 21, 2012 at 3:19 pm

I agree with Bluto, we need more rigorous testing.

After all, could we expect a certain self-selection bias in these results?

People who choose to pursue online learning (like Udacity or MRU) are probably fairly motivated individuals who might perform better in any circumstance. They are, after all, often choosing to pursue education for the sake of education.

Are we only measuring students who complete these courses? And if so, what is the retention rate of online courses versus traditional courses. If it is much lower, then the benefits might be suspect.

Stephanie Wright November 24, 2012 at 6:28 am

“Are we only measuring students who complete these courses? And if so, what is the retention rate of online courses versus traditional courses. If it is much lower, then the benefits might be suspect.”

Great question. At my college, we only measure completers, and we do have a much higher completion rate in both face-to-face courses and hybrid courses than online. We also don’t see online performing as well as these other methods except in very small pockets of the college where it appears to be an instructor effect (really fantastic OL delivery) rather than the method itself. This is consistent throughout each term we test and across all terms since OL delivery began.

Alex Tabarrok November 21, 2012 at 3:51 pm

All studies in the meta analysis are RCTs.

Stephanie Wright November 24, 2012 at 6:37 am

No, they weren’t. It would be impossible to randomly assign students to the delivery method of the courses they were taking across colleges and universities. They used controlled designs (to the best of their ability). I would argue that a case study shouldn’t even qualify.

“Use a controlled design (experimental or quasi-experimental). Design studies, exploratory studies or case studies that did not use a controlled research design were
excluded. For quasi-experimental designs, the analysis of the effects of the intervention had to include statistical controls for possible differences between the treatment and control groups in terms of prior achievement.” p.12 element 5

John H. November 22, 2012 at 1:33 am

There are many different measures of effect size; the D of Ed report appears to report Cohen’s d. But d=0.30 is a small effect size, and 0.10 is downright minuscule. D is in standard deviation units; to put it in perspective, d=0.10 is like an IQ difference of 1.5 points. Statistically significant, sure, but no real practical significance if you focus on learning. Cost effectiveness may be a different story.

gwern November 23, 2012 at 8:29 pm

Turns out to be Hedge’s g.

Stephanie Wright November 24, 2012 at 6:45 am

Agreed. The effect sizes are driven by n and virtually meaningless. For many schools, perhaps cost effectiveness may impact the utilization (e.g., no facilities usage), because a non-effect is better than a negative effect of online instruction. For us, there’s no net cost benefit of online delivery. So, I’m not sure how others view this.

I’ve been waiting on a report like this for some time from DoE. I’m very disappointed in the exclusion of 39% of the sample because of lack of statistical control. Student will take what they will take, and the data are still useful and important so long as comparison courses were taught in the term.

Andrew Woburn November 22, 2012 at 3:04 am

I often fantasized about on-line education in the 1960’s as I attended university lectures in theaters with 300 students. Why couldn’t I just stay home and watch this on television? How would it really be different?

Jaininder November 22, 2012 at 6:05 am

Online learning definitely provides a lot of advantages over the traditional teaching methods. With so many online learning platforms available, one can easily use them to enhance the learning process. I would like to mention here about WizIQ which is one such online learning platform that connects educators and students through its WizIQ Virtual Classroom technology. Available for individual teachers as well as organizations, WizIQ is a highly scalable solution for both synchronous teaching and asynchronous tutorials and assessments. Used for everything from teaching hybrid courses at major universities to offering guitar lessons one-on-one across thousands of miles, WizIQ is an incredibly flexible tool for delivering and enhancing any type of training or instruction.

Bill November 22, 2012 at 8:45 am

I had a hoot reading Alex’s post in Cato.

Because, like all tenured full time faculty, defending their status, identity, and paycheck (along with tenure), one of the REASONS Alex maintains that online is better is that college is inhabited by adjuncts, and other non-tenured faculty, and that online would expose students to tenured faculty:

From the Cato post:

“As Kevin Carey notes “I would find this more persuasive if I had not taken many traditional college courses myself.” Vaidhyanathan counters that:
The cartoon Tabarrok draws of higher education being a series of huge lectures taught by less-than-master professors does not accurately capture the diversity of higher education in the United States or the world.

Here Vaidhyanathan seems not to know the facts, for the facts are that a large majority of college teachers in the United States today are adjuncts, and they are neither tenured nor on the tenure track. At the undergraduate level a majority of courses are taught by adjuncts and graduate students, especially the large, introductory courses. Adjuncts are often teaching heavy course loads for low pay on a part-time basis, sometimes even cobbling jobs together from multiple universities. Part-time faculty alone make up nearly half of all college teachers,,,,,”

So, it comes down to: colleges are not hiring people in my Union and therefore the quality is not good..

Scott Gustafson November 24, 2012 at 10:40 pm

Over time, students will take courses from great teachers rather than great institutions just like people read individuals on the web rather than publications. Over time, it won’t matter which union you belong to.

Comments on this entry are closed.

Previous post:

Next post: