New Teaching Tools

Gary King and Maya Sen discuss some of innovative ways that they use technology to teach Gary’s legendary class in statistics and social science at Harvard. Here is one bit

Instead of prohibiting smart phones in class, we require them …We then automatically deliver to their device a difficult conceptual question. We then give students a few minutes without discussion to reflect on the question and to indicate their answer on their device.

…Next, our system automatically puts students into groups of 2–5 [the system tells the students which other students to talk with and where to move in the classroom to find their group, AT]…We use an empirical approach to create the groups so that the conversation will be maximally productive. This is a system that is continually
updated, but for predictors we begin with data collected to characterize each student at the
start of the semester and add each student’s initial answer to the question just asked, their
answers to all previous CAPI questions and answers, their experience in the system, and
how productive previous CAPI discussions they participated in were. Finally, data from
thousands of other similar students in hundreds of other classrooms taking similar courses
can be used as well.

We then ask the students to try to persuade the other members of their group of the
veracity of their answers. Since social connections motivate, we often get highly animated
discussions…Since teaching teaches the teacher, the students trying to persuade their
classmates improves their understanding of the subject matter.

…We then deliver the same question to each student’s device again and have them answer
it. A minute or two later we project on the screen in front of the classroom a summary
of the answers before and after discussion, which gives them immediate feedback.

…When it works best — which, like in survey research, is primarily a function of us
asking sufficiently clear questions — the proportion of correct student answers increases
from 20% to more than 80%.

Note that this particular technology recommendation is a bit of plumping for Gary’s company with Eric Mazur, Learning Catalytics. Nevertheless, I think these types of technologies pair very well with online education. It’s a mistake to think that online education is just delivery of lectures–online lecture delivery is merely a leading example of how information technology is revolutionizing education.


Fairly recently (which seems surprising now) I was corresponding with an ed-tech skeptic. He was a teacher who thought "he'd seen videos before." I said no, as a computer science guy "I've seen this before, it's the start of a revolution." That revolution continues, and is probably in its early stages yet. It's too soon to pick winners, but soon enough to say the old order is crumbling. The revolutionaries would not be getting traction in a stable state. It should take, what, 20 years to fully shake out?

I'm content to pick losers ;)

Computer Science guys don't have a great track record at picking winners and losers in Technology.....

Does anyone?

I've used similar techniques in teaching my own classes. However, they are most definitely NOT suited for online courses. The key for these techniques to be effective is that the students have to be actively interacting with each other, engaging in freeform discussion and trying to convince each other. This simply cannot take place effectively if the students are not sitting together in the same location. The internet does not permit the same kind of interactivity and immediacy. You need to be able to see the other student rolling her eyes, watch her draw a quick sketch on a piece of paper that you immediately then add to, while two other students maintain a running dialogue on the whole thing. I would NEVER recommend this for a situation where the students are not physically present to each other.

Yes, online for the lectures. In class for working on problems.

Unfortunately separating the in-class problem-working from the lectures is less effective. Part of the goal of Mazur's approach is to interrupt the lecture frequently with interactive components, both on account of attention span but also to regularly move the audience from passive to active engagement. In addition, mixing the problem-working into the lecture gives the lecturer instant feedback on what the students are understanding and what they are not.

I am correspondingly skeptical of the benefits of online lecturing. Without the constant interactivity a lecture is no better than reading a book, and in many ways worse.

Many college lecturers are marginal at best. The so called constant interactivity occurs much less often than you suppose- and I attended long after the undergraduate years. The fact is all to often a book is as good as the lecture.

Have you used Skype? Online interaction can come much closer to in-person interaction than you think. All of the examples you mentioned are possible (replace the piece of paper with shared sketchpad software).

Its less the technology than the data--big data, about students past and present--that is used to foster dialogue..

Sounds like the sort of thing that leaves the planner more impressed with the experience than the participants are.

On these methods in general, it might be worth checking out "Improved Learning in a Large-Enrollment Physics Class," Deslauriers et al., Science, May 2011, vol. 332 no. 6031 pp. 862-864, (non-gated at ). Here's the abstract: "We compared the amounts of learning achieved using two different instructional approaches under controlled conditions. We measured the learning of a specific set of topics and objectives when taught by 3 hours of traditional lecture given by an experienced highly rated instructor and 3 hours of instruction given by a trained but inexperienced instructor using instruction based on research in cognitive psychology and physics education. The comparison was made between two large sections (N = 267 and N = 271) of an introductory undergraduate physics course. We found increased student attendance, higher engagement, and more than twice the learning in the section taught using research-based instruction."

Interesting---and believable. I'll have to see what methods were actually used. Traditional lecture can be either just lecture or lecture with interaction, which works better.

The study gets at one very important point: we need to find methods which work with mediocre teachers. That's especially true at the K-12 level, where we are going to have ed majors with union jobs doing the teaching. The solution isn't to complain about the teachers; it's to work the material at hand.

Well that's great if you're one of the lucky kids who gets into Harvard and can afford the tuition. What about the other 7 billion people on Earth?

And that's the real benefit of the online tools. Scale. Not some unattainable level of quality. 90% of the quality for 0.1% of the cost and 100% of the Earth's population. (In theory. Access to connectivity and computing hardware still required. But more than half of Earth's population already meets this minimum, and the growth in India, China, SE Asia and Africa is only improving the number)

You could get really close to this with a MOOC. A Google+ Hangout can support up to 10 people speaking together in real time. It's a pretty good system for social interaction of just this sort. As good as being in the same room? Obviously not. But it's a hell of a lot better than sitting in the library by yourself. I've used Google+ Hangouts to play pen-and-paper role-playing games, which isn't education obviously, but it's very social, absolutely depends on body language and vocal tone, and was something that before last year could only be done in person. Now it can be done online. There's even plugins that allow for collaborative sketchpads.

Assuming there's some 200,000 people enrolled in a MOOC at any one time (probably a very low number for a popular course like statistics, globally), the software could group people into discussion groups over a Hangout very similar to how it's described in this class. The AI would also be responsible for generating quizzes and judging the answers.

For those who are really stuck and need some personal tutoring though, there are online companies that are working on that problem too. Khan Academy (to pick one example; there are others) is working to pair struggling students with students who recently mastered the same material, to act as a tutor. The tutor gets credit in the system for helping the other student.

This is the future. Not because it's just as good as going to Harvard in person and getting taught by Gary King personally, but because it's actually an option for the other 99.9999%, and a much better option than otherwise exists.

I say it's too soon to place bets ... but this is where I'd put a small wager ;-)

Brock, your post is fine, but just a comment on the only-rich-kids-go-to-Harvard meme. Having been to Harvard on a full needs-based scholarship, I feel like I have to reiterate this: If your family earns less than $65,000 per year, you go to Harvard for free (or something like $2000/year parental contribution plus student contribution). Between $60,000 and $120,000 in income, you pay on a sliding scale from 0-10%, or no more than $12,000. For the "too rich to qualify for financial aid, not rich enough to pay full tuition" people, tuition costs 10% of income, from $120,000 to $180,000 in household income (soon to be reduced to $150,000). About 2/3 of students get financial aid, and it covers on average 2/3 of their tuition. Harvard is significantly more affordable than almost any mid- to upper-tier university in the US for those in the lower-middle income bracket. (

Students will be required to pay $12 per semester and must be required to own a smartphone and the data plan that goes with it. Obviously not a problem at Harvard.

But it's a good place to start.

Cengage might have been there first if they had further developed Aplia for new devices instead of just letting it stagnate.

In my opinion the most important thing that's missing from both online and traditional colleges is mentoring. Engineering has it right by offering co-op programs. All the other majors should require internships that teach students real-world things like how to give a presentation, persuasion, how to run a meeting etc.

Who are these people who think this is just about lectures by robots? It will free up instructor time to do one-on-one or small-group mentoring and instruction. We are going to need a lot of red Solo cups.

Comments for this post are closed