Better Textbooks Raise Student Achievement Cheaply and Effectively

by on March 4, 2016 at 12:07 pm in Books, Education | Permalink

Teacher quality can be measured using value-added student achievement scores. Value-added scores, however, let us do much more. We can measure the value not only of different teachers but of different teaching methods. Thus, value-added scores and more generally big data are tools not just to weed out low-quality teachers but to raise the quality of all teachers.

At Brookings Thomas Kane reports on new research evaluating textbooks.

We matched each teacher to the students they were teaching and assembled data on students’ demographic characteristics, performance on prior state tests, and the averages of such characteristics for the peers in their classroom. We also estimated each teacher’s impact on student performance in the prior school year (2013-14) to use as a control. (We wanted to account for the fact that more effective teachers may choose to use particular textbooks.) After controlling for the measures of student, peer, and teacher influences above, we estimated the variance in student outcomes on the new assessments associated with the textbook used.

The textbook effects were substantial, especially in math. In 4th and 5th grade math classrooms, we estimated that a standard deviation in textbook effectiveness was equivalent to .10 standard deviations in achievement at the student level. That means that if all schools could be persuaded to switch to one of the top quartile textbooks, student achievement would rise overall by roughly .127 student-level standard deviations or an average of 3.6 percentile points. Although it might sound small, such a boost in the average teacher’s effectiveness would be larger than the improvement the typical teacher experiences in their first three years on the job, as they are just learning to teach.

What makes this research especially important is that textbooks have no unions and it’s easy to replace one textbook with a better textbook. Moreover:

An annual report on the effectiveness of textbooks would transform the market, by providing publishers and software developers with a stronger incentive to compete on quality.

1 hamilton March 4, 2016 at 12:18 pm

The specifics of the textbook finding:

“We found no statistically significant difference in achievement for students using three of the textbooks. However, two textbooks were statistically significantly related to students’ performance—one positively and one negatively. The average student using GO Math! (Houghton Mifflin Harcourt) as their primary textbook scored 0.1 standard deviations higher (p < 0.05) than similar students using other textbooks or no textbook at all. In contrast, the average student using another textbook scored 0.15 standard deviations lower (p < 0.05) on the new math assessments. (We are not releasing the name of the second textbook because we could not confirm which edition teachers were using.) Both estimates are sizable, implying that textbook choice is a high-stakes decision."

2 Handle March 4, 2016 at 12:21 pm

They can assert it all the want, but the estimates sure don’t seem very sizable.

3 anon March 4, 2016 at 12:27 pm

Perhaps not, but it is something to measure and iterate upon. Perhaps the gen 10 textbooks will be 1 sd.

4 Dane March 4, 2016 at 12:52 pm

Agreed. The effects are not sizeable. Also, there are very likely to go away completely if student ability were properly controlled for.

One basic assumption of the value-added metric is that controlling students on past attainment is sufficient to control for differences in ability between students. Behavioral genetics says this is not so, as heritability estimates for ability increase with age. The claims made by Chetty, et al, are wildly overstated. Repeated VA measurements show less validity when applied to teachers than when applied to students.

This is a good summary:

5 anon March 4, 2016 at 12:57 pm

On the other hand More Money Really Does Make Schools Better

Economists Julien Lafortune, Jesse Rothstein, and Diane Schanzenbach have a new paper looking at court orders, much like the earlier study did — but this paper looks at episodes after 1990. The authors find that in places where courts told public schools to spend more on low-income students, those students’ achievement levels began to gradually but steadily increase. As far as the authors can tell, the improvement in low-income student performance continues even several years after the funding increase.

This is exactly the kind of thing we would not try, if we just presumed it was impossible.

6 MC March 4, 2016 at 5:26 pm

“There also was a large increase in single parenthood, which probably means those kids get less education at home as well. …But it does seem that certain types of spending, especially on non-instructional items like capital improvements and support services, make a significant contribution.”

So increasing the salaries of teachers and administrators does not really improve student performance. Instead, we are supposed to pay social workers to pick up the slack caused by a lot of women believing that it’s acceptable to choose to become single mothers. Thanks, feminists!

7 So Much For Subtlety March 4, 2016 at 5:45 pm

We have tried it. It did not work.

After 1990? Did they check to see whether they were measuring the effect of gentrification?

8 anon March 4, 2016 at 6:25 pm

In response to any study it is important to say yeah maybe it doesn’t matter. Shows sophistication.

9 mpledger March 4, 2016 at 6:38 pm

If the father of her child walks away, what should a pregnant women do?

10 So Much For Subtlety March 4, 2016 at 6:39 pm

I take it that means no, they did not. So they are replacing Black youth with Asian and White children. Better results follow. No surprise there really.

And of course if you have an outlier, if you have a study that produces a surprising result, you should look at it. If they are carefully picking a time frame that seems to produce the result, you should question it. Basic research skills.

11 MC March 4, 2016 at 8:13 pm

@mpledger: She should get married first so that the father of her child is also her husband. A husband is much less likely to bolt. But defining deviancy down means that a 40% illegitimacy rate is now part of a “woman’s right to choose” that we are expected to subsidize.

12 Nathan W March 4, 2016 at 9:50 pm

MC – Is it better for a child to grow up in a house where it is “normal” and “OK” to beat up mommy or is it better for that child to grow up with one parent?

Also, mpledger’s point is key. Women should not be blamed for deadbeat dads.

Sure, a healthy two-parent family is much better than one parent. However, many relationships are sufficiently bad that it may be better to be raised by one parent, and for the parent to seek other endeavours to expose the children to role models from the other sex.

13 MC March 4, 2016 at 10:48 pm

Wife beaters are not the primary cause of a 40% illegitimacy rate. And I refuse to valorize women who put their children at a major disadvantage in life because they feel that a husband-father is an optional fashion accessory (and no, mere role models are not an adequate substitute).

14 White Knight March 4, 2016 at 11:44 pm

“Also, mpledger’s point is key. Women should not be blamed for deadbeat dads.”

Yes, how misogynist to blame the victim. We can’t expect women to be accountable for their actions or decisions. We should take money from other people and give it to single mothers for choosing to get knocked up by deadbeat dads.

15 mpledger March 5, 2016 at 12:17 am

The women are accountable for their actions – they are the ones who stayed and bring the children up at considerable cost to themselves.

It’s the parents who do nothing that are the ones you should be slamming for not being accountable for their actions.

16 So Much For Subtlety March 5, 2016 at 1:28 am

mpledger March 4, 2016 at 6:38 pm

If the father of her child walks away, what should a pregnant women do?

She should not have children with low life scum balls in the first place. No one forced her to. She had plenty of choice in men. She chose the bad boy. Why is she surprised? Why should everyone else have to pay?

12 Nathan W March 4, 2016 at 9:50 pm

Is it better for a child to grow up in a house where it is “normal” and “OK” to beat up mommy or is it better for that child to grow up with one parent?

The evidence is in – it is better for the child to grow up with wife beating.

Also, mpledger’s point is key. Women should not be blamed for deadbeat dads.

Why not? Did you get her pregnant? Did you force her to go out with the sexy bad boy instead of the boring reliable geek? Did you ruin her relationship through nagging? How is it anyone else’s fault?

However, many relationships are sufficiently bad that it may be better to be raised by one parent, and for the parent to seek other endeavours to expose the children to role models from the other sex.

So the standard orthodoxy goes. But no, it does not appear to be so.

15 mpledger March 5, 2016 at 12:17 am

The women are accountable for their actions – they are the ones who stayed and bring the children up at considerable cost to themselves.

At considerable cost to the community. They are the ones paying.

It’s the parents who do nothing that are the ones you should be slamming for not being accountable for their actions.

There is no shortage of blame to go around.

17 Nathan W March 5, 2016 at 4:20 am

“The evidence is in – it is better for the child to grow up with wife beating”

Saying so doesn’t make it true.

“Did you get her pregnant? … How is it anyone else’s fault?”

Ummmm, takes two to tango. The original comment placed all the blame squarely on the women, as though we should prefer that they stay in abusive relationships and that women are the ones to hold responsible for absent dads, as opposed to observing that the dads are absent.

18 So Much For Subtlety March 5, 2016 at 7:26 am

Nathan W March 5, 2016 at 4:20 am

Saying so doesn’t make it true. …. Ummmm, takes two to tango. The original comment placed all the blame squarely on the women, as though we should prefer that they stay in abusive relationships and that women are the ones to hold responsible for absent dads, as opposed to observing that the dads are absent.

I like how you are so certain you are right that you can’t even grasp an alternative view. Denying it doesn’t make it untrue either. There are communities where there is some level of domestic abuse but rarely divorce – Asian immigrant communities for instance. Their children do just fine. It turns out divorce is even worse than we used to think. It is the single worst thing you can do to affect a child’s life chances. So we should prefer they stay in abusive relationships.

It does take two to tango. Specifically it takes a woman to say yes to sex. I don’t tell underclass women to do this. I assume you do not. It is a decision they make on their own. The consequences are their responsibility, not everyone else’s. Women are responsible for the inevitable and foreseeable consequences of their own decisions. I didn’t force them to do it. They wanted to. And then they could not make it work. Who else shares even the smallest part of the blame except perhaps to some extent the only people we do blame – the fathers?

19 Roger Sweeny March 5, 2016 at 9:22 am

If you aren’t sure male will stick around and he says, “I can’t use a condom; it doesn’t feel natural,” then you “just say no.”

20 Nathan W March 5, 2016 at 1:05 pm

SMFS – it does in fact seem to me that, systemically, we are rather quick to put all the blame on the fathers and none on the mothers. (However, I also personally know two men who, after divorce, won custody of the kids and the wife was the one who had to pay child support. This is definitely not the typical situation though … )

However, what I am reacting to is the precise opposite.

I highlight the only unambiguously sensible thing that you said here, in my opinion, which is that there is plenty of blame to go around.

But to suggest that women should feel morally obliged to stay in an abusive relationship is a viewpoint that troubles me greatly. NO ONE, and I mean no one, for any reason whatsoever, should feel morally obliged to stay in an abusive relationship. Failing to support women who exit abusive relationships is complicity in the abuse, and I wholeheartedly support the use of collective resources (tax money) to ensure that they are economically able to do so in cases where the male half of the reproductive scenario somehow escapes the obligation to pay up.

21 mpledger March 5, 2016 at 5:00 pm

mpledger said
The women are accountable for their actions – they are the ones who stayed and bring the children up at considerable cost to themselves.

So Much For Subtlety
At considerable cost to the community. They are the ones paying.

The community would be paying even more if she also walked away. And she would be considerably better off – greater choice of jobs, greater ability to find a higher status mate, greater ability to find better accommodation, less hate directed at her, more time for herself …

The person being subsidised by the community is the parent who walks away. The parent who stays also subsidises the walker.

22 mpledger March 4, 2016 at 8:53 pm

The western world needs children because we are barely at replacement rates. Subsidising children is a good thing if you want economic growth.

The people who are being subsidised are the people who want sex and walk away from the consequences.. That’s who you are really subsidiing.. Single parents are probably paying for and doing more than their fair share of their children’s upbringing.

23 Byron R March 5, 2016 at 12:20 am

Given the high heritability of IQ and other cognitive traits, if one wants economic growth, it would be better to encourage people of higher human capital to have more children, rather than deadbeat dads and the women who let themselves get knocked up by them.

Surely as a statistician (I saw below), you recognize this? Apologies in advance if you are being sarcastic… but if even American statisticians are oblivious to the heritability of cognitive traits…. perhaps economic growth isn’t in the cards for such a country ; )

Strange times we live in, where we encourage smart women (and men) to delay or forgo children in favor of education (or “education”) and rat-race-like careers, while subsidizing the offspring of those on the left-side of the intelligence distribution.

24 mpledger March 5, 2016 at 12:53 am

I’m not American either and I’ve watched Idiocracy.

I’m all for encouraging women not to get knocked up by losers.

I don’t believe we should put active constraints on who can breed because 1) it’s impossible to know what future we are breeding for and 2) any test used to measure IQ and cognitive traits would be fallible and able to be gamed by the rich e.g. SATs.

25 Byron R March 5, 2016 at 1:24 am

“I don’t believe we should put active constraints on who can breed.”

Nor do I, nothing about constraints nor coercion. I just wouldn’t wish on my worst enemy for their country to subsidize the reproduction of its left-side of the bell curve.

“2) any test used to measure IQ and cognitive traits would be fallible and able to be gamed by the rich e.g. SATs.”

What do you mean, like rich Americans bribing SAT officials? I don’t think that happens, but you never know…

It should be expected that parental income would correlate with offpspring IQs/SATs, as offpsring inherit their parent’s IQs (the parent-child correlation of IQ is accounted for chiefly by heritability in countries like the US)–and smarter people/parents are able to make more money, on average.

Test like the SATs have high predictive power for things like college grades and future income, across and within different ethnicites, both sexes, and across different economic classes; and as per a paper posted on another recent MR thread, future STEM doctorates, STEM tenure rates, and patent rates for SATs taken at ages 12-13.

26 Nathan W March 5, 2016 at 4:24 am

“What do you mean, like rich Americans bribing SAT officials?”

No, like tutors, etc. Rich parents will observe the barriers to success for their children, and spend bucketloads of cash to ensure their children can vault over these barriers. Poor parents are not in such a good position to do so.

SATs may have strong predictive power. However, there may be a fallacy in that the same underlying explanatory factor explains both. For example, wealthy parents get their kids all the resources they need to get a high SAT score, and then later hook them up with networks that get them good jobs and business contacts, etc.

27 Byron R March 5, 2016 at 1:07 pm

@NathanW, it is ironic that you use the word “fallacy,” when it is the common fallacy of many Americans (although not saying you are necessarily American) to reach for unparsimonious rationalizations when faced with evidence that standardized test results are valid across different income groups, ethnicities, etc. Amusingly, they assert such rationalizations so confidently, so it only further draws attention to their foolishness.

For example, Sackett et al. estimated that among the national population, the SAT-college grade correlation was .53; .50 between SAT and grades once socioeconomic status was controlled for (barely moves); -.01 between SES and grades once SAT was controlled for (so no SES-grade correlation once SATs are accounted for).

Arciadocono and team were able to control for course-difficulty and found that SAT alone explains grade performance differences among Duke students, and not income, family education, or ethnic background.

Briggs estimated an effect of about 20 points from test preparation–which is small, and would be a ceiling on explaining any differentials between groups, as it is not like all “rich” students have test preparation and all “poor” students do not, for varying definitions of rich and poor.

28 Nathan W March 5, 2016 at 2:30 pm

“Amusingly, they assert such rationalizations so confidently.”

Um, you’re the one expressing things “so confidently”.

Ever heard of multicolinearity? Or, that the same underlying factor can explain a non-causal correlation between other variables? Both of which completely screw with standard analytical techniques. I have already proposed underlying factors which can be reasonably understood to cause such problems, at least to some degree. But I do not assert things “so confidently”, unlike you.

29 Byron R March 5, 2016 at 3:21 pm

Your sarcasm does you no favors. Correlation != causation is but a cliche’d bleating.

Waiting on the statistical references that show that the income-test score correlations is due to SES, and that the consistent predictive power of SAT across income strata is due to “rich” parents regularly buying outcomes for their children in standardized test scores, grades, future income, and things like STEM PhDs, tenure rates, and patent rates. I have shared multiple that reinforce the notion that SAT provides signal when SES is controlled for–but not vice versa–and that test prep effects are de minimus.

Sasquatch, UFO believers, etc. also constantly claim things “[they] have already proposed underlying factors which can be reasonably understood to cause such problems [that lead to no proof of Sasquatch/UFO existence], at least to some degree.” e.g. Recorded coyote noises are really sasquatch calls, because sasquatches imitate coyotes noises…

30 Nathan W March 5, 2016 at 8:57 pm

I wasn’t being sarcastic. I meant precisely what i said.

31 mpledger March 4, 2016 at 11:09 pm


But why are you making it all about the women when they’re mostly doing all they can – 76% work, 57% only have one child from an absent father, 36% are over 40.

It’s the absent, uncontributing parents you should be down on.

And what about the 17% of solo parents who are male? Why aren’t you railing against them for having sex before marriage or considering a wife optional?

32 3rdMoment March 4, 2016 at 12:58 pm

That’s crazy, these are big effects for something as simple as a better textbook. Nobody said this was a panacea, but marginal gains are nothing to sneer at. (Although I don’t know if this finding will hold up.)

33 Steve Sailer March 4, 2016 at 5:25 pm

In education there are a bunch of changes that would each improve test scores by 0.1 or, more likely, 0.01 standard deviations. There are no magic bullet 1.0 fixes. But we could make some progress.

34 Steve Sailer March 4, 2016 at 5:59 pm

By the way, it’s important to try to figure out why one textbook works better? Will it’s advantage replicate? What are the causes of its advantage that can’t be incorporated into other textbooks.

It’s a little bit like telling basketball players to be more like Steph Curry and make a high percentage of 30 footers. It would really help to teach them whatever it is that Curry does that lets him make 30 footers.

35 Nathan W March 4, 2016 at 9:58 pm

Among other things, making good use of all the senses is useful – building in methods that work across learning styles, for example learning by reading, writing, seeing (graphics, etc.), speaking, and a range from simple exercises to exercises which force students to think. Also, coming up with examples and analogies which are age appropriate are useful – say, learning which capital cities you can find linguini, sushi and curry in, rather than presenting them with a list of capital cities and instructing them to learn them. I recall visiting some math classes in Burkina Faso, and was interested to find that most examples were geared towards understanding development problems that were relevant to the locality, for example to calculate the materials cost of various infrastructure needs or the value of farm holdings – however, I doubt they had a textbook geared towards this, it was all on the teacher.

36 Spotted Toad March 5, 2016 at 8:18 am

There’s also the matter of diminishing marginal returns. So, for example, KIPP- at least the first few dozen franchise schools- really seems to produce impacts on the order of 0.3 SD; it’s at least been replicated in two separate RCTs. That’s a big effect, but not when you consider that the KIPP kids are in academic classes for almost twice as many hours per year as the comparison group. More importantly, you get most of that effect in the first year, and if you continue from a KIPP middle school to a KIPP high school, there’s no detected effect. (While the kids going from regular middle schools to KIPP high schools get the full 0.3 SD or so.) Smaller interventions like an enthusiastic teacher in a single secondary class or a different math textbook or a 45-minute CBT pep talk at the beginning of the year *should be expected* to have microscopic effects, but more importantly there’s no reason to expect that you can combine them linearly. The promoters of VAM will sometimes say that if a kid just gets three great teachers in a row, the achievement gap will be closed. Aside from my skepticism about VAM that someone linked to above, there’s just no evidence that is true;even if you think teachers are wholly responsible for the VAM of their classes, it is much more likely that almost all the effects will happen in the first year, with little or no additional increase from another great teacher, but fade out with an ordinary one.

Part of me thinks the whole “effect size” rhetoric is misplaced. What we are observing aren’t little clay ash trays being pressed into shape; we’re looking at people adapting to different social environments. You see most of the change in behavior the first year of the intervention, because that’s the first year they are in that new social environment. Bring them back to something closer to their old social environment, and they adapt to that one, perhaps in ways less congenial to the outcomes we want to observe.

37 dearieme March 4, 2016 at 5:44 pm

Astonishing: good books are better than bad books. Unheard of.

38 BC March 4, 2016 at 8:17 pm

Why even use Big Data statistical techniques here? Wouldn’t it be fairly straightforward to do RCTs for textbooks? If the null hypothesis is that textbooks don’t matter, then there should be no objection to randomly assigning different textbooks to different classrooms. One could do a statewide RCT bake-off every few years to select textbooks for the next few years.

39 mpledger March 4, 2016 at 9:53 pm

RCTs would be better than Big Data techniques. RCTs cost money to do well. And private companies won’t want to do them. Whereas testing companies love Big Data.

40 Nathan W March 4, 2016 at 10:01 pm

Would you like your child to participate in an RCT, or get whatever is perceived as the best book right now?

41 mpledger March 4, 2016 at 10:38 pm

I am a statistician so I’d go with the RCT every time. And even more so in this case where the benefit of the best book is so tiny.

In medical RCTs, new treatments are always meant to be compared against the best known treatment (or placebo if none) and the new treatment has to be thought of as a better treatment before being considered for an RCT. So if the educational RCT was run like that then the “best book right now” would be the lesser option.

42 BC March 5, 2016 at 2:55 pm

I would be pretty neutral due to a strong belief in the null hypothesis. In any event, given that textbooks aren’t replaced very often (??), one could probably stagger the RCT schedule so that any given student would be using an experimental textbook in a given subject only 1-2 years during their K-12 education. Test the 8th grade book one year, the 7th grade book the next year when the former 8th graders are in 9th grade, etc.

43 mm March 5, 2016 at 10:03 am

any parent in the 90s can tell you the result of this study- the heyday of the “whole language” instruction movement. A total, absolute, mind numbingly stupid method that couldn’t teach a dog to bark. So abject a failure that it is hard to describe how it was sold to the public- “child centered, holistic -yadda yadda yadda”. Total bunk. The worst part was that the local school system went while hog on it & replaced many textbooks with this instructional method. It was bad enough in writing & lit classes, but we rapidly replaced them. The real tragedy was the foreign language courses, where textbook are kept far longer d/t cost. The kids had to suffer for years in those classes. It is bad enough when a instructional method can’t teach kids their native language, imagine how badly that technique fails in teaching a foreign language.

44 rayward March 4, 2016 at 12:22 pm

It’s ironic that Kane mentions math, since the new textbooks for teaching math are either a great breakthrough in the method for teaching math or will render today’s children math idiots, unable to solve the simplest problems.

45 Ray Lopez March 4, 2016 at 12:35 pm

+1 – Agreed. I helped on homework for a US elementary student in California with the “new math” and it was ridiculous. No multiplication tables are taught, just factoring. 64 = 32 + 32 = 16 + 16 + 16 + 16, etc. Absurd. The kids should be taught to rote memorize 12×12 multiplication tables the same way we learned in the bad ole days, when we had to trudge through snow to go to school, uphill both ways!

46 Baphomet March 4, 2016 at 12:53 pm


47 rayward March 4, 2016 at 1:18 pm

Of course, it’s not just the memorization of multiplication tables, but the algorithms we were taught to use to solve math problems (e.g., “long division”). For those not familiar with the new math (Everyday Math), it comes from (of all places) the University of Chicago. The principal at my Godson’s public school selected Everyday Math many years ago when my Godson was in elementary school, and the kids subject to it almost universally under-performed (as compared to older kids not subject to Everyday Math) in math as they progressed through middle school and then high school. Why principals select certain textbooks is a mystery. I agree with the point of Tabarrok’s blog entry, but not necessarily in the way he intended.

48 Nathan W March 4, 2016 at 10:06 pm

I’ve always been doubtful of the benefits of learning long division in an era of calculators, but rote learning for multiplication tables has easily paid off a hundredfold in my life.

49 Paul March 5, 2016 at 4:42 pm

+1 also.

Consider the lifetime payoff vs. the small cost.

50 Daniel O'Neil March 4, 2016 at 12:31 pm

“textbooks have no unions”

Which is a shame because unions provide a localized, relatively transparent method for negotiating collectively for accountability and performance, whereas textbook selection is largely a non-transparent, almost entirely political and corrupt exercise where appointed committees from three large states – New York, Texas, and California – create the publishing imperative for the rest of the country. This is because the huge markets in those states are the only ones that publishers want to work towards and as a result all the other states have to buy the same books too.

This may change with an increase in digital publishing as the majority of textbook costs are in printing but that is still relatively far in the future.

I’m shocked Alex doesn’t understand this. No I’m not shocked.

51 efcdons March 4, 2016 at 12:56 pm

And the people who effectively pick text books for everyone in the country are hard headed technocrats like this lovely lady:

52 Joe in Boston March 4, 2016 at 1:06 pm

So is the article a headline because there is a teacher who is a bigot or because they are, apparently, not a Democrat?

I was going make the same general point but most of my examples come from the other end of the political spectrum. The teacher unions are overwhelmingly supporters of the Democrat party (I don’t believe that is a radical claim) and adopt many of the parties positions. They also are highly influential in textbook selling process. I’m in the industry at a small text book publisher and the sales process is incredibly political.

53 Daniel O'Neil March 4, 2016 at 1:11 pm

What is this “Democrat” party you are speaking of? I know of a Democratic party. You might look into that.

It would seem to me that it’s good that teachers are involved in the textbook selection process, as they are the ones who would be expected to teach with it. I think this is regardless of their political affiliation.

54 Joe in Boston March 4, 2016 at 1:35 pm

Clever; and I don’t disagree that teachers should be involved but they, as you raised (and why I responded to you), are not necessarily neutral and bring their own opinions and beliefs. Unfortunately today, possibly because of unions, or the pensions, or some other reasons there is a lot of politics involved in just about everything relating to k-12 teaching. The text book evaluation board members are not selected by random (from what I have seen) and the ones that volunteer often have views/agendas that are not necessarily about finding the best textbooks. Even when they do they often bring opinions about methodology that are not based on efficacy. This is how we got things like ‘new math’.

55 David Condon March 5, 2016 at 7:56 am

Textbooks are chosen by elected officials in Texas; not technocrats. Technocrats would be a huge step forward.

56 TMC March 5, 2016 at 2:25 pm

Obama keeps a tight lid on the details of his past. Who’s to say this lady is wrong? 🙂

57 Fazal Majid March 4, 2016 at 2:03 pm

Nobel physics laureate Richard Feynman served on the California textbook commission, and had very insightful comments on the process, including overt corruption:

58 Bob from Ohio March 4, 2016 at 3:42 pm

“unions provide a localized, relatively transparent method for negotiating collectively for accountability and performance”

Were you laughing when you wrote that?

59 Urstoff March 4, 2016 at 5:51 pm

A+ troll. Would read again!

60 anon March 4, 2016 at 12:53 pm

I mentioned iteration above. One of the things in early cycles is open resources, for example:

I think a rational government would push this harder. We can actually pay people to write open textbooks. Or to try them in RCTs and improve them.

61 prior_test1 March 4, 2016 at 1:26 pm

Did you happen to notice the Amazon affiliate links to the left? Profs. Cowen and Tabbarok have, at minimum in terms of an older discussion, several thousands dollars at stake merely from those links, much less their actual royalties, to dispute such a position.

62 anon March 4, 2016 at 1:29 pm

Everybody has to have a gig, but in the broader context, America might be a little off-kilter creating property when it should be spending for public goods.

I have much bigger fish to fry than that authors who can, do.

Why is Public Broadcasting in America copyrighted? Why isn’t it at a minimum Creative Commons?

63 prior_test1 March 4, 2016 at 2:40 pm

‘Everybody has to have a gig’

Nope – there are a select number of people who do not need a gig. This includes the fortunate children of those who are enjoying the fruits of America’s almost perpetual copyright regime – ‘What is entering the public domain in the United States? Not a single published work. Once again, no published works are entering the public domain in the United States this year. Or next year. In fact, no publication will enter our public domain until 2019.’

I am fairly confident that as both Profs. Cowen and Tabbarok are parents, they would likely be pleased at their children being able to enjoying the fruits of their copyrighted efforts for decades after their own death. In part so that their own children might not need the sort of ‘gig’ that both putatively libertarian and definitely tenured professors paid at Commonwealth of Virginia taxpayer expense enjoy, with its more than faint hint of pure hypocrisy.

64 anon March 4, 2016 at 2:49 pm

That’s funny. I’ve given money to Duke’s Center for the Study of the Public Domain. So I guess we are having an argument about which emphasis should be placed on which syllable.

Maybe I just personalize it less because, when it comes down to it, I’m not a stalker.

65 Nathan W March 4, 2016 at 10:13 pm

I’m amazed that more public resources are not dedicated to grants for open textbooks. This leads me to be suspicious of the role of textbook publishers in influencing text decisions and pedagogy decisions. Also, deference to free market thinking and anti-government bias may make this politically less feasible. But it’s not like we’d be talking large sums of money. Say, 100k grants to develop each of a few open texts, and leave it to local boards to select the ones they prefer – additional resources could be directed towards the most promising texts. The market will insist that it costs a lot more than this, but there are some truly dedicated people out there who would invest massive personal effort if there were some piddling little carrots like this.

66 matt H March 4, 2016 at 1:27 pm

What this says to me is their are large gains to be made from A/B testing course materials. I’ve always suspected this, glad to see some confirmation.

Imagine using software to teach kids with a teach standing by to help them. Imagine that the software works much worse than teachers at first, but through hundreds of rounds of optimization the software can teach kids better than the best teachers. Thats always been my vision for education.

67 Steve Adams March 4, 2016 at 6:49 pm

Yep! A\B the books, teachers, school structure- everything!!!

68 Nathan W March 4, 2016 at 10:27 pm

Perhaps when they find a way to get computers to keep students motivated and on task.

69 education realist March 4, 2016 at 2:25 pm

The significant factor here is the grade level. At the ES level, if the teachers don’t know much math, the book choice may be significant.

That said, teachers who don’t use books at all are very often dealing with low ability kids who can’t work at the textbook’s level. I’m much more likely to use textbooks in higher level classes than lower.

70 anon March 4, 2016 at 2:52 pm

There is the whole “Russian math” thing too.

Probably covered at MR one time or another

71 prognostication March 4, 2016 at 3:31 pm

Yes, there was some recent media coverage of the generally poor understanding of math among elementary educators and how that influences students attitudes’ towards, and achievements in, math.

72 Joe March 4, 2016 at 3:21 pm

Doesn’t all the evidence indicate that value added teacher scores are hugely flawed and suffer from massive variance year to year?

73 mpledger March 4, 2016 at 9:59 pm
74 mkt42 March 5, 2016 at 2:18 am

Thanks, I was going to comment that Alex seems to be putting too much faith into Value Added Measures. Most of the measure are based on standardized test scores, which have low reliability and low predictive power.

75 B.B. March 4, 2016 at 3:46 pm

Textbooks may have no union, but they have copyright monopoly.

From December 2001 to December 2015, college textbook prices rose 125%, using CPI data. By contrast, total CPI rose 34%. That is twice as fast as medical prices rose. It is even faster than cigarette prices rose. It is slightly faster than college tuition prices rose.

It is outrageous. The textbooks have monopolies, and students have no choice but to buy what is assigned, and revisions destroy the used book market, and teachers have no incentive to shop for lower priced textbooks for their students. What a scam. Donald Trump couldn’t have done it better.

I say let’s migrate to open-access free online textbooks in the public domain. Information wants to be free. Maybe Bill Gates can provide seed money to get this thing going before we bury our young people in a mountain of subprime student loans that they can never pay off.

76 Nathan W March 4, 2016 at 10:31 pm

Most profs offer alternative question sets from earlier editions and place texts on short-term loan at the school library. It is very rare for updated editions to contain substantive differences, especially in STEM fields (surprisingly). Once I figured this out, I only ever bought a handful of texts that I understood to be of high enough quality to keep for reference purposes (got rid of them when I realized that wikipedia had everything I needed for reminders on known subjects.)

Publishers are trying to milk the remaining few students who still purchase texts.

77 Hazel Meade March 4, 2016 at 4:11 pm

I believe this.

I’ve had both great teachers and shitty teachers. But when I had a good textbook, I could always fall back on reading the text and render the shitty teacher irrelevant. The textbook allows self-motivated students to teach themselves.

78 Ted Craig March 4, 2016 at 4:41 pm

“it’s easy to replace one textbook with a better textbook.”

I’m guessing it’s not.This isn’t college, where tje students directly bear the costs of the books. In addition to the cost, teachers have to be trained in the material. Also,text books are usually part of a program that includes several grades, so there might be transition issues for some students. For example, you might have been taught X in fourth grade using the old system, but now its taught in third grade. So what do you do with all the fourth graders who haven’t learned it?

All of these challenges can be dealt with, but there will be costs of different kinds, so it won’t necessarily be easy.

79 Urso March 4, 2016 at 4:45 pm

The first step in any economic analysis is to assume away all transaction costs.

80 David Condon March 5, 2016 at 7:49 am

I would expect the cost of evaluating, firing and replacing bad teachers to be much higher than the transaction costs of textbooks. Easy is relative.

81 Komori March 5, 2016 at 1:44 pm

It really isn’t, and it isn’t a new problem. Feynman’s book “Surely You’re Joking, Mr. Feynman!” has a story about the time he served on a textbook selection panel (see link). It doesn’t seem like much has changed since.

82 Urso March 4, 2016 at 4:42 pm

Prof. Tabarrok is such a sucker for these “one weird trick/hundred dollar bill just lying on the sidewalk!” stories; never mind whether any of them actually ever pan out. Maybe he should read his own posts about publication bias and replicability.

83 carlolspln March 4, 2016 at 6:52 pm

“An annual report on the effectiveness of textbooks would transform the market, by providing publishers and software developers with a stronger incentive to compete on quality”

Typical feeble minded obsession with education ‘metrics’.

File under ‘Speculative’.

84 March 4, 2016 at 7:13 pm

Jerome Dancis, Ph.D. (math), Associate Professor Emeritus, Math Dept, Univ of Maryland :
“Try Singapore Math Textbooks. Your students will learn Math”

VOA: “Singapore Math Adds Up for US Teachers”

85 March 4, 2016 at 7:18 pm


“What’s Singapore Math? … Mastery, Not Memorization”

86 David Condon March 4, 2016 at 10:55 pm

“An annual report on the effectiveness of textbooks would transform the market, by providing publishers and software developers with a stronger incentive to compete on quality.”

Consumer information is NOT the same thing as an incentive. Just because consumers have greater access to information does not necessarily mean that they will access it, nor does it necessarily mean that the relationship between their decision and the incentive has changed.

87 ChrisA March 4, 2016 at 11:57 pm

Math and methods vary so widely, isn’t this likely a case more of better alignment between the methodologies used in the test and those methodologies and examples presented in the text book?

To take an extreme example – if I knew exactly what questions were going to be on the test, I could write an extremely effective text book.

My view is that less than 1% of the population are capable or need to learn any maths beyond simple algebra and schools would be better off teaching the 99% these capabilities over and over again until they get it. Like language teaching in school, math teaching goes far too advanced for most people, they would be better off staying simple and repetitive for most people.

88 prognostication March 5, 2016 at 10:56 am

A tremendous percentage of U.S. students NEVER get past simple algebra, even in college.

89 prior_test2 March 5, 2016 at 1:41 am

Man, who knew that linking to and talking about the world offering an abundance of textbook authors would lead to the next iteration.

It is always hard to predict what facts will receive the silent treatment here – considering the self-promotion at the left, you would think linking to a textbook publisher would be fine, but nope. Though the rule of thumb about the more accurate, the more probable, still remains true.

90 Nathan W March 5, 2016 at 4:16 am

One problem I have frequently encountered is private education companies that peddle their own low-grade teaching materials as a part of the offering, instead of shelling out for higher quality texts. I guess this saves on money, and they see it as an investment in potentially being able to sell their own copyrighted teaching materials to other companies. This has been a problem each and every time I’ve had texts pre-selected by an education company that both hires teachers and sets textbooks decisions without consulting teachers.

This criticism would then apply mostly at the level of branded chain schools, and not, say, individual private schools which rarely have the resources to develop their very own learning materials and also have a strong interest to select the text on the basis of perceived quality and little else.

Textbook decisions should be separated from the profits/wages involved in the supply of educational services. (Although it doesn’t bother me if some profs use their own textbooks, because presumably they are highly familiar with the content and methods as presented in the text, and thus this is highly consistent with respecting teacher’s decisions with regards to methods, etc. which work best for that teacher.)

91 prior_test2 March 5, 2016 at 5:18 am

‘Textbook decisions should be separated from the profits/wages involved in the supply of educational services.’

A noble thought, especially if you can convince text book authors such as Profs. Cowen and Tabarrok to forego the money they undoubtedly – and rightfully, in legal terms – receive for their contributions to the textbook publishing industry.

At the university level, professors able to publish textbooks – often with the quite reasonable supposition that the professor’s textbook is more suited than anyone else’s for what that professor wishes to teach – very much consider their textbook decisions in that light. After all, if they sell l 300 copies to the class they teach, that is just how things work in the world of tenured university professors. And for those unaware of how it works, without bothering to spend the time to type ‘Krugman textbook’ into a seach machine, just check into how commonly various famous economist written textbooks are sold. It is a nice amount of money for the most successful authors, whose own textbook decisions are invariably also influenced by something that resembles profits/wages.

92 Money March 5, 2016 at 9:42 am

Hard to do it “cheaply” when that “better textbook” costs $283

93 Paul March 5, 2016 at 4:53 pm

My friendly book rep sent me their latest version of one of their intermediate macro texts. One with three distinguished authors. Lots of promo for their online resources.

It has 15 chapters. And a whopping ONE chapter on long-run growth.

Yet lots of professors will be using it, I suppose.

Comments on this entry are closed.

Previous post:

Next post: