Results for “evidence-based” 37 found
DARE to Look at the Evidence!
We must have Drug Abuse Resistance Education…I am proud of your work. It has played a key role in saving thousands of lives and futures.
Speaking at the 30th DARE Training Conference, Attorney General Jeff Sessions was enthusiastic and strongly supportive of DARE, the program started in Los Angeles in 1983 that uses police officers to give young children messages about staying drug free and resisting peer pressure.
And what do our excellent colleagues at GMU’s Center for Evidence-Based Crime Policy say about DARE?
D.A.R.E. is listed under “What doesn’t work?” on our Review of the Research Evidence.
Rosenbaum summarized the research evidence on D.A.R.E. by titling his 2007 Criminology and Public Policy article “Just say no to D.A.R.E.” As Rosenbaum describes, the program receives over $200 million in annual funding, despite little or no research evidence that D.A.R.E. has been successful in reducing adolescent drug or alcohol use. As Rosenbaum (2007: 815) concludes “In light of consistent evidence of ineffectiveness from multiple studies with high validity, public funding of the core D.A.R.E. program should be eliminated or greatly reduced. These monies should be used to fund drug prevention programs that, based on rigorous evaluations, are shown to be effective in preventing drug use.”
A systematic review by West and O’Neal (2004) examined 11 published studies of D.A.R.E. and reached similar conclusions. D.A.R.E. has little or no impact on drug use, alcohol use, or tobacco use. They concluded that ““Given the tremendous expenditures in time and money involved with D.A.R.E., it would appear that continued efforts should focus on other techniques and programs that might produce more substantial effects” (West & O’Neal, 2004: 1028).
Recent reformulations of the D.A.R.E. program have not shown successful results either. For example, the Take Charge of your Life program, delivered by D.A.R.E. officers was associated with significant increases in alcohol and cigarette use by program participants compared to a control group (Sloboda et al., 2009).
Straight thinking about Bayer and Monsanto
That is my latest Bloomberg column, hardly anyone has a consistent and evidence-based view on this deal. Here is one bit:
Critics who dislike Monsanto for its leading role in developing genetically modified organisms and agricultural chemicals shouldn’t also be citing monopoly concerns as a reason to oppose the merger — that combination of views doesn’t make sense. Let’s say for instance that the deal raised the price of GMOs due to monopoly power. Farmers would respond by using those seeds less, and presumably that should be welcome news to GMO opponents.
Yet on the other side:
What does Bayer hope to get for its $66 billion, $128-a-share offer? The company has argued that it will be able to eliminate some duplicated jobs and expenses, negotiate better deals with suppliers and invest more funds in research and development. Maybe, but the broader reality is less cheery. There is a well-known academic literature, dating to the early 1990s, showing that acquiring firms usually decline in value after tender offers, especially after the biggest deals. Mergers do not seem to make companies more valuable or efficient.
And this:
The whole Bayer-Monsanto case is a classic example of how a vociferous public debate can disguise or even reverse the true issues at stake. If Bayer fails to close the deal for Monsanto, Bayer shareholders may be the biggest winners. The biggest losers from a failed deal may be its opponents, who will spend the rest of their lives in a world where misguided judgments of corporate popularity have increasing sway over laws and regulations.
Do read the whole thing.
Former Dem CEAs Write Open Letter to Sanders
A strongly worded letter from Krueger, Goolsbee, Romer and Tyson to Sanders and his economic team chastising them for unrealistic, unscientific numbers. (No indent).
Dear Senator Sanders and Professor Gerald Friedman,
We are former Chairs of the Council of Economic Advisers for Presidents Barack Obama and Bill Clinton. For many years, we have worked to make the Democratic Party the party of evidence-based economic policy. When Republicans have proposed large tax cuts for the wealthy and asserted that those tax cuts would pay for themselves, for example, we have shown that the economic facts do not support these fantastical claims. We have applied the same rigor to proposals by Democrats, and worked to ensure that forecasts of the effects of proposed economic policies, from investment in infrastructure, to education and training, to health care reforms, are grounded in economic evidence. Largely as a result of efforts like these, the Democratic party has rightfully earned a reputation for responsibly estimating the effects of economic policies.
We are concerned to see the Sanders campaign citing extreme claims by Gerald Friedman about the effect of Senator Sanders’s economic plan—claims that cannot be supported by the economic evidence. Friedman asserts that your plan will have huge beneficial impacts on growth rates, income and employment that exceed even the most grandiose predictions by Republicans about the impact of their tax cut proposals.
As much as we wish it were so, no credible economic research supports economic impacts of these magnitudes. Making such promises runs against our party’s best traditions of evidence-based policy making and undermines our reputation as the party of responsible arithmetic. These claims undermine the credibility of the progressive economic agenda and make it that much more difficult to challenge the unrealistic claims made by Republican candidates.
Sincerely,
Alan Krueger, Princeton University
Chair, Council of Economic Advisers, 2011-2013
Austan Goolsbee, University of Chicago Booth School
Chair, Council of Economic Advisers, 2010-2011
Christina Romer, University of California at Berkeley
Chair, Council of Economic Advisers, 2009-2010
Laura D’Andrea Tyson, University of California at Berkeley Haas School of Business
Chair, Council of Economic Advisers, 1993-1995
Algorithm Aversion
People don’t like deferring to what I earlier called an opaque intelligence. In a paper titled Algorithm Aversion the authors write:
Research shows that evidence-based algorithms more accurately predict the future than do human forecasters. Yet, when forecasters are deciding whether to use a human forecaster or a statistical algorithm, they often choose the human forecaster. This phenomenon, which we call algorithm aversion, is costly, and it is important to understand its causes. We show that people are especially averse to algorithmic forecasters after seeing them perform, even when they see them outperform a human forecaster. This is because people more quickly lose confidence in algorithmic than human forecasters after seeing them make the same mistake. In five studies, participants either saw an algorithm make forecasts, a human make forecasts, both, or neither. They then decided whether to tie their incentives to the future predictions of the algorithm or the human. Participants who saw the algorithm perform were less confident in it, and less likely to choose it over an inferior human forecaster. This was true even among those who saw the algorithm outperform the human.
People who defer to the algorithm will outperform those who don’t, at least in the short run. In the long run, however, will reason atrophy when we defer, just as our map-reading skills have atrophied with GPS? Or will more of our limited resource of reason come to be better allocated according to comparative advantage?
Tuesday assorted links
1. “Ekki staðalbúnaður í smalamennsku!” With video, of course, and implying the advantages of water transport.
3. Steven Landsburg makes some good points, but Summers may be able to invoke threshold effects.
4. Harvard faculty actually seem to hate the best parts of Obamacare. Bravo to this article. And quick summaries of evidence-based medicine.
5. “I’m the poster child of evil [art] speculation…” An excellent piece, also NYT.
6. How big is the sexism problem in economics? Kimball and anon.
Is the Volcker rule a good idea?
Treasury Secretary Jacob J. Lew has strongly urged federal agencies to finish writing the Volcker Rule by the end of the year — more than a year after they had been expected to do so — and President Obama recently stressed the importance of the deadline.
By the way, five (!) agencies are writing the rule, which is not a good sign. As for the Volcker rule more generally, here are a few points:
1. If restricting activity X makes large banks smaller, that will ease the resolution process, following a financial crack-up. That is a definite plus, although we do not know how much easier resolution will be.
2. It is not clear that banning bank proprietary trading will lower the chance of such a financial crack-up. The overall recent record of real estate lending is not a good one, and as Edward Conard pointed out, restricting banks to the long side of transactions is not obviously a good idea. I do see the moral hazard issue with allowing banks to engage in the potentially risky activity of proprietary trading. Still, so far the data are suggesting that the banks which cracked up during the crisis did so because of overconfidence and hubris, not because of moral hazard problems (i.e., they still were holding lots of the assets they otherwise might have been trying to “game”).
3. There is no strong connection between proprietary trading and our recent financial crises.
3b. Today the bugaboo is “big banks” but once it was “small banks” and for a while “insufficiently diversified banks.” Maybe it really is big banks, looking forward, or maybe we just don’t know. Small banks have their problems too.
4. There is some chance that proprietary trading will be pushed to a more dangerous, harder to regulate corner of our financial institutions.
5. There is some danger that loopholes in the regulation itself — especially as concerns permissible client activities — may undercut the original intent of the regulation. This will depend on exactly how well the regulation is written, but past regulatory history does not make me especially confident here. And the distinction between “speculation” and “hedging” cannot be clearly defined. Should we be writing rules whose central distinctions may be arbitrary? And yet CEOs will have to sign off on compliance (with 950 pp. of regulations) personally. Is that a good use of CEO attention? Here is a good FT piece about how hard (and ambiguous) it will be to enforce the rule globally.
6. I do not myself shed too many tears over the “these markets will become less liquid without banks’ participation” critique, but I would note this is a personal judgment and the scientific status of such a claim remains unclear.
7. Many people, even seasoned commentators, approach the Volcker rule with mood affiliation, starting with how much we should resent our banks or our regulators or how we should join virtually any fight against either “big banks” or regulators. I see many analyses of this issue which spend most of their time on “mood affiliation wind-up,” as I call it, and not so much time on the actual evidence, which is inconclusive to say the least.
8. We still seem unwilling to take actions which would transparently raise the price of credit to homeowners. We instead prefer actions which appear to raise no one’s price of credit and which are extremely non-transparent in their final effects. You can think of the Volcker rule as another entry in this sequence of ongoing choices. That should serve as a warning sign of sorts, and arguably that is a more important truth than the case either for or against the rule.
When I add up all of these factors, I come closer to a “don’t do the Volcker rule” stance in my mind. The case for the rule puts a good deal of stress on #1, but overall it does not fit the textbook model of good regulation. I probably have a more negative opinion of “an extreme willingness to experiment with arbitrary regulatory stabs” than do most of the rule’s supporters, and that difference of opinion is perhaps what divides us, rather than any argument about financial regulation per se.
I really do see how the Volcker rule might work out just fine or even to our advantage. I also see the temptation of arguing “I am against big banks, this is the legislation against big banks which is on the table, so I am going to support it.” But let us at least present to our public audiences just how weak is the evidence-based case for doing this.
Addendum: You will find a different point of view from Simon Johnson here. Here is a counter to his claim that prop trading losses were significant in 2008: “Loan losses didn’t just dwarf trading losses in absolute terms. Loan losses as a share of banks’ total loan portfolios also exceeded trading losses as a share of banks’ trading accounts. Yet no one’s arguing banks should stop lending in order to protect depositors (and rightly so). In short, those expecting the Volcker Rule to be a fix-all for Wall Street’s ills have probably misplaced their hope—the rule seems like a solution desperately seeking a problem.”
Sarah Constantin replies on MetaMed
Not long ago I linked to this Robin Hanson blog post on MetaMed. I was sent this reply, which I will put under the fold:
I noticed you linked Robin Hanson’s article on MetaMed on Marginal Revolution. I’m the VP of research at MetaMed, and I just wanted to tell you a little bit more about us, because if all you know about us is the Overcoming Bias article you might get some misleading impressions.
Medical practice is basically a mass-produced product. Professional and regulatory bodies (like the AMA) put out guidelines for treatment. At their best, these guidelines follow the standards of evidence-based medicine, which means that on average they will produce the best health outcomes in the general population. (Of course, in practice they often fall short of that standard. For example, checklists are overwhelmingly beneficial by an evidence-based medicine standard, and yet are not universally used.)
But even at their best, the guidelines that are best from a population-health standpoint need not be optimal for an individual patient. If you have the interest and the willingness to pay, investigating your condition in depth, in the context of your entire medical history, genetic data, and personal priorities, may well turn up opportunities to do better than the standardized medical guidelines which at best maximize average health outcomes.
That’s basically MetaMed’s raison d’etre. And it’s a pretty conservative hypothesis, in fact. We may harbor a few grander ambitions (for example, I come from a mathematical background and I’m working on some longer-term projects related to algorithmically automating parts of the diagnostic process, and using machine learning principles on biochemical networks in novel ways) but fundamentally the thing we claim to be able to do is give you finer-grained information than your doctor will. We’re, of course, as yet unproven in the sense that we haven’t had enough clients to provide empirical evidence of how we improve health outcomes, but we’re not making extraordinary claims.
Robin Hanson seems to be implying that MetaMed is claiming to be useful only because we’re members of the “rationalist community.” This isn’t true. We think we’re useful because we give our clients personalized attention, because we’re more statistically literate than most doctors, because we don’t have some of the misaligned incentives that the medical profession does (e.g. we don’t have an incentive to talk up the benefits of procedures/drugs that are reimbursable by insurance), because we have a variety of experts and specialists on our team, etc.
The “rationalist” sensibility is important, to some degree, because, for instance, we’re willing to tell clients that incomplete evidence is evidence in the Bayesian sense, whereas the evidence-based medicine paradigm says that anything that yet hasn’t been tested in clinical trials and found a 5% p-value is completely unknown. For instance, we’re willing to count reasoning from chemical mechanisms as (weak) evidence. There’s a difference in philosophy between “minimize risk of saying a falsehood” and “be as close to accurate as possible”; we strive to do the latter. So there’s a sense in which our epistemic culture allows us to be more flexible and pragmatic. But we certainly aren’t basing our business model on a blanket claim of being better than the establishment just because we come from the rationalist community.
What do I think of Obama’s universal pre-school proposal?
Of course there are no significant details yet, but here are a few points.
1. The evidence that this can be done effectively in a scalable manner is basically zero. Aren’t massive policies (possibly universal?) supposed to be based on evidence? (How about running a large-scale RCT first, a’la the Rand health insurance experiment? And by the way, here is a quick look at the evidence we have on pre-school, and here, not nearly skeptical enough in my view. And think in terms of lasting results, not getting kids to read nine months earlier, etc. You can find evidence for persistent math gains in Tulsa, OK, but no CBA.)
2. That doesn’t mean we should do nothing.
3. Let’s say we have “the political will” to do something effective (debatable, of course). Is adding on another layer of education, and building that up more or less from scratch in many cases, better than fixing the often quite broken systems we have now? I know well all the claims about “needing to get kids early,” but is current kindergarten so late in life? Why not have much better kindergartens and first and second grade experiences in the ailing school districts? Or is the claim that by kindergarten “it is too late,” yet a well-executed government early education could fix the relevant problems if applied at ages three to four? Would such a claim mean that we are currently writing off many millions of American children, as it stands now?
4. This is what federalism is for. Let’s have an experiment emanating from the state and/or local level.
5. What should we infer from the fact that no such truly broad-based state-level experiment has happened yet? (Georgia and Oklahoma have come closest.) That the states are lacking in vision, relative to the Presidency? Or that a workable version of the idea is hard to come up with, execute, and sell to voters?
6. In Finland government education doesn’t really touch the kids until they are six years old. Don’t they have a very good system? Some call it the world’s best. Maybe the early years are very important, but perhaps pre-schooling is not the key missing piece of the puzzle. (NB: See the comments for dissenting views on Finland.)
Addendum: Here are good comments from Reihan. See also this Brookings study: “This thin empirical gruel will not satisfy policymakers who want to practice evidence-based education.”
More on Online Education
At Cato Unbound I respond to some of the critics of my article Why Online Education Works. Here is one bit:
We do need more studies of offline, online, and blended education models, but the evidence that we do have is supportive of the online model. In 2009, The Department of Education conducted a meta-analysis and review of online learning studies and found:
- Students in online conditions performed modestly better, on average, than those learning the same material through traditional face-to-face instruction.
- Instruction combining online and face-to-face elements had a larger advantage relative to purely face-to-face instruction than did purely online instruction.
- Effect sizes were larger for studies in which the online instruction was collaborative or instructor-directed than in those studies where online learners worked independently.
- The effectiveness of online learning approaches appears quite broad across different content and learner types. Online learning appeared to be an effective option for both undergraduates (mean effect of +0.30, p < .001) and for graduate students and professionals (+0.10, p < .05) in a wide range of academic and professional studies.
There Will Be Blood
Economists often reduce complex motivations to simple functions such as profit maximization. Writing in The Economist, Buttonwood ably criticizes such simplifications. Buttonwood is too quick, however, to conclude that simplification falsifies. For example, Buttonwood argues:
If there is a shortage of blood, making payments to blood donors might seem a brilliant idea. But studies show that most donors are motivated by an idea of civic duty and that a monetary reward might actually undermine their sense of altruism.
As loyal readers of this blog know, however, the empirical evidence is that incentives for blood donation actually work quite well. Mario Macis, Nicola Lacetera, and Bob Slonim, the authors of the most important work on this subject (references below), write to me with the details:
The decision to donate blood involves complex motivations including altruism, civic duty and moral responsibility. As a result, we agree with Buttonwood that in theory incentives could reduce the supply of blood. In fact, this claim is often advanced in the popular press as well as in academic publications, and as a consequence, more and more often it is taken for granted.
But what is the effect of incentives when studied in the real world with real donors and actual blood donations?
We are unaware of a single study of real blood donations that shows that offering an incentive reduces the overall quantity or quality of blood donations. From our two studies, both in the United States covering several hundred thousand people, and studies by Goette and Stutzer (Switzerland) and Lacetera and Macis (Italy), a total of 17 distinct incentive items have been studied for the effects on actual blood donations. Incentives have included both small items and gift cards as well as larger items such as jackets and a paid-day off of work. In 16 of the 17 items examined, blood donations significantly increased (and there was no effect for the one other item), and in 16 of the 17 items studied no significant increase in deferrals or disqualifications were found. No study has ever looked at paying cash for actual blood donations, but several of the 17 items in the above studies involve gift cards with clear monetary value.
Although many lab studies and surveys have found differing evidence focusing on other outcomes than actual blood donations (such as stated preferences), the empirical record when looking at actual blood donations is thus far unambiguous: incentives increase donations.
Given the vast and important policy debate regarding addressing shortages for blood, organ and bone marrow in developed as well as less-developed economies, where shortages are especially severe, it is important to not only consider more complex human motivations, but to also provide reliable evidence, and interpret it carefully. The recent ruling by the 9th Circuit Court of Appeals allowing the legal compensation of bone marrow donors further enhances the importance of the debate and the necessity to provide evidence-based insights.
Here is a list of references:
Goette, L., and Stutzer, A., 2011: “Blood Donation and Incentives: Evidence from a Field Experiment,” Working Paper.
Lacetera, N., and Macis, M. 2012. Time for Blood: The Effect of Paid Leave Legislation on Altruistic Behavior. Journal of Law, Economics and Organization, forthcoming.
Lacetera N, Macis M, Slonim R 2012 Will there be Blood? Incentives and Displacement Effects in Pro-Social Behavior. American Economic Journal: Economic Policy 4: 186-223.
Lacetera N, Macis M, Slonim R.: Rewarding Altruism: A natural Field Experiment, NBER working paper.
“Ethos of the Unit”
This is from a child and adolescent mental health group at University College London, but it could and should also count as “Ethos of the Blogger”:
•All research is provisional
•All research raises as many questions as it answers
•All research is difficult to interpret and to draw clear conclusions from
•Qualitative research may be vital to elaborate experience, suggest narratives for understanding phenomena and generate hypotheses but it can’t be taken to prove anything
•Quantitative research may be able to show hard findings but can rarely (never?) give clear answers to complex questionsAnd yet, despite all the challenges, it is still worth attempting to encourage an evidence-based approach, since the alternative is to continue to develop practice based only on assumption and belief.
For the pointer I thank Michelle Dawson.
Assorted links
1. Via Chris F. Masse, is Star Wars a remix?
2. When is death an aphrodisiac?
3. David Henderson on TGS. And more from Michael Mandel. And Bryan Caplan, you can scroll down there for additional posts.
Portuguese drug decriminalization
Caitlin Elizabeth Hughes and Alex Stevens have written a new study:
The issue of decriminalizing illicit drugs is hotly debated, but is rarely subject to evidence-based analysis. This paper examines the case of Portugal, a nation that decriminalized the use and possession of all illicit drugs on 1 July 2001. Drawing upon independent evaluations and interviews conducted with 13 key stakeholders in 2007 and 2009, it critically analyses the criminal justice and health impacts against trends from neighbouring Spain and Italy. It concludes that contrary to predictions, the Portuguese decriminalization did not lead to major increases in drug use. Indeed, evidence indicates reductions in problematic use, drug-related harms and criminal justice overcrowding. The article discusses these developments in the context of drug law debates and criminological discussions on late modern governance.
Assorted links
1. Shoup replies to O'Toole on minimum parking requirements.
3. Interview with Oliver Sacks.
4. "too much justice?" — interview with Harvey Mansfield.
Questions that are rarely asked: why so many retired cops?
JIm Crozier, a loyal MR reader, asks:
Why do cop movies and TV shows so often begin with an older (and often jaded) officer that is just about to retire? It is quite astounding how often this unrealistic plot trick is employed, and the psychological grounding seems weak at best.
I don't have the viewing experience to give you an evidence-based response. I would think the answer might lie in marginal utility theory plus behavioral economics. Perhaps all his life that officer has failed to achieve some desired end, such as catching a criminal, bringing an evil politician to justice, reforming the corrupt police force, or whatever. If the officer is near retirement, we know we are watching a very dramatic story which will define the life and career of that officer for ever and ever. It is harder for the viewer to have the same feeling if the officer has four years, three months remaining on the force. Failure would not mean final failure.
On the behavioral front, our impressions of experiences, and the memories we form, very often depend on what comes last. Judges are more impressed by the group which sings last in the Eurovision contest, even though it is randomized. The viewer thus implicitly knows that the cop really cares about the final segment of his or her career, reinforcing the point about decisiveness and marginal utility.
Viewers, can you do better?