The two people (Wolfers and Ozimek) who did the empirical work did a great job, but much of the rest of the exchange from other commentators has missed the point.
If you approach the debate as an emotional referendum on how good or bad Trump (Clinton) would be, you’re probably going to get it wrong. You will view yesterday’s exchange as being about choosing the Wolfers estimation or the Ozimek one, the latter showing that increases in Trump’s odds didn’t seem to hurt the stock market up through a particular date. If then you sided with Wolfers, you could keep a very negative view of what Trump would be like, or if you sided with Adam’s investigation you could still wonder to a greater extent.
The better way to think about the exchange is that Adam (and I) raised a puzzle. Given that economists as a whole don’t like Trump (look at endorsements), why haven’t the regular fluctuations in his odds had more of an impact on the stock market?
Now comes the Justin Wolfers study, showing the stock market went up a lot as Hillary Clinton was winning the debate. That makes the puzzle bigger not smaller. It adds to the preexisting prior about what correlations we should find in those earlier data points. Why for instance didn’t Trump’s fairly rapid pneumonia-inspired, pre-debate rise from 30 to 36 spook the markets in a big way? Why didn’t Trump’s longer-term rise from near zero to 36 bring a lot of market turmoil? And since a strong economy should help the incumbent Party, the puzzle is all the stronger; you can’t expect a strong economy to boost both the stock market and Trump’s odds as a confounding third factor. And note of course that Justin himself, in other contexts including on Twitter, will assign weight to the churning movements in the prediction markets, even if he doesn’t consider them any kind of decisive test.
A few of the options on the table are to say gridlock is stronger than we had thought, prediction markets less reliable, other candidates less reliable, or that Trump cutting taxes on capital relative to HRC will for the stock market outweigh some of the costs of his presidency. I’m not pushing any one of those, I am suggesting that at least one from this and a broader list ought to be true.
Many many of you have responded to such conundrums with answers starting with but not ending with the concept of noise and low-powered tests. That is a perfectly fine set of responses but then you must apply the resulting beliefs consistently to all other spheres. You could say for instance: “So much noise comes along in our economy. I do prefer Clinton to Trump, but because of all this noise I’m really not so sure Clinton will work out better for the economy. All of the other intervening economic events is what the prediction market and stock market data were picking up and that is why Adam’s test was imperfect. My judgments are imperfect too.”
That is an entirely permissible answer, if you really believe it and embrace it. The error is to segment your belief space. If you say “Wolfers beats Ozimek because Ozimek doesn’t consider noise enough, therefore I stick with my belief that Trump is really bad for the economy,” well that kind of mistake belongs in a Jonathan Haidt novel. I find few people are willing to embrace the more consistent statistical preference plus agnosticism, rather they play the game of “statistical noise for thee but not for me.”
Plenty of statistical tests have low power, including, believe it or not, the ones you run with your political intuitions.
Most generally, don’t look to throw out information, or see one study as trumping another, rather seek to use and interpret all of the information available.
By the way, one possible answer that fully reconciles the data of both Wolfers and Ozimek is to suggest stock markets started seeing Trump as “incurably terrible” only during the debate itself. That is hardly a confirmed hypothesis (we’ll see going forward), but it is another way of recognizing why Wolfers and Ozimek have not produced competing hypotheses, rather two pieces of information for revising a broader Bayesian mosaic.