Remembering Who Was Right
We spend endless hours arguing who is right in current controversies, but minutes or less remembering who was right before. Oh we sometimes brag about selected cases, but we rarely collect systematic statistics. (Rare exceptions include weathermen, business analysts, and sports punters.)
Yet such track records are just what we need to figure out who is right today. You might think it enough to know which side is smarter or better informed. But a janitor can consistently beat his arrogant CEO, if the janitor is careful to only disagree on topics where he clearly knows more. When disputants are aware of each others’ opinions, it is those who better know when to defer and when to stand their ground that should be right more often.
Yes it would be hard to track and score everything everyone says, but we could do a lot more than we now do. Widespread idea futures or David Brin’s prediction registries could help us estimate which individuals tend to be right more often. And it should be even easier to evaluate standard demographic categories.
When a husband and wife disagree, who tends to be right? How about a parent and child, a student and teacher, a boss and employee, a liberal and conservative? For a few thousand dollars, we could bring dozens of such pairs into the lab, ask them various questions together, and see who is right when they disagree. Perhaps lab disputes differ from field disputes in unknown systematic ways, but it would be a great first step.
Perhaps even more useful, we could take a sample of real media disputes and see both who tends to take which side, and which side seemed more right in the end. I have just finished one such analysis, on the dispute over the policy analysis market (PAM), a.k.a. terrorism futures. Four readers rated 555 media articles on which gave favorable or unfavorable impressions of PAM, and these ratings were regressed on sixteen features of articles, publications, and authors.
The result? Since five strong indicators of more informed articles agreed on a more favorable rating, the favorable position looks like the “right” one here. In the case of PAM, these groups were right more often: men, conservatives, web or broadcast media over print and books, and those who talked to people with firsthand knowledge, wrote longer articles, wrote news as opposed to editorials, and wrote for specialty publications with larger circulations and more awards.
Of course we need to look at more disputes to see which of these indicators holds more generally. But a few tens of thousands of dollars should pay for that. And with good indicators in hand, we could in real time predict which sides are probably right in current disputes. Wouldn’t that be something?