Popularizing the Arrow Impossibility Theorem

Arnold Kling presents it as an under-popularized idea, so here is my attempt to popularize it:

Let's say you had two people on a desert island, John and Tom, and John wants jazz music on the radio and Tom wants rap.  Furthermore any decision procedure must be consistent, in the sense of applying the same algorithm to other decisions.  In this set-up (with a further assumption), there is only dictatorship, namely the rule that either "Tom gets his way" or "John gets his way."

Arrow's axioms, by invoking "the independence of irrelevant alternatives" (read: pairwise comparisons) require that all information in the problem be ordinal.  A decision procedure such as: "Turn the radio (and allocate other resources) to which station makes the winner happier" is ruled out per se.  Which person is happier cannot be expressed in the language of pairwise comparisons, which allow you to claim only that "John prefers jazz to rap" and so on.  "Let Tom and John trade" also won't do, whether or not transactions costs are high.  That's not a social welfare function as Arrow defined it.

I sense the above paragraph isn't a very charming popularization (it wouldn't make the Freakonomics movie), but let's continue.

That's a two-person example.  With 17 persons, Arrow's clever method of partitioning shows that conflicted decisions ultimately must break down to one-on-one comparisons of different individual preferences, returning us to the island/radio example.

For a truly popular audience, you could go on about the difference between inter-profile and intra-profile social welfare functions.

I believe that a popularized version of Arrow's theorem, which explains where the main result actually comes from (strong IIA), is not very impressive to most people.  That may be one reason why the theorem doesn't get popularized so much.

Here is one very good transparent version of a proof, which requires virtually zero mathematics.  If you work through this paper, you will understand the theorem and where it comes from.  Here is a related journal article.

Comments

um, so the way I'm understanding this is that it says there are situations in which no negotiation is possible.

Negotiation is not possible by voters. Voters elect representatives that have the power to negotiate... but would Democrat voters negotiate with Republican voters, for example?

Now that I think it through I realize that the impossibility of negotiation is one of the conditions here, not the implication.

Keep in mind that Arrow's theorem applies ONLY to the specific kind of voting that arrow considered. It does not preclude other methods maximizing a welfare function.

Guru Jazzmatazz volume 1 & they're both happy!

"With 17 persons, Arrow's clever method of partitioning shows that conflicted decisions ultimately must break down to one-on-one comparisons of different individual preferences, returning us to the island/radio example."

This is the hard part. Intuitively I would expect the law of large numbers to likely sort things out, but apparently not.

I skimmed Don Boudreaux's lecture but he doesn't seem to explain in it why things hold when N is large.

I still do not think that the average person (or I) can understand exactly what Arrow showed to be impossible. That is the hard part of the theorem. What, exactly, is impossible?

Mike Munger does the best job of popularizing the theorem by connecting it to a prediction: democracies (in the sense of voting on everything without limits) are inherently unstable because no matter which party is in charge, there is also some coalition which forms a majority and wants the dominant party out - at least assuming that the party in charge doesn't already have a majority.

Heard at an IHS seminar.

Nick, I was going to nominate Digable Planets.

Cliff,

All you have to do is play a game of Republic of Rome to confirm the power of agenda setters.

"[...] group decisions represent outcomes of certain agreed-upon rules for choice after the separate individual choices are fed into the process. There seems to be no reason why we should expect these final outcomes to exhibit any sense of order which might, under certain definitions of rationality, be said to reflect rational social action."

"Arrow seems to suggest, implicitly, that such social rationality is an appropriate criterion against which decision-making rules may be judged. [...] For a more extensive critique of this aspect of the Arrow work along the lines developed here, see James M. Buchanan, “Social Choice, Democracy, and Free Markets,” Journal of Political Economy, LXII (1954), 114-23."

Read the linked paper I really cannot fathom the need for the concept of neutrality and any extension of it as being a requirement that cannot be violated. If I like apples over oranges and bananas over strawberries but am indifferent between the two sets then if I get apples and strawberries sure I am only 50% satisfied but would expect that to be a reasonable outcome. For better are worse this is seems to be a common situation in U.S. politics that results in "vote trading" as a resolution form to overcome many of the "impossiblility" situations that the theory predicts.

Also, the fact that the outcome resulted in a dictator does not mean that the dictator caused the outcome - which is, in my opinion, the bigger concern. As long as the next vote is not guarenteed to go the dictator's way then the fact that the current vote did just means that someone "got lucky" in that all their non-indifferent preferences were met.

I, like others, am admittedly missing something along the lines of "what is this trying to actually prove". Cliff's point is probably the most useful in that when dealing with a ranking system relegating the results to a series of binary votes can result in situations where a "fair" outcome is impossible. My conclusion, then, is that a negotiated resolution is necessary in most situations in order to obtain "fair" outcomes.

Popularizing the notion that there are no perfect voting systems would not be very popular.

I think the growing use of plurality with runoff is a huge step in the right direction. Although not as efficient at the Haare or Coombs methods, plurality with run-off is almost as efficient, easily implemented, and easily understood.

Plurality with run-off would have spared Chile from social/political/economic upheaval and murder under Allende and Pinochet. Good voting rules stop civil wars.

Showing people how logrolling in a representative democracy can result in policies which are opposed by a majority might bring earmarks and "omnibus" bills to an end.

You mean its basically a long-winded way of saying that in a society where people have differing preferences not every can be happy when it comes to group decisions...

The "size of the market" has nothing to do with Arrow's proof.

It's easiest to see in the Condorcet paradox, which is sort of the kernel of Arrow's proof anyway.

Suppose we have three people and three options:

1: A>B>C
2: B>C>A
3: C>A>B

Imagine that we want to have a "voting system" where these agents submit a report of their preferences and the result will be (1) Pareto optimal, meaning that no one can do better without anyone doing worse and (2) We aren't simply picking some individual's preferred alternatively (making someone a dictator).

Well, we might as well do three run-off votes, like a tournament. If the agents are honest (big if), then putting A against B means A wins (1 and 3 vote for A, 2 votes for B). Putting up A against C means C wins (2 and 3 vote for C, 1 votes for A). And putting up B against C means B wins (1 and 2 vote for B, 3 votes for C).

But this means that B beats C, C beats A, and A beats C. So... A beats C beats B beats A... etc. There aren't any "group preferences" emerging, just a cycle. This could be resolved by running a lottery where each person has a 1/3 chance of winning and letting the winner just pick an alternative, but that is just resorting to dictatorship.

Arrow's theorem basically says that this "misbehavior" is generic. Any voting rule will suffer its own kind of Condorcet paradox, making Pareto optimality and dictatorship clash.

The literature then began looking for "possibility" results by adding strategic behavior and restrictions on the classes of preferences that were considered. For instance, there are "single-peaked preferences" and the like, that give the possibility of "nice" voting rules.

Your island example is poor, as it is not covered by Arrow's theorem. With two individuals and only two alternatives, weak Paretianism and independence of irrelevant alternatives does NOT imply dictatorship.

It is the notion of 'dictator' used in this proof that most puzzles me. It doesn't seem to conform to the standard one. The fact that one person in the population happens to have preferences that coincide with 'society's' doesn't make him a dictator, it makes him the lowest common denominator. No?

Arrow's theorem gets proved by mathematicians. They naturally try to see how weak they can make the conditions and still have the conclusion hold, and so they weaken them to things that are hard to understand.

Here are some criteria that tend to lead to Arrow's Theorem being true:

1. Imagine that you have a US election with Bush, Clinton, Nader, and Perot running. Nader takes votes from Clinton and Perot takes votes from Bush. Clinton and Bush are both way ahead, but which of them wins is determined by which of Nader or Perot siphons more votes away. This is a terrible system. There ought to be a way so you can vote for the third party guy and still have your vote count between the two major candidates.

If we had a decent voting system we might have avoided 8 years of Clinton AND 8 years of Bush! The IIA principle basicly says that third parties shouldn't affect the ratio of votes between the main parties.

2. Given two alternatives X and Y, where more voters prefer X over Y, X should come out ahead of Y. (I may not have this right, but I think they say that if you can have Y come out ahead when the majority prefers X over Y, then you can find a case where Y comes out ahead even when *everybody* prefers X over Y. )

3. If you find a conflict that prevents you from doing 1 or 2 above, one way to make it work is to throw out some of the votes until you get rid of the problem. We don't want to throw out any votes. It turns out if a voting system has a problem with 1 or 2 above, it can still have the problem when there are only two votes. So to use that method to make sure to avoid the problem, you might have to throw out all but one vote -- the "dictator". I am not sure that's actually true -- you can have problems with 2 votes, but maybe there are voting systems where for large N you never have to throw out more than N-3 votes provided you get to pick which 3 votes to keep. But who really cares? The point is, you don't get to throw out votes.

And the theorem says there are always potential problems. We saw an unacceptable problem with simply voting for one of 3 or more candidates, the US system. Here's an alternative, called IRV (for Instant Runoff Voting).

---------
Everybody votes for as many candidates as he wants, in his order of preference. So, say that there are three candidates, Democrat, GOP, Green. One vote might be Green, then Democrat. Call it Vote 1. Vote 2 could be Green, then Republican.

When the votes are cast, each vote counts for its first choice. So Vote 1 is for Green and Vote 2 is also for Green. And say that when all the votes are counted, it goes

435 Democrat
450 Republican
115 Green.

Then the Green candidate is eliminated and his votes are redistributed. Vote 1 switches from Green to Democrat, the second choice. Vote 2 switches from Green to Republican.

After the Green votes have changed, the numbers then are

535 Democrat
465 Republican.

This would mean that 100 Green voters had Democrat for their second choice, and 15 had Republican second.

The result is, you get to vote for as many third party candidates as you want, and voting for them does not keep your vote from counting when they lose. A big improvement. And if people can vote for them without losing their vote, then a lot more people might vote for them. They might win.

But this can fail in subtle ways. For example:

It ought to be that the Democrats and Republicans should come out in the same ratio whether the Greens run or not. But try this contrived example:

301 vote straight Democrat.
400 vote straight Republican.
299 vote first Green, second Democrat.

This is a Democrat win, 600 to 400.

But what if instead it goes:

301 vote straight Democrat.
350 vote straight Republican.
299 vote first Green, second Democrat.
50 vote first Green, second Republican.

Then Republican wins against Green, 350 to 349.

Voting for Greens has changed the balance between Democrats and Republicans. There were 600 votes for Democrats and 400 votes for Republicans and Republicans won -- with a minority -- because of the votes for Greens.

This will not happen very often. Whenever you have the same two candidates left at the end, all the votes for those two parties will count. So it doesn't really matter what order the minor parties get removed, provided it doesn't change the final runoff. But that could happen.

Some people have suggested that this flaw allows "strategic voting". People would vote for candidates they don't want, to manipulate the balance. But that is hard to do. If slightly too few Republicans vote Green it doesn't work. If slightly too many do it, they horror-of-horrors make Green win. Democrats can stop them by voting Green second. To make it work you need both superb prediction of everybody's votes, and also great coordination among your own voters.
--------------

There are more complicated systems which are even less likely to give bad results.I prefer IRV because it's simple, and the simple clear procedure does not really allow anyone to manipulate the results.

There are people who say that we absolutely must have the best voting system, and Condorcet is the best, and we have to stay with what we have until the voters are ready to switch to Condorcet. Some of them prefer the current deeply flawed system, and they are really arguing against any change. They are trying to be strategic, to split the opposition to the status quo into tiny irrelevant splinter groups.

I would propose instead that everybody who opposes the current system should work together to get it changed. And whoever votes on changing the system (voters of each state? legislatures of each state? national referendum?), the vote-counters should count the votes according to every different voting system under consideration, and switch to the voting system which wins according to the majority of the new ways to count the vote. I predict the majority of voting systems will agree on that particular occasion. And if Condorcet wins that's fine, I won't mind at all.

Every alternative that is under serious consideration is an improvement. Even acceptance voting. We don't need to argue about which is the very best, we need to get something workable quickly.

Eight years of Clinton. Eight years of Bush. If it takes 16 more years to get a better voting system, who knows what might happen?

BTW, I deny the existence of well-defined, fixed, individual preference relations. People can change their preferences. In particular, people can (and do) change their preferences given data such as - other peoples' preferences. So the axioms of this theorem do not apply to humans.

Sonic Charmer,

That's cute. You must have trouble picking out socks in the morning.

Baphomet says above:

"Your island example is poor, as it is not covered by Arrow's theorem. With two individuals and only two alternatives, weak Paretianism and independence of irrelevant alternatives does NOT imply dictatorship."

As I've said to you in email, Baphomet is exactly right.

Here is a social welfare function that works in the island case: If Tom and John have identical preferences, those are the preferences of the island. Otherwise, the island prefers jazz.

(So in terms of what we'll tune the station to: If Tom and John agree, tune to whatever they agree on; otherwise tune to the jazz station.)

This social welfare function satisfies all of Arrow's assumptions and is not dictatorial, so Arrow's theorem cannot apply to your example. The problem, as Baphomet says, is that the theorem does not apply in the case of two voters and two options. So your example cannot illustrate the theorem.

I was referring to this:

"BTW, I deny the existence of well-defined, fixed, individual preference relations. People can change their preferences. In particular, people can (and do) change their preferences given data such as - other peoples' preferences. So the axioms of this theorem do not apply to humans."

If your preferences are so mercurial that you can't participate in a vote, I just thought you might have trouble with other things, like deciding to get out of bed or not... standing in front of the open refrigerator oscillating back and forth from OJ to milk to prune juice... going to the garage and not being able to pick whether to decide to ride a bike or drive because of what people might think of you... It must be hard being a human.

abel,

If your preferences are so mercurial that you can't participate in a vote,

Having preferences that are contingent doesn't mean you "can't" participate in a vote (or, more generally, a choice). It just means that when you do participate in a vote/choice, that choice is not well-described by a 'preference relation' of the form that is prescribed by the properties postulated to be held by a preference relation. In other words: the preference relation axioms aren't a good model for how humans actually form preferences, therefore, theorems which follow from those axioms don't apply to human society.

As an example, suppose you would prefer this movie to that movie. Then you find out your date wants to see that movie. You think about it, and it turns out, you decide you want to go see that movie after all. Etc. The point being, 'preferences' can change. This is perfectly normal. It also would be impossible if everyone actually had a static 'preference relation' xRy that obeyed the axioms as laid out in the paper linked above...

Not sure what socks or prune juice has to do with any of this.

The state of the world determines my preferences,

Which was my point (though oddly, your prune juice example doesn't illustrate this, because it has you only 'pretending' not to prefer prune juice). If the state of the world can influence your preferences then you don't have a 'preference relation'. Or rather, your preference relation depends on the state of the world, thus doesn't satisfy the axioms of Arrow's theorem.

Comments for this post are closed