I vote that this post deserves a follow up post with more clarification. If anyone is against this please express your vote with inaction. Us laymen would like to understand this a little better. For the record, the reason I got on a tangent about the law of large numbers was that I watched Boudreaux's lecture and understood it in terms of 3 parties but kept thinking if there were 3000 parties it was unlikely that exactly 1000 would have preference A, 1000 preference B, and 1000 preference C. I guess I'm used to thinking in terms of run-off elections and not the sort in the example. That is why I couldn't grasp why things should "collapse" back to an island situation where n = 2 or 3.
Return to the oft-neglected difference between intra-profile and inter-profile versions of the theorem. Most commentators and expositors have in mind an intra-profile version of the theorem. They set up an example of people and preferences and show how cycling or some other paradox of choice or voting is possible. Observers then wonder whether this cycling is likely as the number of people increases, or as preferences change, and indeed sometimes it is not, as Gordon Tullock pointed out long ago and as Dirk above wonders.
That's interesting stuff, but those fun and practical-sounding expositions are not Arrow's theorem as Arrow wrote it up. Think of Arrow's theorem as modal in nature: "Maybe there is no paradox with current preferences, but there exist possible preferences where everything goes screwy, under any decision rule satisfying a few criteria." Arrow showed that claim is related to something like: "if we apply a specified decision-making procedure across all possible preference configurations, consistent application means the same person gets her way each time."
That's called Arrovian dicatorship, but it does not have to be either harmful or unjust or not even necessarily undemocratic. It just means that one person — the same person — is always getting her first choice, across these modal worlds with differing preference configurations.
This more metaphysical and more originally Arrovian version of the theorem is perhaps why Arnold Kling finds it difficult to apply the theorem to practical problems. It is not about the likelihood or relevance of cycling (though it is a jumping-off point for those analyses). It is instead a deep result about the implications of consistency, combined with limited information about the value of ordinally ranked outcomes.
The intra-profile versions are still important. For intra-profile versions of Arrow, start with Kemp and Ng (1976). Here is a good summary article on that literature. Samuelson, by the way, remained somewhat recalcitrant when it came to the theorem.
Allowing in even limited amounts of interpersonal comparability defuses the paradox, as shown by Kevin Roberts (ReStud, 1980) and Amartya Sen (see the essays in Choice, Measurement, and Welfare). That said, interpersonability can lead to other paradoxes, as shown by Derek Parfit and his Repugnant Conclusion. Paradoxes everywhere, and you must choose which ones to live with.
I take the practical upshot of Arrow's interprofile theorem to be this: when you make a judgment, it is our assessment of the interpersonal comparisons (or intersport importance comparisons, for scoring a decathlon) which is doing all the work. Be very careful with those.
Neither Tullock nor Samuelson was happy with Arrow's theorem, especially when it came to practical implications, so it is fine if you wish to add your name to that list. But I also think they each missed Arrow's point a bit and that of the major economists of his time he was probably the deepest thinker, albeit not the best practical thinker.