Interpreting Contracts via Surveys and Experiments

by on April 13, 2017 at 2:27 am in Current Affairs, Law, Travel | Permalink

By Omri Ben-Shahar and Lior Strahilevitz, both at University of Chicago Law School:


Interpreting the language of contracts is the most common and least satisfactory task courts perform in contract disputes. This article proposes to take much of this task out of the hands of lawyers and judges, entrusting it instead to the public. The article develops and tests a novel regime — the “survey interpretation method” — in which interpretation disputes are resolved though large surveys of representative respondents, by choosing the meaning that a majority supports. The article demonstrates the rich potential under this method to examine variations of the contractual language that could have made an intended meaning clearer. A similar survey regime has been applied successfully in trademark and unfair competition law to interpret precontractual messages, and the article shows how it could be extended to interpret contractual texts. To demonstrate the technique, the article applies the survey interpretation method to five real cases in which courts struggled to interpret contracts. It then provides normative, pragmatic, and doctrinal supports for the proposed regime.

Just to be clear, I do not favor such a regime, but I think it is what we will be getting.

For the pointer I thank William the Irishman.

1 Thanatos Savehn April 13, 2017 at 3:06 am

Tyler – “Let’s have lay people, who follow not some blog about the UCC but rather tweets by a Kardashian publicist about Kardashian back sides, decide what contracts mean” – sure, why not? It’s a post-rational, post-modern world after all. What could possibly go wrong? And even if it does the Singularity will sort it all out.


2 kevin April 13, 2017 at 7:56 am

While not ideal, as recently highlighted, neither is the status quo where we are presented a 37000 word disclaimer we are expected to check before being allowed to buy airline tickets. If the public doesn’t understand what they are signing (clearly, in many cases they don’t) such a policy may force companies to write more clear/concise ToC, which on net I think would be a good thing


3 JWatts April 13, 2017 at 9:08 am

Yes, I agree. Having Legal experts write contracts that can only be understood by other Legal experts, but asking the general public to interpret and accept/decline them, is a terrible solution.

Also, reading from the post, they are not using the lay public write them, just that a “large surveys of representative respondents” will be used to resolve disputes. If a company writes a contract that the majority of it’s customers can’t understand, then I’m inclined to let them deal with any negative fall out. Write better contracts.


4 Anon April 14, 2017 at 2:20 am

How often is that type of contract the subject of a legal dispute? Aren’t most lawsuits between commercial entities where there were lawyers on both sides?


5 Thanatos Savehn April 13, 2017 at 12:13 pm

I have some experience with juries in states like California that let them express the public understanding via the “consumer expectation test”. Most people who show up for jury duty believe that in a commercial transaction the consumer is owed the entirety of the bargain and the only thing the Seller gets is an unlimited duty of omniscience and omnipotence to be exercised to the benefit of the consumer.


6 LearnedHand April 13, 2017 at 6:59 pm

First time commenting so I feel obliged to say – thanks TC for this blog, its one of the treasures of the internet!

@Thanatos Savehn

Yes, I agree entirely.

The mode individual in a random sample of society would likely be a consumer and risk aversion among consumers in a survey/experiment based approach to contractual interpretation would, in my mind, be likely to flip caveat emptor on its head.

@ Tyler Cowen – How do you feel about caveat emptor as a principle? I would like to know what others think.


7 prior_test2 April 13, 2017 at 3:18 am

‘Interpreting the language of contracts is the most common and least satisfactory task courts perform in contract disputes. This article proposes to take much of this task out of the hands of lawyers and judges, entrusting it instead to the public.’

Contract law as mob rule – sounds like a real step up from the current system.

‘Just to be clear, I do not favor such a regime, but I think it is what we will be getting.’

Yes, mob rule is always a safe prediction when talking about how a polity will make decisions in its future.


8 Roy LC April 13, 2017 at 5:28 am

I would be shocked to see this happen, though considering the state of contract law not particularly horrified.

When exactly in the history of our legal system has any democratization occurred in interpretation?


9 y81 April 13, 2017 at 6:24 am

I’m with Roy LC (and contra Tyler). Why is there any likelihood of such a system being adopted? Is there some populist groundswell out there against judicial interpretation of contracts?


10 MTC April 13, 2017 at 9:32 am

See Tyler’s earlier posts on the United incident; he basically argues the social media-fueled backlash against the airline is one of many instances we will continue to see of a “populist groundswell against judicial intepretation of contracts.”



11 Alan April 13, 2017 at 6:36 am

But how will the laity obtain justice without the intervention of the priests?


12 rayward April 13, 2017 at 6:59 am

From reality tv to reality courts of law. I suppose there’s some logic to it. I’m not sure what “problem” the authors are attempting to identify or to resolve. Is it poor draftsmanship by lawyers or poor textual interpretation by judges? “Interpreting the language of contracts is the most common and least satisfactory task courts perform in contract disputes.” Least satisfactory in what sense? Most judges don’t have a background in corporate law or contract drafting, as most have a background in litigation (drafting briefs rather than contracts, advocacy rather than intent). Of course, context (intent) is everything; and by separating context (which is what witnesses are for) from textual interpretation, one might as well have monkeys interpret the text. I’m reminded of a certain school of statutory interpretation, in which text is interpreted not according to the context (what the authors of the text meant) but what the founders would have meant if they had written the text. Having said all that, I suspect the “problem” the authors are attempting to resolve is complexity: to encourage lawyers to use simple language in contracts that any layman could understand rather than legalese. I might agree if the contract is with a layman (such as a contract between an airline and a passenger), but complex commercial transactions require complex contracts. To give a concrete example, such contracts often contain detailed definitions of important terms, and use the term in the contract according to its definition not the term’s everyday meaning. How would a monkey deal with that? As for law professors (the authors are law professors), I don’t know them or their background, but many if not most law professors at elite schools such as Chicago don’t have much practice experience, as they go from law school to judicial clerkship to teaching and writing; they are academics not practitioners. Indeed, contract drafting is not even taught at most law schools; drafting briefs, which are advocacy (one party’s intent), for sure, but not contracts (which should reflect the parties’ mutual intent). And even that is often taught by part-time professors or students. I’ll end this long comment with criticism of practitioners. There are two types of contract draftsman: the draftsman who attempts to reflect the parties’ mutual intent and the draftsman who attempts to reflect her own preference (i.e, to slant every word in the text to the client’s advantage). I’m all in on sending the latter to peoples’ court!


13 Jack April 13, 2017 at 7:24 am

Typical academic article — first contrive a problem, “least satisfactory.” Probably 99.99 percent of contracts never end up in litigation because the parties or their lawyers can predict what will happen if it does. Who knows what results this proposed rule would produce? Most likely a lot more uncertainty.


14 Mark April 13, 2017 at 7:30 am

A solution in search of a problem. Classic example of Ivory towerism. If you were to go ask representatuve participants in contract dispute resolution which method they would prefer, this idea would not have much traction. Then, ask those of us who are non-participants if we want to be bombarded daily with communications inviting / requiring us to adjudicate contract disputes, would we find this a gain in welfare? This is a phenomenally un-utilitarian proposal.


15 Boonton April 13, 2017 at 8:48 am

“If you were to go ask representatuve participants in contract dispute resolution which method they would prefer…”

That’s a tiny self selected sample. Most contracts do not end up in disputes. Once you are in a contract dispute you may indeed prefer a judge. Your lawyer might know how some judges operate and target their message accordingly. But if this resulted in contractual language being closer to common-sense, everyday usage, there might be fewer contract disputes and the incentive to try to game contract language by constructing difficult and obscure wording would be reduced.

That might cost those who make a living off contract disputes but provide a small benefit to many, many more people who use contracts everyday.


16 JWatts April 13, 2017 at 9:18 am

“A solution in search of a problem. Classic example of Ivory towerism.”

That’s exactly backwards! The current system is an ivory tower system. Where lawyers write contracts in tortuous legalese that no one but another expert in the field (living in the ivory tower) can interpret. And they occasionally don’t agree to the interpretation.

This takes Terms of Service and similar type contracts out of the Ivory Tower.


17 Bill April 13, 2017 at 8:17 am

I get to write the survey questions.


18 Jon April 13, 2017 at 9:07 am

I also don’t support such a scheme, but the problem it attempts to solve is a real one. For markets to function well, it’s important that people in the market clearly understand the contracts that they are entering into. It’s also important that people don’t need a huge resource expenditure in time or money in order to understand the contract in order to take advantage of a common product or service, as this represents a large transaction cost.

The United kerfuffle is a perfect example; the market would function more efficiently if all consumers fully understood the terms of their contract with airlines, but almost no flyers have the time or ability to understand thousands of words of legalese. Airlines would be able to compete on having more customer friendly terms if people actually understood the terms!

I’m not sure what the solution is, but it seems like a real problem.


19 ContractLawyer April 13, 2017 at 9:48 am

This is a terrible idea. We can’t have the general public resolving these disputes. A large company pays a team of highly compensated lawyers to write these contracts in the way that is most beneficial to the company. Letting the general public have a say would inevitably lead to a bias in favor of simpler language that was easier for the average High School graduate to understand. That would have a egregious economic impact, not only to the corporations involved, but to the contract lawyers they hire.


20 y81 April 13, 2017 at 9:52 am

“almost no flyers have the time or ability to understand thousands of words of legalese.”

True, but most people don’t have the time or the inclination to read thousands of words of plain English with respect to any consumer transaction. I can write a car rental contract, or a home mortgage, into plain English if it makes you happy, but you still won’t read it. And what if you did? Would you really evaluate airlines based on the bumping protocols specified in their contracts? Or would you choose the cheapest fare?

I really don’t think that contract interpretation is the issue in the United imbroglio, and it certainly isn’t the impetus for the cited article.


21 Jon April 13, 2017 at 10:15 am

Considering that it’s days later and people are still arguing about whether United had the legal and/or contractual right to kick the customer off the plane, I would say contract interpretation has *something* to do with the United imbroglio. The “common sense” understanding of airline seat assignment procedures is that you can get bumped at the gate, but once you’ve been assigned an actual seat and you’re sitting in it, it’s yours.


22 y81 April 13, 2017 at 11:45 am

But that “common sense” understanding has nothing to do with the words of the contract, whether they are legalese or plain English, or whether the contract is 1 page or 50. Yours is an argument for eliminating written contracts altogether, at least in a consumer context, and relying on popular understanding of what the transaction terms are. I don’t think that such a system will produce optimal results, but I could be wrong.


23 Jon April 13, 2017 at 12:11 pm

Well as I said initially, I *don’t* support such a system as is outlined in the post. But that doesn’t change the fact that a clearer understanding of the contract for all involved may have lead to a better outcome.

I mean, if the contract said in plain English “we may kick you off the plane at any time for any reason, knocking you unconcious if necessary,” then maybe the customer would’ve left the plane willingly. As it was, the customer had no conception that the airline believed they had the authority to do what they did, which is why he was defiant.

And again, people are *still* arguing about whether United actually had the authority to get cops to remove the man by force, so there’s obviously a clarity issue here. As I said, I’m not sure what the solution is.


24 Lior S April 13, 2017 at 9:56 am

I’m one of the authors of the paper. Thanks for flagging our research, Tyler. And thanks for the thoughtful comments above. It’s always hard to summarize a lengthy article in a short abstract, but if people are interested in engaging with our full argument, they can download it for free here:

The core case we are addressing is the problem of consumer contracts. These are mass contracts between large companies and their customers. The same language is used with thousands or millions of consumers, and there’s ample opportunity to vet the terms via surveys to make sure that the contract drafter and consumers understand the language at issue to mean the same thing. Several of the commenters above identify potential benefits of this approach and we discuss others in the full draft. To be clear, we are NOT suggesting that complex contracts between businesses (e.g., a merger agreement) should be interpreted via our method. If those sorts of contracts are to be interpreted via surveys they would need to be interpreted by surveys of experts in the relevant fields, and we talk a little bit in the paper about how that would be done and what some pitfalls would be.

Finally, a good chunk of the paper reports on the results of several surveys and experiments that we conducted to test consumers’ understandings of these sorts of contracts. We used language from contracts that were actually litigated, such as insurance contracts, employee bonus agreements, employee noncompete agreements, privacy policies from Facebook and Gmail, etc. Most of the time, a majority of American consumers who read the contracts agreed with the way courts had interpreted the contracts in question or they agreed with the court that language was ambiguous. (We used a census-weighted representative sample of adult US citizens with Internet access and paid people for their time.) In some significant cases, though, there was a wide divergence between what the judges thought the contract meant and what ordinary consumers thought it meant, and in none of these instances was the majority popular sentiment remotely nonsensical. We also showed precisely how randomly varying a word here and there in the contract could alter the meaning that a majority of consumers took from the language. This showed us that a good number of people paid reasonably careful attention to contract language, at least when we asked them to do so.

We’ve certainly vetted our arguments with contract lawyers — some love the idea and some reject it — but we are always interested in hearing more perspectives from people who work on these issues, so feel free to send any thoughts about the draft paper to us via email. You can reach me at my first name .


25 Ray Lopez April 13, 2017 at 10:07 am

April Fools? The closest you might get is to convince the drafters of the various “UCC” model statutes around the USA, which then become adopted by most states. Fat chance of that IMO. Asking trolls on internet sites is good maybe for publicity but that’s about it. You also have a constitutional problem in the USA: there’s no right to a jury trial (essentially what you propose) for state court civil actions, only for federal civil actions, see: (“To many Americans, jury trials seem to be the normal way of deciding civil cases. Television programs and movies show exciting scenes of juries deciding important non-criminal disputes involving individuals, government officials, and companies.The reality is different. Juries decide less than one percent of the civil cases that are filed in court. This lack of jury trials may seem strange, as the Seventh Amendment guarantees the right to jury trial in certain civil cases.”)

Good luck! One constant they taught us in law school: efficiency may rule the marketplace, but whenever efficiency creeps up in the legal system (such as for example the “Plain English” movement about a generation ago to make every contract ‘readable to the average person’ by law, such as in NY state), the lawyers conspire to complicate the law and defeat it. Same with “bright line” ‘black letter’ law movements, such as rules like rear-ending a driver will result in automatic liability for the party who rear ended the car, but not (the complications!) if the driver in front was not moving, or was backing up, or somehow did it on purpose (history of engaging in accidents to collect insurance)…” No bright line tests either. Lawyers always get paid. Like Heisenberg. Always remember that.


26 y81 April 13, 2017 at 11:50 am

I confess, I haven’t read the article, but does it address what seems to me like the real problem: most people don’t read consumer contracts (airline ticket agreements, car rental agreements, homeowner’s policies) at all. So the average person has no clear understanding of what he or she agreed to until a dispute arises.

Mind you, most of the parties to a commercial contract have not read it either, but at least their lawyers have. With consumer contracts, the only person who read it is the lawyer for the corporate supplier (and possibly one or more regulators, who have their own theories of what is good for consumers and the industry).


27 not a lawyer April 13, 2017 at 12:08 pm

There is an old principle in law, or so I’ve heard, that if you draw a distinct line in the sand between what is legal and illegal then people will tend to move as close as possible to the line without actually crossing it. Ambiguity has it’s virtues. One of the trade-offs with any increase in the definitions of legal terms might be removing discretionary power of interpretation from the court (the judge) and increasing the tendency to move as close as possible to the safe side of the line. (Note: Ray Lopez above mentions bright line, black letter law movements.) (On the other hand of course having clear language facilitates the drawing up of contracts. So there may be significantly different implications for corporate and criminal law. But I’m not a lawyer.)


28 rayward April 13, 2017 at 12:13 pm

I’ve shared this story before: I had drafted a contract for some commercial transaction and was meeting my counterpart to review it. He asked me why the contract had to be so long, as he flipped through the pages. I apologized for the length but explained that many of the provisions are what I call litigation avoidance provisions (more accurately, risk allocation provisions), contract provisions that would determine the outcome if something unexpected occurred. He gave me a puzzled look, looked down at the contract, flipped through some more pages, and then looked straight at me and said “Why would we want to avoid litigation?” Why, indeed.


29 rayward April 13, 2017 at 12:36 pm

I have a good friend who has more than once bragged about the quality of his lawyer, that of the half dozen or so contracts that he had drafted that ended up in litigation, he prevailed in the litigation in all but one case. .


30 y81 April 13, 2017 at 3:58 pm

As the saying goes, “If our clients don’t have problems, we don’t have clients.”


31 responsible D April 13, 2017 at 5:11 pm

In limited instances, the court of public opinion will get a say in whether, for example, an airline should have sought to strictly enforce its rights when looseing its purse strings just a tiny bit could have given greater satisfaction to a customer (not to mention avoiding a huge hit to good will).

But generally speaking it would boggle the average person’s mind if they knew how much contract verbiage governs every aspect of their lives, and how essential it is the structure of the society they live in. Most of it would be unfathomable without extensive explanation of context and purpose. Turning contract interpretation over to popular opinion is suicide.


Leave a Comment

Previous post:

Next post: