Claude 3 Opus Also Fails Steve Landsburg’s Economics Exam
Almost one year ago, Steve Landsburg tried GPT-4 on one of his exams. It failed, badly. I tried out some of the same questions on Claude 3 Opus, by many accounts now the leading AI. It failed, badly. Steve’s exams are very clever. They aren’t technically difficult but they are tricky in the sense that the answer you would get by reading the textbook is almost certainly wrong. Not because the textbook is wrong but because you have failed to really understand. Here’s two examples:
Q. Apples are provided by a competitive industry. Pears are provided by a monopolist. Coincidentally, they sell at the same price. You are hungry, and would be equally happy with an apple or a pear. If you care about conserving societal resources, which should you buy?
A. If you care about conserving societal resources, you should buy the apple provided by the competitive industry rather than the pear provided by the monopolist. Here’s why:
In a competitive industry, firms are price takers and must operate efficiently to survive. They produce at the point where marginal cost equals price. This means they are producing at the socially optimal level, using society’s resources efficiently.
In contrast, a monopolist has market power and can set prices above marginal cost. They produce less than the socially optimal quantity in order to drive up prices and maximize profits. This results in a deadweight loss to society – there are potential gains from trade that are not being realized.
Even though the apple and pear are selling at the same price to you as the end consumer, the way that price was arrived at is very different:
- The competitive apple price reflects the true marginal cost of production
- The monopoly pear price is above the marginal cost of production
By buying the competitively produced apple, more of your money is going towards covering the actual costs of production. By buying the monopoly pear, some of your money is going towards monopoly profits rather than productive use of resources.
Of course, this is a simplification and there are other considerations, like quality, variety, innovation incentives, etc. But in general, competitive industries lead to more efficient outcomes than monopolies. So if your goal is conserving societal resources, buying from competitive producers is usually the better choice. The apple is the way to go in this case.
Claude has read the textbook. It knows that competitive industries are good and that monopolies are bad and it can explain why. Everything it says is correct and yet Claude gets the answer wrong. Do you see why? Pause and check your solution.
In Claude’s defense, I was able to prompt Claude via the Socratic method to the correct answer but it took several rounds of questioning:
The social cost of producing a good is the opportunity cost of the resources used in its production. In a competitive market, price equals marginal cost, so the price reflects the social cost of production. However, in a monopoly market, the price is higher than the marginal cost, so the price does not reflect the social cost.
Given:
- The price of apples (Pa) equals the price of pears (Pp)
- Apples are produced competitively, so Pa = MCa
- Pears are produced by a monopolist, so Pp > MCp
Therefore, MCa = Pa = Pp > MCp
The marginal cost of producing a pear (MCp) is lower than the marginal cost of producing an apple (MCa).
From society’s perspective, the cost of producing an additional pear is lower than the cost of producing an additional apple. The resources needed to produce a pear (MCp) are less than the resources needed to produce an apple (MCa).
Therefore, if you want to minimize the societal cost of your fruit consumption, you should choose the pear. By consuming a pear, you are using up fewer societal resources than if you consume an apple.
The monopoly pricing of the pear is a separate issue from the social cost of production. The higher price of the pear reflects a transfer from consumers to the monopoly producer, but it does not affect the underlying cost of the resources used to produce the pear.
So in conclusion, if your goal is to minimize the cost to society of your fruit consumption, you should buy the monopoly produced pear, as it has a lower marginal cost of production than the competitively produced apple, despite being sold at the same price.
A good answer and very clearly written with nice notation so Claude can “learn” within a session. Still, this isn’t very useful if you don’t already know the answer. Ok, one more.
Q. The town of Mayberry is thinking of expanding its airport. One problem with the expansion is that it would result in more airplane noise. For people who live near the airport, hearing that noise would cause as much unpleasantness as the collective loss of $100,000 every year. True or False: When Mayberry weighs the costs and benefits of its airport expansion, that $100,000 should count as a cost
A. True, the $100,000 in collective unpleasantness experienced by residents near the airport due to increased noise should be counted as a cost when weighing the costs and benefits of the airport expansion. Here’s why:
Externalities: The airplane noise is a negative externality – a cost imposed on third parties (the nearby residents) who are not directly involved in the economic activity (the airport expansion). Externalities lead to market inefficiencies because the full costs are not being accounted for by the decision-makers.
….a bunch more similar reasoning, all correct.
Once again, Claude gives a textbook answer and yet the answer is wrong. In a way this is more surprising than failing the first question because Coase got a Nobel prize for giving the correct answer (and it’s not, bargaining will solve the externality if that is what you are thinking.) I will let commentators work this one out. Do read the question carefully, it’s subtle. Again with Socratic prompting Claude got there eventually.
I wouldn’t underestimate the GPTs, textbook answers can be of great value, especially when the textbooks are long and diverse. Medical diagnostics, legal reasoning and coding are ideal tasks for GPTs. Economic reasoning less so, at this stage.
Congratulations to Steve Landsburg for the excellent questions.
Claude 3 Opus does Austrian economics
TC: Let’s say you were Peter Boettke, and looking to pen a critique of Kirzner’s theory of entrepreneurship. You come from a slightly different branch of the Austrian school. How would you use that differential background to express your differences with the Kirznerian theory, which emphasizes alertness above all else as an entrepreneurial characteristic?
“If I were Peter Boettke, coming from a slightly different branch of the Austrian school, I would likely emphasize the following points in my critique of Kirzner’s theory of entrepreneurship, which focuses heavily on alertness:
- The role of uncertainty: While Kirzner’s theory emphasizes alertness to opportunities, it may not fully capture the fundamental role of uncertainty in entrepreneurship. Drawing on the work of Ludwig von Mises and Friedrich Hayek, I would argue that entrepreneurship is not just about spotting opportunities, but also about bearing uncertainty and making judgments about the future in the face of incomplete information.
- The importance of capital and resources: Kirzner’s theory focuses on the cognitive aspect of entrepreneurship (alertness), but it may neglect the importance of access to capital and resources in enabling entrepreneurs to act on their insights. As a Boettke coming from the Rothbardian tradition, I would stress the crucial role of savings, investment, and capital accumulation in facilitating entrepreneurial action.
- The institutional context: Kirzner’s theory tends to treat the institutional framework as given, focusing on how entrepreneurs operate within existing market structures. However, as a Boettke influenced by the work of Douglass North and other institutional economists, I would argue that the institutional context (property rights, legal systems, cultural norms, etc.) plays a crucial role in shaping entrepreneurial opportunities and incentives.
- The dynamics of competition: While Kirzner emphasizes the equilibrating role of entrepreneurship in driving markets towards efficiency, I would draw on the work of Hayek and others to highlight the dynamic, rivalrous nature of competition. Entrepreneurship is not just about alertness, but also about continuous innovation, experimentation, and adaptation in the face of changing market conditions.
- The subjectivity of opportunity: Kirzner’s theory tends to treat entrepreneurial opportunities as objective phenomena waiting to be discovered. However, drawing on the subjectivist tradition in Austrian economics, I would argue that opportunities are ultimately subjective and dependent on the perceptions, beliefs, and creativity of individual entrepreneurs.
In summary, as Peter Boettke, I would acknowledge the important insights of Kirzner’s theory, particularly its emphasis on the central role of entrepreneurship in driving market processes. However, I would also seek to enrich and extend the theory by drawing on other strands of Austrian thought, emphasizing the roles of uncertainty, capital, institutions, competition, and subjectivism in shaping entrepreneurial activity.”
That is from Claude 3 Opus.
Brazil fact of the day
Agriculture is now almost single-handedly propping up the nation’s economic fortunes. In the first quarter of 2023, the sector grew by an extraordinary 21 per cent and by more than 15 per cent across the year. This growth — the strongest since 1996, according to available data — helped bolster overall national growth, which surpassed expectations at 2.9 per cent.
A recent study of an elite group within Brazil’s richest people — those in the 0.1 per cent — found that the highest earners were from Mato Grosso, according to the Fiscal Policy Observatory at the Getulio Vargas Foundation. “In terms of state GDP per capita, today it is among the highest in the country,” says Sérgio Gobetti, an economist who authored the study. But he highlights that inequality is also growing, with the wealth heavily concentrated among the landowners — typically from Brazil’s south — and not the workers, usually from the country’s poorer north-east.
Here is more from Bryan Harris at the FT. Brazil’s trade surpluses are growing rapidly.
Monday assorted links
1. Claude 3 on Bach’s Goldberg Variations.
2. How raw milk became a political issue.
3. “First, we don’t find that increasing corporate competition driven by M&A is important for workers either through concentrating the market for the products the workers produce, which would in theory increase worker wages, or through concentrating the labor market, which would in theory decrease their wages.” Link here, Canadian.
DEI vs. the Chips Act
The Hill has a good op-ed by Matt Cole and Chris Nicholson on how labor and DEI regulations are strangling the CHIPS act. It’s somewhat over the top, failure is overdetermined, but this is an important op-ed and directionally correct.
The Biden administration recently promised it will finally loosen the purse strings on $39 billion of CHIPS Act grants to encourage semiconductor fabrication in the U.S. But less than a week later, Intel announced that it’s putting the brakes on its Columbus factory. The Taiwan Semiconductor Manufacturing Company (TSMC) has pushed back production at its second Arizona foundry. The remaining major chipmaker, Samsung, just delayed its first Texas fab.
This is not the way companies typically respond to multi-billion-dollar subsidies. So what explains chipmakers’ apparent ingratitude? In large part, frustration with DEI requirements embedded in the CHIPS Act.
…The law contains 19 sections aimed at helping minority groups, including one creating a Chief Diversity Officer at the National Science Foundation, and several prioritizing scientific cooperation with what it calls “minority-serving institutions.” A section called “Opportunity and Inclusion” instructs the Department of Commerce to work with minority-owned businesses and make sure chipmakers “increase the participation of economically disadvantaged individuals in the semiconductor workforce.”
…Handouts abound. There’s plenty for the left—requirements that chipmakers submit detailed plans to educate, employ, and train lots of women and people of color, as well as “justice-involved individuals,” more commonly known as ex-cons. There’s plenty for the right—veterans and members of rural communities find their way into the typical DEI definition of minorities. There’s even plenty for the planet: Arizona Democrats just bragged they’ve won $15 million in CHIPS funding for an ASU project fighting climate change.
…tired of delays at its first fab, [TSMC]flew in 500 employees from Taiwan. This angered local workers, since the implication was that they weren’t skilled enough. With CHIPS grants at risk, TSMC caved in December, agreeing to rely on those workers and invest more in training them. A month later, it postponed its second Arizona fab.
Now TSMC has revealed plans to build a second fab in Japan. Its first, which broke ground in 2021, is about to begin production. TSMC has learned that when the Japanese promise money, they actually give it, and they allow it to use competent workers. TSMC is also sampling Germany’s chip subsidies, as is Intel.
Intel is also building fabs in Poland and Israel, which means it would rather risk Russian aggression and Hamas rockets over dealing with America’s DEI regime. Samsung is pivoting toward making its South Korean homeland the semiconductor superpower after Taiwan falls.
…The CHIPS Act’s current identity as a jobs program for favored minorities means companies are forced to recruit heavily from every population except white and Asian men already trained in the field.
Remember that there is an Extreme Shortage of High-IQ Workers. The United States is big and rich and can afford to handicap itself in many ways but less so when it comes to high-end semiconductor manufacturing, the most difficult and complicated manufacturing process ever attempted by human beings. For that we want the Dream Team, the very best, chosen on merit alone.
Addendum: Scott Lincicome has a good overview of the problem.
TikTok divestiture
I’ve blogged this in the past, and don’t have much to add to my previous views. I will say this, however: if TikTok truly is breaking laws on a major scale, let us start a legal case with fact-finding and an adversarial process. Surely such a path would uncover the wrongdoing under consideration, or at least strongly hint at it. Alternately, how about some research, such as say RCTs, showing the extreme reach and harmful influence of TikTok? Is that asking for too much?
Now maybe all that has been done and I am just not aware of it. Alternatively, perhaps this is another of those bipartisan rushes to judgment that we are likely to regret in the longer run. In which case this would be filed under “too important to be left to legal fact-finding and science,” a class of issues which is sadly already too large.
How good a song is Quarter to Three?
You know, the 1961 #1 hit by Gary U.S. Bonds? I’ve been thinking about this question for months. I feel a good amount is at stake. If songs such as Quarter to Three (or done live with dancers) are still great, our assessment of early times rises considerably. But if they are dispensable throw-aways, the history of popular music (and film) in the earlier twentieth century needs to be rewritten.
What makes the song such a classic? Claude praises “the upbeat rhythm, engaging call-and-response vocals, relatable lyrics, catchy melody, historical context, and instrumental breaks,” but none of those seem quite scarce or special enough to elevate the tune to classic status. With a bit of prodding Claude also cited “raw, unpolished energy,” a genuine sense of fun, and “chemistry amongst the performers.” To that you might add a creative use of repetition and small, stepwise changes, plenty of syncopation, and the hooks are iconic. The use of echo and phase shifting looks to the future, and the shuffle-like groove drew on calypso influences and also ska. Nonetheless the chord structure, while effective, is hardly revelatory.
So I’m still wondering — if a song has that ineffable “something” — how much is that the product of our collective imaginations? How much is it real and objectively there? Or does a Generation Z teen, with a very different ear, dismiss it as muddled and mediocre rather than memorable? After all, Gary’s career was not replete with enduring creations.
A critic could allege the dance lyrics are ordinary and the production sloppy. But was that all part of the calculation? Wikipedia relates:
The single was recorded with very rough sound quality (compared to other records at the time). Producer Frank Guida has been quoted on subsequent CD reissues that his production sound was exactly what he wanted it to sound like.
Bob Roman wrote:
The song opens with muffled crowd noise and a bandleader counting off the beginning of a song. It’s not a live recording, but it sounds like one — and not even like a good one. It sounds like an amazing party happening down the street — wild, frenzied, mysterious, its sound obscured by what might as well be a couple of sets of walls. In any era, it’s crazy that a record this lo-fi managed to hit #1. In the pre-Beatles era where labels were pushing cleaned-up teenage dreamboats, it seems especially strange.
So we’ve got amazing hooks, controlled chaos, and extreme innovation?
The song also has a lineage. Bill Wyman put it on one of his solo albums. It inspired Dion’s “Runaround Sue.” Bruce Springsteen played it regularly in his concerts, and later worked with Gary, writing songs for him and doing two albums together. Most importantly, Paul McCartney references it in his Sgt. Pepper classic “When I’m Sixty-Four“:
If I’d been out ’til quarter to three, would you lock the door?
In essence Paul is teasing us with the notion that the 64-year-old McCartney might someday still be out there, dancing, rather than knitting tea cozies on the Isle of Wight. And true to Straussian form, Paul released the dance song “Dance Tonight” when he was sixty-four, days before turning sixty-five.
In 1963, during a Beatles European tour, Gary U.S. Bonds was the headliner for them.
You will note that the lineage of the song runs mostly through white performers, though Gary U.S. Bonds was black (or possibly mixed race). Perhaps one special feature of Quarter to Three is how it spans black and also white R&B, a rare feature at the time but hearkening back to the much earlier years of the blues, when black and white musical styles could be hard to distinguish. In addition to the Caribbean vein, Gary could span Latino styles as well.
Just as we are finding it impossible to rebuild Notre Dame cathedral as it was, a mere sixty-three years later could any of us still make something akin to “Quarter to Three”? Or have we lost those “technologies”?
I, for one, have decided to vote in favor of masterpiece status for Quarter to Three. At least for now. And by the way Gary U.S. Bonds is still on tour.
Sunday assorted links
1. The suburban YIMBY movement (NYT).
2. Chess Fever, a Soviet silent movie. 27 minutes, Buster Keaton-style.
3. Angus Deaton makes a nationalist turn.
4. Is Silicon Valley pricing academics out of AI research? (I hope so.)
5. List of names you cannot give your Icelandic daughter (sorry Abigail! Aisha eventually was approved, though). For men, they have banned Fabio, but not Elmer. I believe in laissez-faire for names, but if you are going to ban anything, surely Elmer is worth some consideration?
6. Are Florida voters tiring of the culture wars?
7. “Mr. Musk has not hired any staff for his foundation, tax filings show. Its billions are handled by a board that consists of himself and two volunteers, one of whom reports putting in so little time that it averages out to six minutes per week.” (NYT, quite possibly he is doing this well?)
It’s happening, economic science edition
“Our contribution is to demonstrate that it is possible to create a system without human input at any step – to simulate the entire social science process.” We’ve created a whole new method for doing economics, and a whole new way to model human behavior. How often does that happen? We don’t yet know how useful it will prove, but at the moment how many people even are noticing!? That is from Benjamin S. Manning, Kehang Zhu, and John J. Horton, of MIT, Harvard, and MIT. Worth a ponder, at the very least.
Which U.S. firms have grown in profitability?
China’s admission into the WTO in 2001 heralded a new era of globalization, increasing both import competition in domestic markets and foreign opportunities for US firms. In the aggregate, the average annual profitability of US public firms during the post globalization period (2003-2019) increased by 11.5% of the corresponding pre-globalization period (1984-2002) profitability. This increase in overall aggregate profitability was primarily driven by foreign profitability increasing by 47.4% for firms in the S&P 500 index, which are larger and have more intangible assets created by R&D and SG&A expenditures. In contrast, following globalization, the average aggregate domestic profitability of US firms remained flat, and firms employed more capital to generate sales. Firms with higher intangible assets benefited more from globalization.
That is from a new NBER working paper by Bullipe R. Chintha, Ravi Jagannathan, and Sri S. Sridhard. When Average is Over was published about eleven years ago, in talks and media appearances I used to commonly draw a distinction between people/firms who are exporting their products — yes economists too — and those who are not. Which category do you belong to?
What are the exact elasticity assumptions behind the terms of trade argument [for tariffs]?
“The terms of trade argument for protectionism relies on specific assumptions about the elasticities of demand and supply for a country’s exports and imports. The key elasticity assumptions are:
- Inelastic foreign demand for the country’s exports: This means that when the price of the country’s exports increases, the quantity demanded by foreign countries does not decrease much. In other words, the percentage change in quantity demanded is smaller than the percentage change in price.
- Elastic domestic demand for imports: This means that when the price of imports increases (due to a tariff), the quantity of imports demanded by the domestic country decreases significantly. In other words, the percentage change in quantity demanded is larger than the percentage change in price.
Under these elasticity assumptions, a country can improve its terms of trade by imposing a tariff on its imports. Here’s how it works:
- The tariff raises the domestic price of imports, leading to a decrease in the quantity of imports demanded due to the elastic domestic demand for imports.
- The decrease in import demand leads to a decrease in the world price of the imported goods (assuming the country has a significant share of world import demand).
- The country’s exports are not affected much by the tariff because foreign demand for the country’s exports is inelastic.
- As a result, the country experiences an improvement in its terms of trade: the price of its exports relative to the price of its imports increases.
However, it’s important to note that these elasticity assumptions are not always met in the real world. The elasticities of demand and supply can vary depending on the specific goods being traded, the time horizon, and other market conditions. Moreover, even if a country can improve its terms of trade through tariffs, this does not necessarily imply that the tariffs are beneficial for the country’s overall economic welfare, as they can lead to distortions in production and consumption, as well as potential retaliation from trading partners.”
That is from Claude 3 Opus.
The Irish reject a “Woke” constitutional change
Ireland’s effort to remove old-fashioned family values from its constitution suffered a double defeat Saturday as voters rejected the amendments on offer as maddeningly vague and threatening to property rights…
In final results announced Saturday night, the amendment to change the constitutional definition of family was rejected by 67.7 percent of voters. The proposed changes on family care took an even harsher drubbing, with 73.9 percent against — the greatest defeat of an amendment in Irish constitutional history…
The outcome means that the 1937 constitution, the legal bedrock for the Irish state, will continue to declare marriage a requirement for any family, while women’s value to society comes from delivering “duties in the home.”
Those notions from a bygone era contrast starkly with the reality of Ireland today, where two-fifths of children are born out of wedlock and most women work outside the home.
The government, with support from all the main opposition parties, had wanted the public to accept two amendments.
Here is the full story, via Rich D.
Saturday assorted links
1. Why high drug prices are good for Americans.
2. Open access Sanford Ikeda book on Jane Jacobs.
3. “He’s just a complete parliamentary obsessive and savant, really like no one I’ve ever met, even people in the parliamentarian’s office…” He is a twenty-year-old economics student from Britain, born in Poland. Good piece.
4. OpenAI review is completed.
5. Taylor Swift tour prompts economists to upgrade Singapore growth forecast (Bloomberg).
6. John Cochrane on increasing the number of women in economics.
In Conversation with Próspera CEO Erick Brimen & Vitalia Co-Founder Niklas Anzinger
During my visit to Prospera, one of Honduras’ private governments under the ZEDE law, I interviewed Prospera CEO Erick Brimen and Vitalia co-founder Niklas Anzinger. I learned a lot in the interview including the real history of the ZEDE movement (e.g. it didn’t begin with Paul Romer). I also had not fully appreciated the power of reciprocity stacking.
Companies in Prospera have the unique option to select their regulatory framework from any OECD country, among others. Erick Brimen elaborated in the podcast how this enables companies to do normal, OECD approved, things in Prospera which literally could not be done legally anywhere else in the world.
…so in the medical world for instance you have drugs that are approved in some countries but not others and you have medical practitioners that are licensed in some countries but not the others and you have medical devices approved in some countries but not others and there’s like a mismatch of things that are approved in OECD countries but there’s no one location where you can say hey if they’re approved in any country they’re approved here. That is what Prosper is….Our hypothesis is that just by doing that we can leapfrog to a certain extent and it’s got nothing to do with the wild west or doing weird things.
…so here so you can have a drug approved in the UK but not in the US with a doctor licensed in the US but not in the UK with a medical device created in Israel but not yet approved by the FDA following a procedure that has been say innovated in Canada, all of that coming together here in Prospera.
As it should be
The next time you send your doctor an email, don’t be surprised if they charge you a fee to answer.
More healthcare groups are charging fees to answer patients’ electronic messages, often the ones you exchange via their portal. Doctors say it’s only fair if they’re spending time on the messages and note that an email discussion can often save you the time of having to come in.
The typical cost of an email message claim was $39 in 2021, including both the portion paid by insurance and by the patient, according to a Peterson-KFF Health System Tracker analysis.
Some patients have been taken aback by the charges. They are surprised at the notifications on portals about the change, and irritated at the idea of a new fee.
Dr. Lauren Oshman, a family physician and associate professor at the University of Michigan Medical School, says she initially experienced some patient resistance and anger about the prospect of being billed for emails.
Now, she says, patients are typically pleased that they are able to get a direct response from her through a portal message.
“They’re thrilled when they get me directly,” she says.
Here is more from the WSJ, via the excellent Daniel Lippman.