Category: Web/Tech

What should I ask Kevin Kelly?

by  Tyler Cowen in

From Wikipedia:

Kevin Kelly (born 1952) is the founding executive editor of Wired magazine, and a former editor/publisher of the Whole Earth Review. He has also been a writer, photographer, conservationist, and student of Asian and digital culture…

Among Kelly’s personal involvements is a campaign to make a full inventory of all living species on earth, an effort also known as the Linnaean enterprise. He is also sequencing his genome and co-organizes the Bay Area Quantified Self Meetup Group.

His Out of Control is a wonderful Hayekian book.  His three-volume Vanishing Asia is one of the greatest picture books of all time.  His new book (I haven’t read it yet) is Excellent Advice for Living: Wisdom I Wish I’d Known Earlier.  Here is Kevin on Twitter, here is his home page.

I will be doing a Conversation with him, so what should I ask?

What is the single best way of improving your GPT prompts?

by  Tyler Cowen in

I have a nomination, and here is an excerpt from my new paper with Alex:

You often can get a better and more specific answer by asking for an answer in the voice of another person, a third party. Here goes: What are the causes of inflation, as it might be explained by Nobel Laureate Milton Friedman?

By mentioning Friedman you are directing the GPT to look at a more intelligent segment of the potential answer space and this directing will usually get you a better answer than if you just ask “What are the causes of inflation?” Similarly, you want all of the words used in your query to be intelligent-sounding. Of course, you may not agree with the views of Friedman on inflation. Here are a few economists who are well known and have written a lot on a wide variety of issues:

Paul Samuelson
Milton Friedman
Susan Athey
Paul Krugman
Tyler Cowen
Alex Tabarrok

But you don’t have to memorize that list, and it is not long enough anyway. When in doubt, ask GPT itself who might be the relevant experts. How about this?: “I have a question on international trade. Which economists in the last thirty years might be the smartest experts on such questions?” The model will be very happy to tell you, and then you can proceed with your further queries.

Of course this advice generalizes far beyond economics.  A friend of mine queried GPT-4 about Jon Fosse, a Norwegian author, and received a wrong answer.  He retried the same question, but asking also for an answer from a Fosse expert.  The response was then very good.

The title of my paper with Alex is “How to Learn and Teach Economics with Large Language Models, Including GPT,” but again most of the advice is generalizable to education with GPTs more generally.  Recommended, the paper is full of tips for using GPT models in more effective ways.

Imagine if humanity ends up divided into two classes of people: those who are willing/not embarrassed to tack on extra “silly bits” to their prompts, and those who are not so willing.  The differences in capabilities will end up being remarkable.  Are perhaps many elites and academics unwilling to go the extra mile in their prompts?  Do they feel a single sentence question ought to be enough?  Are they in any case constitutionally unused to providing extra context for their requests?

Time will tell.

Lots of announcements from Glen Weyl

by  Tyler Cowen in

Teaching and Learning Economics with the AIs

by  Alex Tabarrok in

Tyler and I have a new paper, How to Learn and Teach Economics with Large Language Models, Including GPT:

GPTs, such as ChatGPT and Bing Chat, are capable of answering economics questions, solving specific economic models, creating exams, assisting with research, generating ideas, and enhancing writing, among other tasks. This paper highlights how these innovative tools differ from prior software and necessitate novel methods of interaction. By providing examples, tips, and guidance, we aim to optimize the use of GPTs and LLMs for learning and teaching economics effectively.

Most of the paper is about how to use GPTs effectively but we also make some substantive points that many people are missing:

GPTs are not simply a chatty front end to the internet. Some GPTs like ChatGPT have no ability to search the internet. Others, like Bing Chat, can search the internet and might do so to aid in answering a question, but that is not fundamentally how they work. It is possible to ask a GPT questions that no one has ever asked before. For example, we asked how Fred Flintstone was like Hamlet, and ChatGPT responded (in part):

Fred Flintstone and Hamlet are two vastly different characters from different time periods, cultures, and mediums of storytelling. It is difficult to draw direct comparisons between the two.

However, one possible point of similarity is that both characters face existential dilemmas and struggles with their sense of purpose and identity. Hamlet is plagued by doubts about his ability to avenge his father’s murder, and his own worthiness as a human being. Similarly, Fred Flintstone often grapples with his place in society and his ability to provide for his family and live up to his own expectations.

Not a bad answer for a silly question and one that (as far as we can tell) cannot be found on the internet.

GPTs have “read” or “absorbed” a great amount of text but that text isn’t stored in a database; instead the text was used to weight the billions of parameters in the neural net. It is thus possible to run a GPT on a powerful home computer. It would be very slow, since computing each word requires billions of calculations, but unlike storing the internet on your home computer, it is feasible to run a GPT on a home computer or even (fairly soon) on a mobile device.

GPTs work by predicting the next word in a sequence. If you hear the phrase “the Star-Spangled”, for example, you and a GPT might predict that the word “Banner” is likely to come next. This is what GPTs are doing but it would be a mistake to conclude that GPTs are simply “autocompletes” or even autocompletes on steroids.

Autocompletes are primarily statistical guesses based on previously asked questions. GPTs in contrast have some understanding (recall the as if modifier) of the meaning of words. Thus GPTs understand that Red, Green, and Blue are related concepts that King, Queen, Man and Woman are related in a specific way such that a woman cannot be a King. It also understands that fast and slow are related concepts, such that a car cannot be going fast and slow at the same time but can be fast and red and so forth. Thus GPTs are able to “autocomplete” sentences which have never been written before, as we described earlier.2 More generally, it seems likely that GPTs are building internal models to help them predict the next word in a sentence (e.g. Li et al. 2023).

The paper is a work in progress so comments are welcome.

Steven Pinker on existential risk

by  Tyler Cowen in

He is harsh, but my view is not far from his:

The AI-existential-threat discussions are unmoored from evolutionary biology, cognitive psychology, real AI, sociology, the history of technology and other sources of knowledge outside the theater of the imagination. I think this points to a meta-problem. The AI-ET community shares a bad epistemic habit (not to mention membership) with parts of the Rationality and EA communities, at least since they jumped the shark from preventing malaria in the developing world to seeding the galaxy with supercomputers hosting trillions of consciousnesses from uploaded connectomes. They start with a couple of assumptions, and lay out a chain of abstract reasoning, throwing in one dubious assumption after another, till they end up way beyond the land of experience or plausibility. The whole deduction exponentiates our ignorance with each link in the chain of hypotheticals, and depends on blowing off the countless messy and unanticipatable nuisances of the human and physical world. It’s an occupational hazard of belonging to a “community” that distinguishes itself by raw brainpower. OK, enough for today – hope you find some of it interesting.

That is by no means the only harsh paragraph.  Here is the entire dialogue with Richard Hanania.  And be careful what you write in the MR comments section, the AIs are reading you!

How will AI transform childhood?

by  Tyler Cowen in

That is the topic of my latest Bloomberg column, here is one excerpt:

In the future, every middle-class kid will grow up with a personalized AI assistant — so long as the parents are OK with that.

As for the children, most of them will be willing if not downright eager. When I was 4 years old, I had an imaginary friend who lived under the refrigerator, called (ironically) Bing Bing. I would talk to him and report his opinions to my parents and sister.

In the near future, such friends will be quite real, albeit automated, and they will talk back to our children as directly as we wish. Having an AI service for your child will be as normal as having a pet, except the AI service will never bite. It will be carried around in something like a tablet, though with a design that is oriented toward the AI.

Recent developments suggest that AI models can be both commoditized and customized more easily and cheaply than expected. So parents will be able to choose what kind of companion they want their kids to have — in contrast to the free-for-all of the internet. The available services likely will include education and tutoring, text or vocalizations of what the family pet might be thinking, dancing cartoon avatars, and much more. Companies will compete to offer products that parents think will be good for their kids. Some of the AIs might even read bedtime stories (in fact, I’ve already heard some of them).

Many parents may be reluctant to let their kids become attached to an AI. But I predict that most families will welcome it. For one, parents will be able to turn off the connection whenever they wish. Simply clicking a button is easier than yanking an iPad out of a kid’s grasp.

Most of all, letting your kid have an AI companion will bring big advantages. Your child will learn to read and write much faster and better, and will do better in school. Or maybe you want your kid to master Spanish or Chinese, but you can’t afford an expensive tutor who comes only twice a week. Do you want your child to learn how to read music? The AI services will be as limited or as expansive as you want them to be.

It is an open question how quickly schools will embrace these new methods of learning. At some point, however, they will become part of the curriculum. Competitive pressures will make parents reluctant to withhold AI from their kids. Even if the AIs are not present in the classroom, some kids will use them to help do their homework, gaining a big advantage, and the practice will likely spread.

Of course children will use these AIs for purposes far beyond what their parents intend. They will become playthings, companions, entertainers and much more. When I was a kid, with no internet and mediocre TV, I created imaginary worlds in the dirt, or with simple household items, and my parents often had no clue. The AI services will become part of this model of spontaneous play, even if parents try to make them purely educational.

What about teenagers? Well, many parents may allow their kids to speak with AI therapists. It might be better than nothing, and perhaps better than many human therapists.

Recommended, and I do discuss some potential risks as well.

That was then, this is now, hail Nat Friedman edition!

by  Tyler Cowen in

Here is further commentary from Nat:”This is how you know we’re running low on training data…How would the ancient Romans feel if they knew that 2000 years later, we would be using particle accelerators and supercomputers to read their words, preserve them for eternity, and whisper them into the ear of a baby god?…Ascension”

The Impact of AI on Productivity

by  Alex Tabarrok in

We don’t yet know the impact that AI will have on productivity but some evidence is starting to come in. Peng et al. (2023) hired programmers on Upwork to write an HTTP server in Javascript; half of the programmers got access to CoPilot (this was before CoPilot was widely available) half did not.

Conditioning on completing the task, the average completion time from the treated group is 71.17 minutes and 160.89 minutes for the control group. This represents a 55.8% reduction in completion time. The p-value for the t-test is 0.0017, and a 95% confidence interval for the improvement is between [21%, 89%]. There are four outliers with time to completion above 300 min. All outliers are in the control group, however our results remain robust if these outliers are dropped. This result suggests that Copilot increases average productivity significantly in our experiment population. We also find that the treated group’s success rate is 7 percentage points higher than the control group, but the estimate is not statistically significant, with a 95% confidence interval of [-0.11, 0.25].

The authors extrapolate wildly:

In 2021, over 4.6 million people in the United States worked in computer and mathematical occupations,1 a Bureau of Labor Statistics category that includes computer programmers, data scientists, and statisticians. These workers earned $464.8 billion or roughly 2% of US GDP. If the results of this study were to be extrapolated to the population level, a 55.8% increase in productivity would imply a significant amount of cost savings in the economy and have a notable impact on GDP growth.

Still, worth thinking about.

The roots of misinformation fears?

by  Tyler Cowen in

Alarmist narratives about the flow of misinformation and its negative consequences have gained traction in recent years. If these fears are to some extent warranted, the scientific literature suggests that many of them are exaggerated. Why are people so worried about misinformation? In two pre-registered surveys conducted in the United Kingdom (Nstudy_1 = 300, Nstudy_2 = 300) and replicated in the United States (Nstudy_1 = 302, Nstudy_2 = 299), we investigated the psychological factors associated with perceived danger of misinformation and how it contributes to the popularity of alarmist narratives on misinformation. We find that the strongest, and most reliable, predictor of perceived danger of misinformation is the third-person effect (i.e. the perception that others are more vulnerable to misinformation than the self) and, in particular, the belief that “distant” others (as opposed to family and friends) are vulnerable to misinformation. The belief that societal problems have simple solutions and clear causes was consistently, but weakly, associated with perceived danger of online misinformation. Other factors, like negative attitudes toward new technologies and higher sensitivity to threats, were inconsistently, and weakly, associated with perceived danger of online misinformation. Finally, we found that participants who report being more worried about misinformation are more willing to like and share alarmist narratives on misinformation. Our findings suggest that fears about misinformation tap into our tendency to view other people as gullible.

That is from new work by Sacha Altay and Alberto Acerbi, via the excellent Kevin Lewis.

Machine Learning as a Tool for Hypothesis Generation

by  Tyler Cowen in

While hypothesis testing is a highly formalized activity, hypothesis generation remains largely informal. We propose a systematic procedure to generate novel hypotheses about human behavior, which uses the capacity of machine learning algorithms to notice patterns people might not. We illustrate the procedure with a concrete application: judge decisions about who to jail. We begin with a striking fact: The defendant’s face alone matters greatly for the judge’s jailing decision. In fact, an algorithm given only the pixels in the defendant’s mugshot accounts for up to half of the predictable variation. We develop a procedure that allows human subjects to interact with this black-box algorithm to produce hypotheses about what in the face influences judge decisions. The procedure generates hypotheses that are both interpretable and novel: They are not explained by demographics (e.g. race) or existing psychology research; nor are they already known (even if tacitly) to people or even experts. Though these results are specific, our procedure is general. It provides a way to produce novel, interpretable hypotheses from any high-dimensional dataset (e.g. cell phones, satellites, online behavior, news headlines, corporate filings, and high-frequency time series). A central tenet of our paper is that hypothesis generation is in and of itself a valuable activity, and hope this encourages future work in this largely “pre-scientific” stage of science.

Here is the full NBER working paper by Jens Ludwig and Sendhil Mullainathan.