GPT-3, etc.

Here is an email from a reader, I do not really have an opinion of my own yet.  Please note I will not indent any further:

“I wanted to draw your attention to something. Are you familiar with “AI Dungeon,” text-based RPG “open world” game running on GPT-2 / GPT-3?  Here’s the author’s discussion on medium, or you can play the GPT-2 version for free to get a sense of it directly.

But what I really want to draw your attention to is players who are using custom prompts to open up dialogs with GPT-3 about non-game things.

This result is particularly impressive:  https://www.reddit.com/r/slatestarcodex/comments/hrx2id/a_collection_of_amazing_things_gpt3_has_done/fy7i7im/

( Following that string of posts is a little hampered by reddit’s format; here are the posts in order: part 1,  part 2,  part 3)

If the author is to be believed, they’ve had GPT-3 / “Dragon”:

1. write code
2. act as a pharmacology tutor
3. write poetry
4. translate english, french, chinese (the instruction to “balance the intent of the author with artistic liberty” is particularly interesting)

It’s hard to excerpt, I’d recommend reading the whole thing if you have time.

Here’s another user’s eloquent conversation about the experience of being an AI, using a similar mechanism (screencap images of the convo, part 1 and part 2 ), with a sample prompt if you want to converse with GPT-3 yourself via AI Dungeon.

I am increasingly convinced that Scott Alexander was right that NLP and human language might boostrap a general intelligence. A rough criteria for AGI might be something like (i) pass the Turing test, and (ii) solve general problems; the GPT-3-AI-Dungeon examples above appear to accomplish preliminary versions of both.

GPT was published in June 2018, GPT-2 in February 2019, GPT-3 in May 2020.

As best I can tell GPT -> GPT2 was ~10x increase in parameters over ~8 months, and GPT2 -> GPT3 was ~100x increase of parameters over ~14 months. Any number of naive projections puts a much more powerful release happening over the next ~1-2yrs, and I also know that GPT-3 isn’t necessarily the most powerful NLP AI (perhaps rather the most popularly known.)

When future AI textbooks are written, I could easily imagine them citing 2020 or 2021 as years when preliminary AGI first emerged,. This is very different than my own previous personal forecasts for AGI emerging in something like 20-50 years…

p.s. One of the users above notes that AI Dungeon GPT-3 (“Dragon”) is a subscription service, something like ~$6 a week. MIE.”