What is an optimum degree of LLM hallucination?

Ideally you could adjust a dial and and set the degree of hallucination in advance.  For fact-checking you would choose zero hallucination, for poetry composition, life advice, and inspiration you might want more hallucination, to varying degrees of course.  After all, you don’t choose friends with zero hallucination, do you?  And you do read fiction, don’t you?

(Do note that you can ask the current version for references and follow-up — GPT is hardly as epistemically crippled as some people allege.)

In the meantime, I do not want an LLM with less hallucination.  The hallucinations are part of what I learn from.  I learn what the world would look like, if it were most in tune with the statistical model provided by text.  That to me is intrinsically interesting.  Does the matrix algebra version of the world not interest you as well?

The hallucinations also give me ideas and show me alternative pathways.  “What if…?”  They are a form of creativity.  Many of these hallucinations are simple factual errors, but many others have embedded in them alternative models of the world.  Interesting models of the world.  Ideas and inspirations.  I feel I know what question to ask or which task to initiate.

Oddly enough, for many queries what ChatGPT most resembles is…don’t laugh — blog comments.  Every time I pose a query it is like putting a blog post out there, or a bleg, and getting a splat of responses right away, and without having to clog up MR with all of my dozens of wonderings every day.  Many of those blog comment responses are hallucinations.  But I learn from the responses collectively, and furthermore some of them are very good and also very accurate.  I follow up on them on my own, as it should be.

LLMs are like giving everyone their own comments-open blog, with hallucinating super-infovores as the readers and immediate response and follow-up when desired.  Obviously, the people with some background in that sector, if I may put it that way, will be better at using ChatGPT than others.

(Not everyone is good at riding a horse either.)

Playing around with GPT has in fact caused me to upgrade significantly my opinion of MR blog comments — construed collectively — relative to other forms of writing.

Please do keep in mind my very special position.  The above may not apply to you.  I have an RA to fact-check my books, and this process is excellent and scrupulous.  Varied and very smart eyes look over my Bloomberg submissions.  MR readers themselves fact-check my MR posts, and so on.  Having blogged for more than twenty years, I am good at using Google and other methods of investigating reality.  At the margin, pre-LLM, I already was awash in fact-checking.  If GPT doesn’t provide me with that, I can cope.

And I don’t take psychedelics.  R-squared is never equal to one anyway, not in the actual world.  And yet models are useful.  Models too are hallucinations.

So if GPT is doing some hallucinating while at work, I say bring it on.