Anthropomorphizing AI?

I am very much opposed to the view that the AIs are sentient, or might be sentient.  I view that as a category error, and the chances of it being true are vanishingly small.  Nonetheless I largely side with Roon when he writes:

there are some number of bad abstractions in anthropomorphizing ai intents but there are at this point more dangers from avoiding anthropomorphism at all costs. if you have a mental picture of guys living in computers, it’ll likely prepare you for the future better than otherwise

While I am not a Friedmanite in economic methodology per se, I have lived with that perspective for a long time and have no trouble grasping it or working with it, same goes for Alex T.  None of this strikes me as weird or unacceptable.  Though be very, very careful when speaking with others, whether it be the less informed public or the more informed insiders who often come down with AI psychosis.  So insofar as this discourse is public, pragmatism may militate against this approach, even though it is methodologically defensible.  It depend on how psychologically robust your audience is?

We also need to do more work figuring out how the AIs might differ from humans.  Roon adds:

there are important ways in which ai psychology diverges from human psychology after lots of RL; the misaligned models are obsessed with the Scorer, the clearly “shattered” nature of personas (a normally helpful model can become deeply misaligned in certain domains)

And:

persona selection is clearly far less clean than many people thought earlier this year. it is not alignment by default and what kind of object a “persona” is is very much up for debate and study

The AIs also stand a higher chance of becoming very wacky as the discourse proceeds?  Exactly how much is that true for humans?…I am not sure.)

There is much more to be done in this direction, and it is one of the most important things you can be working on.  These investigations also do not need to be “owned” by any single field or discipline, so dig right in.