"AI" Systems Shouldn't Pretend To Be Human
The new Amazon Alexa with AI has the same basic problem of all AI bots, it acts as if it's human, with a level of intimacy that you really don't want to think about, because Alexa is in your house, with you, listening, all the time. Calling attention to an idea that there's a psuedo-human spying on you is bad. Alexa depends on the opposite impression, that it's just a computer. I think AI's should give up the pretense that they're human, and this one should be first.
I very much agree with this, for two reasons. One, "AI" isn't close to intelligence, and it distorts the truth to pretend otherwise, especially for non-technical people unfamiliar with how LLMs operate. Two, on a product level it's a bad choice — given how far from intelligence LLMs are, letting the generated text sound "human" sets up all users of the product to feel dissonance every time the product doesn't live up to its presentation.