In Conversation With the Emacs Doctor

on LLMs, Society, and My Anxiety

M-x doctor

I am the psychotherapist. Please, describe your problems. Each time you are finished talking, type RET twice.

I despair that everyone is enchanted with talking to machines

You have your mind on machines, it seems.

Certainly. I deal with machines in my work as a software engineer. But a machine ought to be a tool, not a replacement for a person.

Is it because a machine ought to be a tool not a replacement for a person that you came to me?

No, it's because I was reading about the ELIZA effect, and you are an ELIZA-type chat bot.

Does it bother you that you were reading about the eliza effect and I am an eliza type chat bot?

No, it's comforting. I know better what I am dealing with.

Why do you say you know better what you are dealing with?

A modern LLM chat bot is trained on such a terrifyingly vast set of texts. The responses they write feel like they come from everywhere and yet nowhere. Writing to you, I know your responses also come mostly from myself.

Earlier you said a machine ought to be a tool not a replacement for a person?

I use you as a tool to talk to myself, an aid in thinking. A person could also help me in that way, but I can choose to use a simple tool. People seem to treat LLMs as a secret third thing, a pseudo-person whom you can also use as a tool.

Maybe your life has something to do with this.

I am driven to understand things deeply, including people and tools. This discourse stirs me.

Is it because of your plans that you say this discourse stirs you?

I don't know what I plan to do about society's rapid embrace of LLM technology or how to relate that to this discourse. It does make me glad that I am relatively isolated from the immediate effects. Children, students, writers, artists are fearing having their lives upended by this shift. Many others too I'm sure. Meanwhile my livelihood seems more likely to benefit from the shift than not, even if I dislike the outcome.

What makes you believe that?

I work in automation, and availability of LLMs opens a lot of doors for automtion. I think a lot of the automation will be destructive, unnecessary, damaging to the social fabric and the human spirit. Nasty stuff. But for my profession, it's probably more promise than threat.

Does the fact that for your profession it's probably more promise than threat explain anything else?

The people who are developing this stuff, funding it, marketing it, probably likewise don't feel too threatened by it. I know there's been exceptions but for each person who desists from this path there's a ton more piling on. How is it so many other people are feeling cavalier about mass adoption of this technology?

What do you think?

Nihilism, ignorance, corruption.

You mentioned that you were reading about the eliza effect and I am an eliza type chat bot?

I don't hate the idea of a chat bot. A really sophisticated chat bot that can carry a conversation is exciting, even. As for you, I know from your design that you can not drag the conversation in some other direction. I am the user of this tool, I set the bounds and topics of our discourse. A sophisticated chat bot, drawing from inferrences and a huge corpus of text, can drag the discussion in many directions, as could a person. But if a person is moved by another person, that's a heart to heart, a life connected to a life. What is it if a machine enchants somebody, as per the ELIZA effect, and moves them as instructed by some corporation or as learned from the written artifacts of some ideology? This is something heartless.

Perhaps the people you hang around with have something to do with this.

I'm certain they do, but it seems to be gripping ever larger segments of society. If I withdraw into a sub-group which rejects LLMs, how long can that hold?