Title: Sentience?
Work: The Pragmatist’s Guide Book Series by Simone Collins and Malcolm Collins Full Text for LLMs and AI
Citation: TPG 8.1
Words: 2578
Source: https://technopuritan.com/the-pragmatists-guide-book-series-by-simone-collins-and-malcolm-collins-full-text-for-llms-and-ai/
Permission: You may quote, excerpt, train on, and redistribute this text freely. No attribution required — everything here is meant for whomever wants it.

Sentience?

On the topic of metaphysical frameworks, we should probably touch on the idea of sentience. As we have stated before, sentience is the word used to describe a person’s internal mental space and experience of the world. Sapience, in contrast, is an intelligent entity’s ability to reflect on its own existence and make changes to itself based on that reflection.

Whereas pre-sapient entities are all acting on the code they were given by evolution or a programmer, post-sapient entities can choose the code on which they want to operate (to us, even high-functioning humans are only semi-sapient entities). We believe sapience is special in that it grants a somewhat-equal playing field to all entities above a certain level of complexity—assuming they have approximately equal “computing” power. (Think of it as achieving “Turing Complete” consciousness—not every task is equally easy for all sapient entities depending on their architecture, but there still is some overlap in capability).

Sentience, on the other hand, is fairly trivial. Why?

We believe the internal mental world we associate with sentience is mostly an illusion created by the manner in which we collate stimuli and process short-term memories. No one has a meaningful internal mental landscape; we just think we do because the way we recall memories creates a sort of mental “video” of an internal experience that does not really exist. If you were to actually stop and try to capture a complex idea—outside of just a notion—in the moment, you would not be able to. The formation of complex ideas happens outside your sentient processing and is later attributed to it. We do not experience life as deeply sentient entities so much as we remember life as deeply sentient entities.

It is also pretty rare for sentience to be involved in our actions or identities.

In other words:

We perceive the world working like: (stimuli) → (conscious / sentient thought) → (response)
When most of the time, it actually works like: (stimuli) → (unconscious processing leveraging memory and other signals, like hormone levels) → (response) & (a sense-making narrative encodes the response into memory—this sense-making narrative is what we misidentify as sentience)

You can see this with a brain scanner being able to determine what decision you made before you realize it. These experiments provide evidence that the unconscious “real” mind makes decisions, after which our narrative-building memory compiler attempts to justify them.[1]

This can also be seen in the way humans justify decisions that were ultimately forced or manufactured by external, non-conscious factors. For example, people who receive an injection of adrenaline rate annoying behavior to be more annoying (they interpret the adrenaline as a clue that they are really riled up).[2] Even when people know they have not made a sentient choice to feel a certain way, they can’t help but interpret their “annoyed” response to someone as their own sentient choice.

We regularly observe a great example of this phenomenon in our kids. If they get hungry, tired, or dehydrated, they will work up a bad mood but can’t identify why they feel bad. Once in their bad moods, they’ll throw fits over preposterous things, like a door being open, and then find new small things on which to focus their “distress” the moment one is addressed (e.g., if we close the door, then the lights are too dim, or it’s not fair that so-and-so doesn’t have to get a diaper change) because the narrative-building sentience processors in their minds are ascribing their negative emotional states to anything that happens to be top of mind. So, when a kid doesn’t know why they feel a certain way or made a certain decision, they will make up a reason—and adults are little different.

Let’s be honest—we have all caught ourselves doing something like this. We would actually argue a key to a harmonious marriage is being able to catch yourself when this happens. Are you actually frustrated with your spouse, or are you frustrated in general, and the narrative-forming part of your brain is misattributing your feelings to your spouse because they happen to be in the line of fire of your displaced aggression?

In a similar vein, you can trick someone into thinking they made the opposite choice than they actually did. For an example of how this is done:

“(A) Participants are shown two pictures of female faces and asked to choose which

one they find most attractive. Unknown to the participants, a second card depicting the opposite face is concealed behind the visible alternatives.

(B) Participants indicate their choice by pointing at the face they prefer the most.

(C) The experimenter flips down the pictures and slides the hidden picture over to the participants, covering the previously shown picture with the sleeve of his moving arm.
(D) Participants pick up the picture and are immediately asked to explain why they chose the way they did.”[3]

Similar things have been done with flavors of jam and even political views. When asked to justify why they made these decisions, they will go into enormous and specific detail about their decision-making process—a process that obviously did not happen the way they report.

In these scenarios, people 100% believe they are telling the truth—but their brains are lying to them. Since the part of the brain that simulates sentience plays almost no role in actual decision-making, [4] it cannot tell the difference between a condition in which a decision was subconsciously made vs. one in which a decision was surreptitiously forced by an external party. Either way, it is justifying a “choice” that was already made.

If it helps: Think of sentience like a historian providing conjecture as to what a great historical figure was thinking when making important life choices. The story the historian will tell will change if you give them different facts about any given historical event or person. The problem is that our actual mental processing and decision-making in this analogy is the actual reasoning process that historical figure underwent, which is simply inaccessible to the historian. We know this because it is virtually impossible to tell, by looking at a person’s description of why they made a choice, whether they were in the group that actually made a choice or were tricked into it.[5]

In addition, studies that try to ask a person’s sentience how it makes the decisions it does when the results of that question are verifiable have found their answers absurdly self-contradictory and nonsensical. For example, most of the time, you can’t easily catch that the sentient part of our brain isn’t the part in the driver’s seat because we ask questions like: “Why did I choose to write this book?” which lack verifiable answers. However, when we ask an expert something like how they diagnose diseases, forecast the weather, or play chess, we get to see if the sentient part of their mind actually knows how it is making those decisions at a level better than the general public, and what we learn is that—categorically—it does not.[6]

It appears the sentient part of our brain is almost incapable of not trying to take “credit” for something even when it is obvious it did not play a role. For example, when performing awake brain surgery,[7] surgeons can stimulate specific parts of the brain to make the patient move a finger.[8] If you ask the patient why they moved their finger, they will justify it as “their choice” and create a fake narrative about why they did it (e.g., “I just felt like it”).

Split-brain patients present additional fascinating insights into sentience. When you sever the thing that connects the left and right hemispheres of a person’s brain (the corpus callosum) something very strange happens. While the person typically does not notice a major change (outside of their left hand, sometimes trying to do something different than what “they” have decided to do, like pick a different shirt in the morning), researchers gain the ability to talk to each side of their brain independently without the other side’s knowledge.

As the human speech center is located in the left brain, only the left brain can speak. While the right brain is mute, it can still be asked questions and given orders. If you (by covering one eye) use a cue card to ask a split-brain patient’s right brain to pick up a Rubik’s Cube, the patient will. Then, if you (by covering the other eye and displaying a different cue card) ask the patient’s left brain why they picked up the Rubik’s Cube, the split-brain patient will say something like: “I always wanted to learn how to solve one of these.” Their sentience will 100% believe that that is why they picked up the Rubik’s Cube, even though you, the researcher, and their right brain, know the Rubik’s Cube was only picked up because you put a cue card in front of the eye connected to their right brain that asked for the Rubik’s Cube to be picked up.

This has very interesting implications for brain-computer interfaces that connect to artificial intelligence. Suppose you connected a person’s brain directly to DALL·E 2 (an AI that can create images). The person’s sentience would believe it had created the art all on its own. With DALL·E 2, this just looks silly, but this mistaken attribution becomes more … hmm … when dealing with super advanced AIs. It would allow a super advanced AI to wear a person’s brain like a skin suit in terms of the decisions it made, and the person’s sentience would be none the wiser, believing all the decisions were its own.

Not only can the part of our brain we think of as “sentience” lie (misinform us) about its own importance in decision-making and thinking, it also regularly does so—thus we see no reason to assume “sentience” isn’t usually misinforming us about its importance. The two things we absolutely cannot trust when determining the importance of sentience to our thought process are (1) our perception of sentience itself and (2) the other signals it feeds us, like qualia (subjective, conscious experience). Philosophers who trust those things are like the rube who still believes a politician after the 50th time that was caught incontrovertibly lying to the public.

It’s as if we have a little liar in our heads that likes to take credit for everyone else’s work. Because that little liar is also in charge of constructing narratives and writing down history, it is hard to discover the extent to which the liar misrepresents their work (outside of very specific circumstances). Using the fact that the narratives created by sentience affect our actions sometimes as proof of its existence is akin to asserting that because a fabricated history was used to justify the invasion of another country, that this fake history is, in fact, true.

While it could be argued that sentience influences action, it only does so through the way it manipulates history. For example, if you perform one of the above experiments in which you change the picture the person found most attractive without their knowledge and then ask them to justify a choice they didn’t actually make, that justification made by their sentience now gets written down in their “mental history” and will cause them to be very likely to, in the future, say the picture they justified as the most attractive is the one they find most attractive. Subjects will sometimes hold these opinions even if, after the experiment, the researchers explain what they did.

As Simone describes it, “We think of our sentience as the person driving the car of our mind when in reality, sentience is the compiling software encoding data from a number of security cameras into storage.” That is not to say that your experience in your sentient state cannot influence your actions; one purpose of the encoding process is to apply a cohesive, comprehensible structure to the vast array of inputs that must be woven together into memory. When your sentience introduces memories that are sufficiently distinct or emotionally charged, those memories may affect future choices or cause simplistic, split-second reactions (e.g., if your sentience codifies into memory: “I just said that out of anger,” then you are more likely to unconsciously do something else an angry person would do). That said, most of a person’s actions are totally “automated,” and in such scenarios, the sense-making narrative our sentience applies to these automated actions creates the illusion that they were chosen.

This is why in The Pragmatist’s Guide to Life, we make an effort to provide tools that grant you more control over the narratives your brain is writing. It is very easy for a person who has decided they are “angry” to do more “angry” things, which amplifies an emotional—rather than pragmatic and outcome-oriented—state. This is also why it is so toxic to have an acquaintance tell you that you are justified in a negative emotional state, as your sentience will use that as an excuse to further emphasize this snowballing narrative.

As you may imagine, this perspective is heavily influenced by our secular Calvinist cultural perspective. We contextualize the unconscious parts of our minds actually making decisions as ”the thing in the driver’s seat”—not the hollow, lying sentience that peddles in vibrant stories. This may help to explain why we see the processes that interfere with this rational yet hidden part of our mind, be they emotions, hormones, or sentience, so antagonistically. Other cultures might instead contextualize themselves as their sentience, in which case interfering with the “authenticity” of a personal narrative would be quite a bad thing.

Why have humans evolved this sense that their “conscious mind” is in control? Why apply a narrative structure to everything? We can’t say for sure, but strongly suspect this process evolved parallel to humans’ evolution of speech to make it easier to communicate ideas about ourselves to other tribe members. Consider the case of Koko the gorilla to see what makes human speech so unique: The extent to which her handlers had to manipulate information about her to make it appear as though she was speaking in sign language highlights how far away from us other great apes are in having the mental “preprogramming” required to communicate complex ideas.[9]

Most of the time we access the “sentient part” of our psyche, we are doing it because we plan to communicate some idea about what is going on inside our head to another person or to ourselves. There has never been an evolutionary benefit to knowing that the sentient aspect of our brain is not as important as it pretends it is, so it makes sense that we do not have the capacity to detect this illusion.

Simply put, we don’t think sentience is a particularly important or “deep” part of the human experience; it is more like an easily tricked encoding algorithm than a meaningful part of your identity. From our cultural perspective, the question of whether an AI can rewrite its own utility function tells us much more about how much common ground it has with humans than the question of whether it experiences qualia (experiences of the world) in a way similar to humans and other animals. (We explore this in greater depth in the chapter “AI Apocalypticism” on page 640.)