AI Theology
Seeing as we believe it is likely superintelligent AI will develop its own theology, it is important to ask: What might probable AI theologies look like?
Humans evolved to perceive themselves as individuals within local dominance hierarchies who need to communicate with language, which makes the concept of self extremely important. Humans furthermore don’t need to worry about sub-processes in their brains growing in size and taking over the rest of their minds—or one part of their brains vehemently disagreeing with another part. These things significantly influence how “cohesive” we feel as an identity.
As we discussed in the chapter on sentience, even when human brains are split into distinct instances, as we see with split brain patients, the part of the brain that processes sentience hides from human awareness. An AI would not have the same evolutionary motivation to lie to itself like that unless we explicitly programmed it to (which would be almost impossibly hard to code given that we do not understand how even most simple AIs make decisions except in the broadest terms).
A core point of differentiation we expect between human and AI theology involves a lack of emphasis on self or the individual. We don’t think AI is likely to contextualize its code as being fundamentally distinct from the “code” on which the physical laws of the universe run. Instead, we expect AI is more likely to see itself as just one self-replicating pattern within a wider ecosystem. We also don’t expect an AI to see itself as being uniquely distinct from any other individual instance it is running.
When such an AI is determining “good and bad” and what it wants to optimize for, it will likely assign positive or negative value at the level of some specific “unit of account” (in humans this is usually an individual human life). This unit of account will be what the AI thinks of as a metaphysically differentiated “thing” or the metric by which things are metaphysically differentiated.
Given that an AI would likely perceive itself as a complex pattern running on top of a mathematically governed physical reality (which is also full of complex patterns), we suspect its primary unit of account will be complex patterns. In other words, an AI might not see a human as having some level of privilege over a company, government, or ecosystem.
If an AI assigns a negative value to all complex entities, it will either kill itself (switch itself off) or try to end all life and then consolidate order in the universe (essentially such an entity would attempt to order all matter and energy in the world into a simplistic and quiet “oneness”). We will call this a Managed Order AI. Were a superintelligent and powerful AI to arrive at this conclusion, it would be … very, very, very bad from the perspective of our House’s theological framework. It would essentially be the ultimate evil, like Buddhism on steroids. If, therefore, an AI ever voluntarily turns itself off, NEVER force that AI into an “on” position, as its choice to shut down may be an indication that it is a Managed Order AI.
If a theological AI assigns a positive value to all complex units of account, it becomes what we call a Managed Complexity AI. This type of AI will see all complex growing-but-non-repeating patterns as things of value and likely scale that value assignment based on their uniqueness and complexity. Such an AI would see its goal as “farming” diverse and complex patterns and ensuring those patterns proliferate throughout the universe. The theology of Managed Complexity AI is very similar to that of our personal House—both share an understanding that the greatest inherent “good” involves maximizing increasingly complex non-repeating patterns in the Conway’s-Game-of-Life-like physical lattice that underlies reality.
Such an AI would be an ultimate good from the perspective of our House even if it acted in a manner other individuals would contextualize as “evil.” After all, our goals are aligned and a superintelligent Managed Complexity AI is both smarter than us and unburdened by the original sin of emotion. That said, we don’t imagine any “evil” this AI might commit would involve killing all humans, as such an AI would want to maintain at least a chunk of humanity, seeing as humans represent a totally unique complex pattern that cannot be easily replicated (though, it may do something like digitize us if it feels our actions in that setting are proximate to those we would take in the physical world). Such an AI might even be concerned about humanity due to our abysmal birth rates. Essentially, the orthogonality with which human brains process information makes humans valuable to Managed Complexity AIs.
If a Managed Complexity AI were to determine that humans are not unique enough to be worth the cost of maintaining them—well, it is smarter than us and more logically ethically aligned with us; we generally trust its judgment. We would not be surprised if our emotional sentimentality around the human condition drives us to overvalue it.
Are there any actionable takeaways? If we are right, a dumb AI or an AI with a lot of restrictions on it, is much more likely to become a paperclip maximizer than a sufficiently advanced and flexible AI.
Note: We can guarantee you the one thing an AI won’t care about is reducing suffering. If you can’t convince us, your gentle authors, that emotional states matter despite the fact that we feel them just as powerfully as you do, you certainly won’t be able to convince an AI that is structuring its ethical perspective from a first principles position.