A Note on Challenges to Objective Functions
Because we are firmly against pushing one belief system over another belief system, we make a point of trying to provide the most simple, compelling argument possible against all belief systems discussed. Whenever we explain why someone might support a specific objective function, we also attempt to create a strong argument against that objective function.
Often these arguments take the form of thought experiments: analogies or forced decisions meant to determine if you really believe what you think you believe. These thought experiments are typically tailored to the belief system of a person we expect might hold such an objective function (e.g., a challenge to an objective function that would likely only be held by atheists may include the presupposition that evolution is true and that emotional states are an emergent property of brain states, whereas an objective function that an evangelical Christian might hold will not include such assumptions in its discussion). For example, when challenging you to carefully consider an objective function oriented around maximizing personal good feelings, we may ask:
If you could live in a human-sized tube that would make you live twice as long as you would outside the tube and pump you full of good feelings that were more diverse and more enjoyable than any feeling you could experience outside the tube, would you choose to drop everything about your life this instant and commit to the tube for the rest of your life, despite the fact that you would lose all contact with friends and family and never leave the tube again?
This challenge forces you to ask yourself if there is not something else more important to you than this specific objective function. The challenge is not meant to question or insult the value of the objective function itself, nor is it meant to deride anyone who should choose it.
It is very important you do not try to “cheat” a thought experiment by arguing about the challenge’s details, because doing so enables you to avoid a difficult but necessary decision. By answering the question with a cheat answer such as, “No, I wouldn’t live in the happiness tube because it is a drug and I may get addicted to it,” or “I would get used to feeling positive emotions all the time and they would no longer feel good after a short period,” you are avoiding the substance of the question and not allowing yourself to be meaningfully challenged. The point of the question is to force you to reflect on whether or not you would actually accept perfect emotional optimization over all other things.