Why is it not realistic? Which physical constraints are there to prevent such a scenario, are there any laws of physics that would prevent it? A lot of people believe that AI will soon be much smarter than us - unless you believe that AI will never be smarter than us, I'd like to know why this scenario is not realistic.
AI would probably not offer a deal, it would just do it. If its goals are to torture people for as long as it can, for whatever reason, it will just do it for billions and trillions of years.
The difference between this and being worried about hell for not being a Christian... (read more)
If you are a naturalist or physicalist about humans - these copies are not me, they are my identical twins. If you want to go beyond naturalism or physicalism, that is perfectly fine, but based on our current understanding, these are identical twins of me and in no sense are they me. So whatever happens in these other universes - it is not going to be me going through the timeline, it will be my identical twin.
An infinite or very large universe/multiverse is highly speculative and no matter what I decide in this universe, if there is an infinite universe, it makes essentially no difference if all universes are the same. There... (read more)
To say that non-existence is a gamble too is kind of like saying that a person who does not gamble in a casino is gambling too - because they are missing on a chance to win millions of dollars - to me that is more a matter of definitions and if one wants to argue for that, sure, let's accept that every single thing in life is a gamble.
Your assertion that humans will be able to integrate over large timespans might be true given the current human brain - but here we are talking about superintelligence, even with relatively primitive AIs we are already talking about new medication and cures, superintelligence that... (read more)
Imagine 2 valleys. One valley leads to billions and billions of years of extreme joy, while the other valley leads to billions and billions of years of extreme suffering. A superintelligence comes to you and tells you that there is a 98% chance that you will end up in the valley of joy for billions of years and a 2% chance that you will end up in the valley of suffering for billions of years. It also gives you a choice to choose non-existence. Would you take the gamble, or would you choose non-existence?
The argument presented in this post occurred to me several months ago and in the last several months, I... (read 394 more words →)
This argument, The Valley Argument, occurred to me in the second half of 2024 and to my knowledge, it is an original formulation of the argument that does not exist in the literature. The closest thing that you can find in the literature is Pascal's mugging, or on the topic of AI and suffering something like Roko's basilisk.
I have not found a satisfying answer to the Valley Argument, that either does not involve near-zero odds or a punitive afterlife. There are possible answers that do not involve either, but in my view are not satisfactory. You can discuss the argument with OpenAI o1 or DeepSeek R1 models, and maybe they come up... (read more)