One thing I believe happens a lot to effective altruists, who are generally (all?) consequentialists of some sort, is that their rational calculus leads them to efface themselves or lead a life much more miserable than they would've if they had spent less time thinking about their altruism. I think...
I have this idea that is not a contender with the serious formal reasoning that people who know a hell of a lot about AI are able to do, but nonetheless I think could be useful for them to hear. The idea is that for a mind (in a broad...
Wouldn't a mind getting better at executing its utility function cause it to create utility for more alternate functions? Is there a saving grace of increasing intelligence?
One thing I believe happens a lot to effective altruists, who are generally (all?) consequentialists of some sort, is that their rational calculus leads them to efface themselves or lead a life much more miserable than they would've if they had spent less time thinking about their altruism. I think EA men and women are at the final frontier of consideration of the link between rational self-interest and true altruism, where their desire to do good from the point of view of the universe is leading them into great punishment and one would have difficulty convincing them the aforementioned link is strong. Nonetheless, I believe it is strong, and would like to... (read 283 more words →)
I don't think there's a physical difference between morality being arbitrary because it is the evolution of blind selfish memes, or morality being limited by practical considerations. If someone had a hypothetical objective morality they could still be a slave-owner and war-monger in one age and a civil-rights activist calling for world peace in a different age.
I have this idea that is not a contender with the serious formal reasoning that people who know a hell of a lot about AI are able to do, but nonetheless I think could be useful for them to hear. The idea is that for a mind (in a broad sense of the word) to have any aim it must simultaneously aim to preserve itself long enough to undertake the actions that serve that aim, and that following from this is an inbuilt cooperative foundation for all minds.
So to try to concretize this I would say that, for example, a human being is preserving their own being from one moment to the... (read more)
Dumb question: In the omnicide outcome agi continues its existence, but whether it's sentient or not no one else from earth does. In the undignified survival outcome agi continues its existence and potentially so do many humans for functionally ever and those human may have lives better than death (even barely). Why is parochial value-alignment necessarily worse than no alignment at all?
Wouldn't a mind getting better at executing its utility function cause it to create utility for more alternate functions? Is there a saving grace of increasing intelligence?
An "ontological crisis" is treated well enough when you note your perception as a belief. If it's literally impossible to not have a best working model of reality in its entirety that undermines profound doubt.
It is, but I'm specifically saying a form of rule consequentialism that serves personal happiness about as well as it could be served is in fact rational (for anyone who is trying to maximize impersonal happiness and probably for anyone who is a consequentialist of any kind).