= 27d567bc2d48d9f40e9ccc20ea370b15)
PhilGoetz comments on Exterminating life is rational - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (272)
I think we're agreeing on the first point - any rational agent is selfish. But then there's no such thing as an unselfish agent, right? Also, no need to use the term selfish, if it's implicit in rational agent. If unselfish agents don't exist, it's easy to out-compete them!
"trying to also maximize someone else's utility function... would not be an 'agent', because agenthood is found at the level of the utility function." What do you mean by this? I read this as saying that a utility function which is directly dependent on another's utility is not a utility function. In other words, anyone who cares about another, and takes direct pleasure from another's wellbeing, is not an agent. If that's what you mean, then most humans aren't agents. Otherwise, I'm not understanding.
On Life-And-Death conflicts, yes, that's what I meant. You haven't given any such argument!
On the LHC, why are the next hundred thousand years a special case? And again, under what conditions should the LHC run?
No; I meant that each agent has a utility function, and tries to maximize that utility function.
If we can find an evolutionarily-stable cognitive makeup for an agent that allows it to have a utility function that weighs the consequences in the distant future equally with the consequences to the present, then we may be saved. In other words, we need to eliminate time-discounting.
One thing I didn't explain clearly, is that it may be that uncertainty alone provides a large enough time-discounting to make universe-death inevitable. Because you're more and more uncertain what the impact of a decision will be the farther you look into the future, you weigh that impact less and less the farther into the future you go.
But maybe this is not inevitably the right thing to do, if you can find a way to predict future impacts that is uncertain, but also unbiased!
EDIT: No. Unbiased doesn't cut it.
On life-and-death conflicts, I did give such an argument, but very briefly:
I realize this isn't enough for someone who isn't already familiar with the full argument, but it's after midnight and I'm going to bed.
The next 100,000 years are a special case because we may learn most of what we will learn over the next billion years in the next 100,000 years. During this period, the risks of something like running the LHC are probably outweighed by how much the knowledge acquired as a result will help us estimate future risks, and figure out a solution to the problem.