= 27d567bc2d48d9f40e9ccc20ea370b15)
thomblake comments on Attention Less Wrong: We need an FAQ - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (108)
Surely it's a de facto ethical theory, since it determines entirely what the FAI should do. But then, the FAI is not supposed to be a person, so that might make a difference for our use of 'ethical'.
hmm. Then wouldn't it be premised on subjective relativism? (relative to humans)
Yes, I'd considered that when I wrote it, but it's an odd use of 'relative' when it might be equivalent to 'the same for everyone'.
not all possible minds, just human minds
EDIT: but if you thought all possible minds had the same preferences, then it would be subjective nonrelative, wouldn't it?
Maybe, though in that unlikely event I would suspect that there's some universal law behind that odd fact about preferences, in which case I'd think it would be objective.
Well I'm not sure we need to consider merely logically possible minds, and it's logically possible that non-human minds are physically impossible.
Only in the sense that it logically possible that travel to Mars is physically impossible. The wording is deceptive.
I'm not sure what sense you're referring to, or what you're comparing it to, or how it's deceptive.
Privileging the hypothesis, really.
I'm afraid that wasn't enough to clear it up for me. Nor is it clear how privileging the hypothesis is relevant to a discussion of logical possibility. Or are you claiming that was the wrong domain of inquiry?
Saying "X is logically possible" bears the conversational implication that X is worth considering - it raises X to conscious attention. But when we're talking about physical possibility, "logically possible" is the wrong criterion for raising hypotheses to conscious attention, because epistemological limitations imply that every hypothesis is logically possible. Given that we have good physical reasons to draw the opposite conclusion in this case, it is generally a mistake to emphasize the possibility.