= 27d567bc2d48d9f40e9ccc20ea370b15)
Giles comments on Why No Wireheading? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (112)
This is just my opinion, not particularly evidence-based: I don't think that there are two different kinds of mind, or if there are it's not this issue that separates them. The wireheading scenario is one which is very alien to our ancestral environment so we may not have an "instinctive" preference for or against it. Rather, we have to extrapolate that preference from other things.
Two heuristics which might be relevant:
I can very well believe that wireheading correlates with personality type, which is a weak form of your "two different minds" hypothesis.
Sorry for the ultra-speculative nature of this post.
Makes sense in terms of explaining the different intuition, yes, and is essentially how I think about it.
The second heuristic about manipulation, then, seems useful in practice (more agents will try to exploit us than satisfy us), but isn't it much weaker, considering the actual wireheading scenario? The first heuristic actually addresses the conflict (although maybe the wrong way), but the second just ignores it.
I agree; the second heuristic doesn't apply particularly well to this scenario. Some terminal values seem to come from a part of the brain which isn't open to introspection, so I'd expect them to arise as a result of evolutionary kludges and random cultural influences rather than necessarily making any logical sense.
The thing is, once we have a value system that's reasonably stable (i.e. what we want is the same as what we want to want) then we don't want to change our preferences even if we can't explain where they arise from.