= 27d567bc2d48d9f40e9ccc20ea370b15)
Will_Newsome comments on Evidence for the orthogonality thesis - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (289)
If the AI's map represents the territory accurately enough, the AI can use the map to check the consequences of returning different actions, then pick one action and return it, ipso facto affecting the territory. I think I already know how to build a working paperclipper in a Game of Life universe, and it doesn't seem to wirehead itself. Do you have a strong argument why all non-magical real-world AIs will wirehead themselves before they get a chance to hurt humans?
Perhaps it's also worth bringing up the example of controllers, which don't wirehead (or do they, once sufficiently complex?) and do optimize the real world. (Thermostats confuse me. Do they have intentionality despite lacking explicit representations? (FWIW Searle told me the answer was no because of something about consciousness, but I'm not sure how seriously he considered my question.))
You are looking for intentionality in the wrong place. Why do thermostats exist? Follow the improbability.
Yes, actual thermostats got their shard of the Void from humans, just as humans got their shard of the Void from evolution. (I'd say "God" and not "the Void", but whatever.) But does evolution have intentionality? The point is to determine whether or not intentionality is fundamentally different from seemingly-simpler kinds of optimization—and if it's not, then why does symbol grounding seem like such a difficult problem? ...Or something, my brain is too stressed to actually think.
Taboo "intentionality".
Yes, discerning the hidden properties of "intentionality" is the goal which motivates looking at the edge case of thermostats.