= 27d567bc2d48d9f40e9ccc20ea370b15)
V_V comments on Solomonoff Cartesianism - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (45)
With respect, no, we do not wirehead in any mathematical sense. Human beings recognize that wireheading (ex: heroin addiction) is a corruption of our valuation processes. Some people do it anyway, because they're irrational, or because their decision-making machinery is too corrupted to stop on their own, but by and large human addicts don't want to be addicts and don't want to want to be addicts.
In contrast, AIXI wants to wirehead, and doesn't want to want anything (because it has no second-order thought processes).
If we understood what was going on cognitively in humans who don't want to wirehead, people like Daniel Dewey, myself and Luke who are trying to think about AI goal-systems that won't wirehead would get a lot further. Some "anti-wireheading math", for instance, would make value learners and reinforcement learners follow their operators' intentions more closely than they currently can be modelled as doing.
What do you mean by wireheading in "mathematical sense"?
Humans clearly aren't simple reinforcement learning agents. They tend have at least a distinction between short-term pleasure and long-term goals, and drugs mostly affect the former but not the latter.
Most drug addicts are ego-dystonic w.r.t. drugs: they don't want to be addicts. This means that addiction hasn't completely changed their value system as true wireheading would.
However, to the extent that humans can be modelled as reinforcement learning agents, they tend to display reward channel manipulation behaviors.
The basic issue, I would have to say, is that human beings have multiple valuation systems. If I had to guess, "we" evolved reinforcement learning behaviors back during the "first animals with a brain" stage, and developed more complex and harder-to-fake valuation systems on top of that further down the line.
In the sense of preference solipsism.