You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

RichardKennaway comments on Why No Wireheading? - Less Wrong Discussion

16 [deleted] 18 June 2011 11:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (112)

You are viewing a single comment's thread. Show more comments above.

Comment author: RichardKennaway 19 June 2011 04:22:29PM 2 points [-]

Rewards already do matter. It describes people's behavior well (see PCT) and makes introspective sense.

PCT is not good to cite in this connection. PCT does not speak of rewards. According to PCT, behaviour is performed in order to control perceptions, i.e. to maintain those perceptions at their reference levels.

While it is possible for a control system to be organised around maximising something labelled a reward (or minimising something labelled a penalty), that is just one particular class of possible ways of making a control system. Unless one has specifically observed that organisation, there are no grounds for concluding that reward is involved just because something is made of control systems.

Comment author: [deleted] 20 June 2011 06:03:51PM 0 points [-]

Good point, I oversimplified here. I will consider this in more detail, but it naively, isn't this irrelevant in terms of wireheading? Maintaining perceptions is maybe a bit trickier to do, but there would still be obvious shortcuts. Maybe if these perceptions couldn't be simplified in any relevant way, then we'd need at least a full-on matrix and that would disqualify wireheading.