thomblake comments on Terminal Bias - Less Wrong

18 [deleted] 30 January 2012 09:03PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (125)

You are viewing a single comment's thread. Show more comments above.

Comment author: WrongBot 31 January 2012 09:19:29AM 3 points [-]

I currently do not wish to be wireheaded, but I'm uncertain whether that aversion will turn out to be coherent. A technique for preventing AIs from wireheading has not yet been discovered, and there is no proof that one exists. So in that sense LW's thoughts on the subject are necessarily incomplete.

I also don't feel that the least convenient wireheading machine has really been dealt with, that I can recall. Say a machine can really, genuinely convince me that I'm getting all the things that I want and I'm not aware I'm being deceived, so much so that I can't distinguish the real world and the machine, even in principle. I don't know what it would mean for me to say that I didn't prefer that world. What are my preferences about, at that point?

Comment author: thomblake 31 January 2012 09:57:47PM 0 points [-]

even in principle

I'm not sure what motivation there might be to call that 'a wireheading machine' and not 'the universe'.

Comment author: WrongBot 01 February 2012 12:35:11AM 0 points [-]

Exactly.