You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Nornagest comments on [Open Thread] Stupid Questions (2014-02-17) - Less Wrong Discussion

3 Post author: solipsist 17 February 2014 05:34AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (136)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nornagest 28 February 2014 05:52:21PM *  3 points [-]

I no longer find it totally implausible that imagined people might, if modeled in enough detail, be in some sense conscious -- it seems unlikely to me that human self-modeling and other-modeling logic would end up being that different -- but even if we take that as given, there's a couple of problems with threatening to imagine someone in some unpleasant situation.

The basic issue is asymmetry of information. You might be able to imagine someone that thinks or even reliably acts like your enemy; but, no matter how good you are at personality modeling, they aren't going to have access to all, or even much, of your enemy's memories and experiences. Lacking that, I wouldn't say your imagined enemy is cognitively equivalent to your real enemy in a way that'd make the threat hold up.

(Skynet, by contrast, might be able to reproduce all that information by some means -- brain scanning, say, or some superhuman form of induction.)