You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Wei_Dai comments on [LINK] Could a Quantum Computer Have Subjective Experience? - Less Wrong Discussion

16 Post author: shminux 26 August 2014 06:55PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (55)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 27 August 2014 10:08:15PM 4 points [-]

“Our starting assumptions are probably right, ergo we can say with some confidence that the future will involve trillions of identical uploaded minds maximizing their utility functions, unless of course the Paperclip-Maximizer ‘clips’ it all in the bud”

I accepted the importance and correctness of their inference, but I ran it in the opposite direction:

“It seems obvious that we can’t say such things with any confidence, ergo the starting assumptions ought to be carefully revisited—even the ones about mind and computation that most scientifically-literate people say they agree with.”

I don't see how Scott's proposed revision of the starting assumptions actually changes the conclusions. Even if he is right that uploads and AIs with a "digital abstraction layer" can't be conscious, that's not going to stop a future involving trillions of uploads, or stop paperclip maximizers.

Comment author: shminux 27 August 2014 10:19:12PM 0 points [-]

If these uploads are p-zombies (Scott-zombies?) because they are reversible computations, then their welfare doesn't matter. I don't think he says that it prevents paperclip maximizers.

Comment author: Wei_Dai 27 August 2014 10:30:08PM 1 point [-]

So Scott meant to argue against "the future should involve trillions of uploads" rather than "the future will involve trillions of uploads"?

Comment author: shminux 27 August 2014 11:03:40PM *  0 points [-]

He suggests that all those uploads might not be conscious if they are run on a quantum computer reversibly (or have some other "clean digital abstraction layer"). He states that this is a huge speculation, but it is still an alternative not usually considered by the orthodox reductionists.