Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Z._M._Davis comments on That Alien Message - Less Wrong

111 Post author: Eliezer_Yudkowsky 22 May 2008 05:55AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (164)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Z._M._Davis 23 May 2008 04:50:25AM 4 points [-]

Unknown, I'm surprised at you. The AI could easily say "I know that ..." while neither being nor claiming to be conscious. When a human speaks in the first person, we understand them to be referring to a conscious self, but an unconscious AI could very well use a similar pattern of words merely as a user-friendly (Friendly?) convenience of communication, like Clippy. (Interestingly, the linked article dilvulges that Clippy is apparently a Bayesian. The reader is invited to make up her own "paperclip maximizer" joke.)

Furthermore, I don't think the anti-zombie argument, properly understood, really says that no unconscious entity could claim to be conscious in conversation. I thought the conclusion was that any entity that is physically identical (or identical enough, per the GAZP) to a conscious being, is also conscious. Maybe a really good unconscious chatbot could pass a Turing test, but it would necessarily have a different internal structure from a conscious being: presumably given a sufficiently advanced cognitive science, we could look at its inner workings and say whether it's conscious.

Comment author: pnrjulius 09 April 2012 05:00:55AM 0 points [-]

Hell, I can write a Python script in five minutes that says it knows all those things. In a few weeks, I could write one that solves Peano arithmetic and generates statements like that ad infinitum. But will it be conscious? Not a chance.