Perplexed comments on Rationality Quotes: April 2011 - Less Wrong

6 Post author: benelliott 04 April 2011 09:55AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (384)
Tags: quotes

Comments (384)

You are viewing a single comment's thread. Show more comments above.

Comment author: Perplexed 07 April 2011 05:34:00PM *  1 point [-]

... we do face the 'further question' of how much moral value to assign ...

Yes, and I did not even attempt to address that 'further question' because it seems to me that that question is at least an order of magnitude more confused than the relatively simple question about consciousness.

But, if I were to attempt to address it, I would begin with the lesson from Econ 101 that dissolves the question "What is the value of item X?". The dissolution begins by requesting the clarifications "Value to whom?" and "Valuable in what context?" So, armed with this analogy, I would ask some questions:

  1. Moral value to whom? Moral value in what context?
  2. If I came to believe that the people around me were p-zombies, would that opinion change my moral obligations toward them? If you shared my belief, would that change your answer to the previous question?
  3. Believed to be conscious by whom? Believed to be conscious in what context? Is it possible that a program object could be conscious in some simulated universe, using some kind of simulated time, but would not be conscious in the real universe in real time?
Comment author: AlephNeil 07 April 2011 07:00:44PM 0 points [-]

The dissolution begins by requesting the clarifications "Value to whom?" and "Valuable in what context?"

My example was one where (i) the 'whom' and the 'context' are clear and yet (ii) this obviously doesn't dissolve the problem.

Comment author: Perplexed 07 April 2011 07:24:14PM *  2 points [-]

It may be a step toward dissolving the problem. It suggests the questions:

  • Is it possible that an intelligent software object (like those in this novella by Ted Chiang) which exists within our space-time and can interact with us might have moral value very different from simulated intelligences in a simulated universe with which we cannot interact in real time?
  • Is it possible for an AI of the Chiang variety to act 'immorally' toward us? Toward each other? If so, what "makes" that action immoral?
  • What about AIs of the second sort? Clearly they cannot act immorally toward us, since they don't interact with us. But can they act immorally toward each other? What is it about that action that 'makes' it immoral?

My own opinion, which I won't try to convince you of, is that there is no moral significance without interaction and reciprocity (in a fairly broad sense of those two words.)

Powered by Reddit Powered by Reddit