Bakkot comments on Welcome to Less Wrong! (2012) - Less Wrong

25 Post author: orthonormal 26 December 2011 10:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (1430)

You are viewing a single comment's thread. Show more comments above.

Comment author: Bakkot 07 June 2012 04:10:34AM 1 point [-]

I'm having a hard time figuring out what you mean when you say that example isn't available to you. Are you familiar with the processes by which source code becomes programs we can execute? (I imagine you are, but if not, I suggest you read up on it - this is something everyone can benefit from a bare familiarity with, I think.)

This seems by far the most appropriate example, so I'm not going to readily give it up. I'd be happy to give a more detailed example, if you'd like - or, possibly better, you could give an example of a case where it would be OK to cancel the compilation process.


We know that the past is completely screened off by the present, so our decisions, including moral decisions, can't ultimately depend on it.

Extremely strongly disagree. There is absolutely no reason to exclude the past from our morality.

Here's just one trivial but hopefully obvious example of why the past is important: if everyone follows just the rule I listed, no one has to worry about getting killed.

Of course, that said, I can still try to present the rule another way. For example, we might say that the rule is that it's immoral to destroy anything which contains within it a complete and unique description (that it to say, it's the only copy) of a person. Thus the passed-out drunk is a person, the running AI is a person, but the compiling AI is not. (Nor is the baby, incidentally.)


Again, the compiling AI seems like an extremely useful example. I'm having difficulty coming up with any rule which includes the AI but not the baby, in fact. (Without referring to things that seems unrelated like "biologically alive", of course.) As such, I'd really like to discuss it with you. Could you do me a favor and explain what would need to happen for you to be able to discuss it?