TheOtherDave comments on an ethical puzzle about brain emulation - Less Wrong

14 Post author: asr 13 December 2013 09:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (55)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheOtherDave 15 December 2013 07:32:38PM 0 points [-]

OK.

So a system S is computed in such a way that some interesting computational property (C) arises, and all the interim S-states are cached. I then execute a process P that at every step might be recomputing S or might be looking up the cached S-state, in such a way that no outside observer can tell the difference via any conceivable test. Yes?

So, sure, P might or might not cause C to arise, and we have no way of telling which is the case.

I'm not quite sure why this is particularly a problem for moral consequentialism. If C arising is a consequence we prefer to avoid, executing P in the first place is morally problematic in the same way that playing Russian Roulette is... it creates the possibility of a future state we prefer to avoid. And creating the cached S-states was definitely an act we prefer to avoid, and therefore immoral on moral-consequentialist grounds.