There's a consensus here that conscious computer programs have the same moral weight as people
No there isn't. I would have remembered something like that happened, what with all the disagreeing I would have been doing.
The mindspace of 'conscious computer programs' is unfathomably large and most of those programs are morally worthless. A "some" and/or "could" inserted in there could make the 'consensus' correct.
I may well have overgeneralized. I was basing the idea on remembering an essay saying that FAI should be designed to be non-sentient, and seeing concerns about how uploads and simulated people would be treated.
I suppose that moral concern would apply to any of sentient programs that humans would be likely to create.
In this video, long about 48:00, Eliezer talks about uploading and about how it wouldn't be murder if his meat body were anesthetized before the upload and killed without regaining consciousness.
It's arguable that it wouldn't be murder, but I'm not clear about why Eliezer would want to do it that way. I've got some guesses about why one might want to not let the meat body wake up (legal and practical complications of a double but diverging identity, the meat version feeling hopelessly envious), but I'm not sure whether either of them apply.
On the other hand, I can think of a couple of reasons for *not* eliminating the meat version-- one is that two Eliezers would presumably be better than one, though I don't have a strong intuition about the optimum number of Eliezers. The other, which I consider to be more salient, is that the meat version is a backup in case the upload isn't as good as hoped.
More generally, what would folks here consider to be good enough evidence that uploading was worth doing?