= 27d567bc2d48d9f40e9ccc20ea370b15)
quanticle comments on The Moral Status of Independent Identical Copies - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (71)
The issue here is that the setup has a firm answer - A - but if it were tweaked ever so slightly, the whole preferences would change.
First of all, consider the following:
A': there is one genuine simulation, and the other 99 are simple copies of that one. Soon, all the copies will be stopped, but the the true simulation will continue.
There are essentially no practical ways of distinguishing A from A'. So we should reason as if A' were correct, in which case nothing is lost from the turning off.
However, if we allow any divergence between the copies, then this is a whole other kettle of barreled fish. Now there are at least a hundred copies of each person, and they are distinct: they think slightly differently, behave slightly differenctly, have slightly different goals, etc... And this divergence will only grow with time.
There are three ways I can think of addressing this:
1) Once copies diverge at all, they are different people. So any divergence results in multiplying by a hundred the amount of people in the universe. Hence, if we suspect divergence, we should treat each copy as totally distinct.
2) An information theoretic approach: given one individual, how much new information is needed to completely descibe another copy. Under this, a slightly divergent copy counts as much less than a completely new individual, while a very divergent copy is nearly entriely different.
Both these have drawbacks - 1) gives too much weight to insignificant and minute changes, and 2) implies that people in general are of nothing like approximately equal worth: only extreme exceentrics count as complete people, others are of much less importance. So I suggest a compromise:
3) An information theoretical cut off: two slightly divergent copies count as only slightly more than a single person, but once this divergence rises above some critical amount, the two are treated as entirely seperate individuals.
What is the relevance of this to the original problem? Well, as I said, the original problem has a clear-cut solution, but it is very close to the different situation I described. So, bearing in mind imperfect information, issues of trust and uncertainty, and trying to find similar situations to similar problems, I think we should treat the set-up as the "slightly divergent one". In this situation, A still dominates, but the relative attraction of B rises as the divergence grows.
EDIT: didn't explain the cutoff properly; I was meaning a growing measure of difference that hits a maximum at the cutoff point, and doesn't grow any further. My response here gives an example of this.
The problem with option 3 is that its fundamentally intuitionist, with arbitrary cutoffs distinguishing "real" individuals from copies. I mean, is there really such a big difference between cutoff - .001 difference and cutoff + .001 difference? There isn't. Unless you can show that there's a qualitative difference that occurs when that threshold is crossed, its much more elegant to look at a distinction between options 1 and 2 without trying to artificially shift the boundary between the two.
Didn't phrase clearly what I meant by cut-off.
Let D be some objective measure of distance (probably to do with Kologomorov complexity) between individuals. Let M be my moral measure of distance, and assume the cut-off is 1.
Then I would set M(a,b) = D(a,b) whenever D(a,b) < 1, and M(a,b) = 1 whenever D(a,b) >= 1. The discontinuity is in the derivative, not the value.
That doesn't resolve quanticle's objection. Your cutoff still suggests that a reasonably individualistic human is just as valuable as, say, the only intelligent alien being in the universe. Would you agree with that conclusion?
No. I grant special status to exceedingly unique minds, and to the last few of a given species.
But human minds are very similar to each other, and granting different moral status to different humans is a very dangerous game. Here, I am looking at the practical effects of moral systems (Eliezer's post on "running on corrupted hardware" is relevant). The thoeretical gains of treating humans as having varrying moral status are small; the practical risks are huge (especially as our societies, though cash, reputation and other factors, is pretty good at distinguishing between people without having to further grant them different moral status).
One cannot argue: "I agree with moral system M, but M has consequence S, and I disagree with S". Hence I cannot agree with granting people different moral status, once they are sufficiently divergent.