brainoil comments on Pascal's Muggle: Infinitesimal Priors and Strong Evidence - Less Wrong

43 Post author: Eliezer_Yudkowsky 08 May 2013 12:43AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (404)

You are viewing a single comment's thread. Show more comments above.

Comment author: earthwormchuck163 06 May 2013 08:17:45AM *  25 points [-]

Mugger: Give me five dollars, and I'll save 3↑↑↑3 lives using my Matrix Powers.

Me: I'm not sure about that.

Mugger: So then, you think the probability I'm telling the truth is on the order of 1/3↑↑↑3?

Me: Actually no. I'm just not sure I care as much about your 3↑↑↑3 simulated people as much as you think I do.

Mugger: "This should be good."

Me: There's only something like n=10^10 neurons in a human brain, and the number of possible states of a human brain exponential in n. This is stupidly tiny compared to 3↑↑↑3, so most of the lives you're saving will be heavily duplicated. I'm not really sure that I care about duplicates that much.

Mugger: Well I didn't say they would all be humans. Haven't you read enough Sci-Fi to know that you should care about all possible sentient life?

Me: Of course. But the same sort of reasoning implies that, either there are a lot of duplicates, or else most of the people you are talking about are incomprehensibly large, since there aren't that many small Turing machines to go around. And it's not at all obvious to me that you can describe arbitrarily large minds whose existence I should care about without using up a lot of complexity. More generally, I can't see any way to describe worlds which I care about to a degree that vastly outgrows their complexity. My values are complicated.

Comment author: brainoil 06 May 2013 12:54:36PM *  2 points [-]

I'm not really sure that I care about duplicates that much.

Didn't you feel sad when Yoona-939 was terminated, or wish all happiness for Sonmi-451?

Comment author: Luke_A_Somers 09 May 2013 10:08:52PM *  4 points [-]

All the other Yoona-939s were fine, right? And that Yoona-939 was terminated quickly enough to prevent divergence, wasn't she?

(my point is, you're making it seem like you're breaking the degeneracy by labeling them. But their being identical is deep)

Comment author: Eliezer_Yudkowsky 09 May 2013 10:37:34PM 2 points [-]

But now she's... you know... now she's... (wipes away tears) slightly less real.

Comment author: Luke_A_Somers 10 May 2013 12:45:57PM 3 points [-]

You hit pretty strong diminishing returns on existence once you've hit the 'at least one copy' point.

Comment author: Jack 10 May 2013 12:51:06AM -1 points [-]

Clones aren't duplicates. They may have started out as duplicates but they were not by the time the reader is introduced to them.