Sebastian_Hagen2 comments on Where Physics Meets Experience - Less Wrong

28 Post author: Eliezer_Yudkowsky 25 April 2008 04:58AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (34)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Sebastian_Hagen2 25 April 2008 01:13:01PM 1 point [-]

Good writing, indeed! I also love what you've done with the Eborrian anzrf (spoiler rot13-encoded for the benefit of other readers since it hasn't been mentioned in the previous comments).

The split/remerge attack on entities that base their anticipations of future input directly on how many of their future selves they expect to get specific input is extremely interesting to me. I originally thought that this should be a fairly straightforward problem to solve, but it has turned out a lot harder (or my understanding a lot more lacking) than I expected. I think the problem might be in the group of 500,003 brains double-counting anticipated input after the merge. They don't stay exactly the same through the merge phase; in fact, for each of the 500,000 brains in green rooms, the re-integrated previously-in-green-rooms brain only depends to a very small part on them individually. In this particular case, the re-integrated brain will still be very similar to each of the pre-integration brains; but that is just a result of the pre-integration brains all being very similar to each other. Treating the re-integrated brain as a regular future-self for the purposes of anticipating future experience under these conditions seems highly iffy to me.