SilasBarta comments on Decision theory: An outline of some upcoming posts - Less Wrong

24 Post author: AnnaSalamon 25 August 2009 07:34AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (30)

You are viewing a single comment's thread. Show more comments above.

Comment author: SilasBarta 28 August 2009 02:36:22PM 0 points [-]

I assumed as much, but my problem with this reasoning starts here:

To categorize something as an agent, you need to look at its internal dynamics. A planet is not an agent because it's not doing any computation that could be interpreted as evaluating multiple possible choices, and it's certainly not predicting their consequences.

Normally, I'd agree, but as I said, Eliezer_Yudkowsky claims that a pebble contains the laws of physics, which are nothing but a network of counterfactuals. So there necessarily is an isomorphism between a planet and "multiple possible consequences".

This is why I say there must be a stronger sense in which you mean that the agent has computations that can be interpreted as evaluating multiple choices/consequences, because all of the universe is doing a sort of efficient version of that. And I don't yet know what this stronger sense is.