MugaSofer comments on Dissolving the Question - Less Wrong

44 Post author: Eliezer_Yudkowsky 08 March 2008 03:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (104)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: SecondWind 27 April 2013 11:51:14PM 7 points [-]

'Free will' is the halting point in the recursion of mental self-modeling.

Our minds model minds, and may model those minds' models of minds, but cannot model an unlimited sequence of models of minds. At some point it must end on a model that does not attempt to model itself; a model that just acts without explanation. No matter how many resources we commit to ever-deeper models of models, we always end with a black box. So our intuition assumes the black box to be a fundamental feature of our minds, and not merely our failure to model them perfectly.

This explains why we rarely assume animals to share the same feature of free will, as we do not generally treat their minds as containing deep models of others' minds. And, if we are particularly egocentric, we may not consider other human beings to share the same feature of free will, as we likewise assume their cognition to be fully comprehensible within our own.

...d-do I get the prize?

Comment author: MugaSofer 29 April 2013 02:03:02PM -2 points [-]

You know, that fits. We often fail to ascribe free will to others, talking about how "that's not like him" and making the Fundamental Attribution Error ("he's a murderer - he's evil!")

This means we have to ascribe free will to any sufficiently intelligent agent that knows about our existence, right? Because they'll be modelling us modeling them modelling us?