Stuart_Armstrong comments on On the Anthropic Trilemma - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (50)
I agree it is troubling if your beliefs don't have any consequences for conceivable decisions, though as far as I know, not fatal.
This alone doesn't seem reason to study anthropics alongside decision theory any more than it is to study biology alongside decision theory. Most of how to make decisions is agreed upon, so it is rare that a belief would only have any consequences under a certain decision theory. There may be other reasons to consider them together however.
As far as the second type of constraint - choosing theories based on their consequences - I don't know why I would expect my intuitions about which decisions I should make to be that reliable relative to my knowledge and intuitions about information theory, probability theory, logic etc. It seems I'm much more motivated to make certain actions than to have correct abstract beliefs about probability (I'd be more wary of abstract beliefs about traditionally emotive topic such as love or gods). If I had a candidate theory which suggested 'unreasonable decisions', I would probably keep looking, but this is mostly because I am (embarrassingly) motivated to justify certain decisions, not because of the small amount of evidence that my intuitions could give me on a topic they are probably not honed for.
I'm not sure why you think there are no constraints on beliefs unless they are paired with a decision theory. Could you elaborate? e.g. why is Bayesian conditionalization not a constraint on the set of beliefs you hold? Could you give me an example of a legitimate constraint?
I haven't participated in UDT-anthropics discussions because working on my current projects seems more productive than looking into all of the research others are doing on topics which may prove useful. If you think this warrants more attention though, I'm listening - what are the most important implications of getting the right decision theory, other than building super-AIs?
"I'm not sure why you think there are no constraints on beliefs unless they are paired with a decision theory."
Because any change to your probability theory can be undone by a change in decision theory, resulting in the same behaviour in the end. The behaviour is where everything pays rent, so its the combination that matters :-)