Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Vaniver comments on Probability is in the Mind - Less Wrong

60 Post author: Eliezer_Yudkowsky 12 March 2008 04:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (186)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Vaniver 24 May 2013 07:26:10PM 2 points [-]

"The ball is blue" only gets assigned a probability by your prior when "blue" is interpreted, not as a word that you don't understand, but as a causal hypothesis about previously unknown laws of physics allowing light to have two numbers assigned to it that you didn't previously know about, plus the one number you do know about.

Note that a conversant AI will likely have a causal model of conversations, and so there are two distinct things going on here- both "what are my beliefs about words that I don't understand used in a sentence" and "what are my beliefs about physics I don't understand yet." This split is a potential source of confusion, and the conversational model is one reason why the betting argument for quantifying uncertainties meets serious resistance.

Comment author: Eliezer_Yudkowsky 24 May 2013 08:25:52PM 5 points [-]

To me the conversational part of this seems way less complicated/interesting than the unknown causal models part - if I have any 'philosophical' confusion about how to treat unknown strings of English letters it is not obvious to me what it is.