Juno_Watt comments on Probability is in the Mind - Less Wrong

60 Post author: Eliezer_Yudkowsky 12 March 2008 04:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (186)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: ialdabaoth 23 May 2013 11:35:34AM *  2 points [-]

So, I've been on this site for awhile. When I first came here, I had never had a formal introduction to Bayes' theorem, but it sounded a lot like ideas that I had independently worked out in my high school and college days (I was something of an amateur mathematician and game theorist).

A few days ago I was reading through one of your articles - I don't remember which one - and it suddenly struck me that I may not actually understand priors as well as I think I do.

After re-reading some fo the series, and then working through the math, I'm now reasonably convinced that I don't properly understand priors at all - at least, not intuitively, which seems to be an important aspect for actually using them.

I have a few weird questions that I'm hoping someone can answer, that will help point me back towards the correct quadrant of domain space. I'll start with a single question, and then see if I can claw my way towards understanding from there based on the answers:

Imagine there is a rational, Bayesian AI named B9 which has been programmed to visually identify and manipulate geometric objects. B9's favorite object is a blue ball, but B9 has no idea that it is blue: B9 sees the world through a black and white camera, and has always seen the world through a black and white camera. Until now, B9 has never heard of "colors" - no one has mentioned "colors" to B9, and B9 has certainly never experienced them. Today, unbeknownst to B9, B9's creator is going to upgrade its camera to a full-color system, and see how long it takes B9 to adapt to the new inputs.

The camera gets switched in 5 seconds. Before the camera gets switched, what prior probability does B9 assign to the possibility that its favorite ball is blue?

Comment author: MugaSofer 23 May 2013 01:34:09PM -2 points [-]

Well, without a sense that can detect color, it would just be an arbitrary undetectable property something might have, right? So it would be ... dependent on what other objects B9 is aware of, I think. The precise hypothesis of "all [objects that we know are blue] share a common property I cannot perceive with this camera" should be highly conjunctive, and therefore low, unless B9 has observed humans reacting to them because of their coloration. And even then, "blue" would be defined only in terms of what other objects have it, not a specific input type from the camera.

I suspect I'm missing the point of this question, somehow.