Peter_de_Blanc comments on Intuitive supergoal uncertainty - Less Wrong

4 Post author: JustinShovelain 04 December 2009 05:21AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (27)

You are viewing a single comment's thread. Show more comments above.

Comment author: Peter_de_Blanc 04 December 2009 07:55:30PM 1 point [-]

Can you give a reference? Because that strikes me as rather un-Jaynesian.

You say that the interval tells us something about how apt the estimate is to move in the face of new evidence. What does it tell us about that? Doesn't it depend on which piece of evidence we're talking about? Do you have to specify a prior over which variables you are likely to observe next?

Comment author: Cyan 04 December 2009 09:00:34PM *  1 point [-]

The material I have in mind is Chapter 18 of PT:LOS. You can see the section headings on page 8 (numbered vii because the title page is unnumbered) here. One of the section titles is "Outer and Inner Robots"; when rhollerith says 72%, he's giving the outer robot answer. To give an account of how unstable your probability estimates are, you need to give the inner robot answer.

What does it tell us about that? Doesn't it depend on which piece of evidence we're talking about?

When we receive new evidence, we assign a likelihood function for the probability. (We take the perspective of the inner robot reasoning about what the outer robot will say.) The width of the interval for the probability tells us how narrow the likelihood function has to be to shift the center of that interval by a non-neglible amount.

Do you have to specify a prior over which variables you are likely to observe next?

No.

Comment author: Eliezer_Yudkowsky 05 December 2009 08:42:18AM 1 point [-]

That is a strange little chapter, but I should note that if you talk about the probability that you will make some future probability estimate, then the distribution of a future probability estimate does make a good way of talking about the instability of a state of knowledge. As opposed to the notion of talking about the probability of a current probability estimate, which sounds much more like you're doing something wrong.

Comment author: Eliezer_Yudkowsky 04 December 2009 08:50:16PM 1 point [-]

Second the question, it doesn't sound Jaynesian to me either.

Comment author: wedrifid 05 December 2009 03:14:06AM 0 points [-]

Second the question, it doesn't sound Jaynesian to me either.

I'm relieved that I'm not the only one who thought that. I was somewhat aghast to hear Jaynes recommend something that is so, well, obviously a bull@# hack.

Comment author: Cyan 05 December 2009 03:48:22AM 0 points [-]

It's curious to me that you'd write this even after I cited chapter and verse. Do you have a copy of PT:LOS?

Comment author: wedrifid 05 December 2009 04:45:40AM *  0 points [-]

It's curious to me that you'd write this even after I cited chapter and verse. Do you have a copy of PT:LOS?

I do have a copy but I will take your word for it. I am shocked and amazed that Jayenes would give such a poor recommendation. It doesn't sound Jaynesian to me either and I rather hope he presents a variant that is sufficiently altered as to not be this suggestion at all. You yourself gave the reason why it doesn't work and I am sure there is a better approach than just hacking the scale when it is near 1 or 0. (I am hoping your paraphrase sounds worse than the original.)

Comment author: timtyler 09 December 2009 04:04:04PM 0 points [-]

Best to give a probabilty density function - but two 2-S-F probabilites typically gives more information than one.

Comment author: timtyler 09 December 2009 04:12:23PM 0 points [-]

It is good to indicate the strength of your priors. Perhaps one could indicate how much you think your opinion is likely to change over some specified timescale - or in response to the next set of pertinent data points.