Psy-Kosh comments on Case study: abuse of frequentist statistics - Less Wrong

25 Post author: Cyan 21 February 2010 06:35AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (96)

You are viewing a single comment's thread. Show more comments above.

Comment author: SilasBarta 21 February 2010 11:03:06PM 0 points [-]

It seems to be the other way around with Bayesian techniques because of a stronger built-in assumption that your assumptions are justified. You can immunize yourself against any particular evidence by having a sufficiently wrong prior.

Someone correct me if I'm wrong here, but I don't think even having a strong prior P(H) against the evidence is much help, because that makes your likelihood ratio on the evidence P(E|H)/P(E|~H) that much stronger.

(This issue is one my stumbling blocks in Bayescraft.)

Comment author: Psy-Kosh 21 February 2010 11:21:17PM 0 points [-]

The likelihood ratio P(E|H)/P(E|~H) is entirely independent of the prior P(H)

Or did I misunderstand what you said?

Comment author: SilasBarta 21 February 2010 11:39:35PM *  0 points [-]

The likelihood ratio P(E|H)/P(E|~H) is entirely independent of the prior P(H)

In theory, yes, but we're talking about a purported "unswayable Bayesian". If someone strongly believes leprechauns don't exist (low P(H), where H is "leprechauns exist" ), they should strongly expect not to see evidence of leprechauns (low P(E|~H), where E is direct evidence of leprechauns, like finding one in the forest), which suggests a high likelihood ratio P(E|H)/P(E|~H).

I remember Eliezer Yudkowsky referring to typical conversations that go like:

Non-rationalist: "I don't think there will ever be an artificial general intelligence, because my religion says that can't happen."
EY: "So if I showed you one, that means you'd leave your religion?"

Comment author: RobinZ 21 February 2010 11:58:42PM 1 point [-]

He did mention pulling that off once, but I don't believe he said it was typical.

Comment author: SilasBarta 22 February 2010 12:53:51AM 0 points [-]

Thanks, that was what I had in mind.

Comment author: Psy-Kosh 22 February 2010 12:29:20AM 0 points [-]

I'm not entirely sure I understand your point. The example you're citing is more the guy saying "I believe X, and X implies ~Y, therefore ~Y", so Eliezer is saying "So Y implies ~X then?"

But the "X implies ~Y" belief can happen when one has low belief in X or high belief in X.

Or are you saying "the likelihoods assigned led to past interpretation of analogous (lack of) evidence, and that's why the current prior is what it is?

Comment author: SilasBarta 22 February 2010 12:37:41AM 0 points [-]

komponisto nailed the intuition I was going from: the likelihood ratio is independent of the prior, but an unswayable Bayesian fixes P(E), forcing extreme priors to have extreme likelihood ratios.

Comment author: Psy-Kosh 22 February 2010 12:48:35AM 2 points [-]

*blinks* I think I'm extra confused. The law of conservation of probability is basically just saying that the change in belief may be large or small, so evidence may be strong or weak in that sense. But that doesn't leave the likelihoods up for grabs, (well, okay, P(E|~H) could depend on how you distribute your belief over the space of hypotheses other than H, but... I'm not sure that was your point)

Comment author: SilasBarta 22 February 2010 01:23:51AM *  1 point [-]

Okay, point conceded ... that still doesn't generate a result that matches the intuition I had. I need to spend more time on this to figure out what assumptions I'm relying on to claim that "extremely wrong beliefs force quick updates".

Comment author: komponisto 22 February 2010 12:29:17AM *  0 points [-]

P(E) = P(E|H) P(H) + P(E|~H)P(~H)

The quantities P(H), P(E|H), and P(E|~H) are in general independent of each other, in the sense that you can move any one of them without changing the others -- provided you adjust P(E) accordingly.

Comment author: SilasBarta 22 February 2010 12:38:22AM 0 points [-]

Thanks, that helps. See how I apply that point in my reply to Psy-Kosh here.