skeptical_lurker comments on Connection Theory Has Less Than No Evidence - Less Wrong

17 Post author: WilliamJames 01 August 2014 10:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (48)

You are viewing a single comment's thread. Show more comments above.

Comment author: skeptical_lurker 01 August 2014 02:11:58PM -2 points [-]

poorly conducted, unpublished case studies

So, there is evidence for. Its just really poor quality evidence.

Comment author: MrMind 01 August 2014 04:38:16PM 0 points [-]

Nay. Remebember that any evidence must be weighted against competing theories. If a case supports Connection Theory but supports better Foobar Theory, then those cases are evidence against CT.

Comment author: ThisSpaceAvailable 02 August 2014 01:19:16AM 0 points [-]

But one of those theories is the null hypothesis.

Suppose I have a coin, and am considering the following hypotheses:

H1 It is a fair coin H2 The coin comes up heads 80% of the time H3 The coin comes up head 95% of the time

Suppose I flip the coin and get heads.

*Condition A: I was originally split between H2 and H3 (prior of .50 for both).

The coin coming up heads is consistent with both hypotheses, but it's more consistent with H3, so I will raise my confidence in H3. Since probabilities must add up to 1, I have to lower my confidence in H2. (If I've calculated correctly, P(H2) is now about .46)

*Condition B: I was strongly predisposed towards H1. So, say, .70 for H1, .15 for H2 and H3.

Posterior probabilities: P(H1) = .57, P(H2) = .20, P(H3) = .23.

*Condition C: Priors P(H1) = .70 P(H2) = .25 P(H3) = .05

Posterior probabilities: P(H1) = .59, P(H2) = .33, P(H3) = .08.

So, yes, if the only two hypotheses are CT and FT, and the evidence supports FT more than CT, then it's evidence against CT. But if there are other hypotheses (and surely there are), then a case can be evidence for both CT and FT. And if CT has a higher prior than FT, then CT can end up with a higher posterior.

It appears that you may have a skewed idea of how Bayesian reasoning works. When doing a Bayesian calculation to find out the posterior for a hypothesis, one does not compare the hypothesis against a particular competing hypothesis; one compares the hypothesis against the entirety of alternative hypotheses. It's not the case that the hypothesis that best fits the data gets its probability increased and every other hypothesis has its probability decreased. In fact, optimizing simply for the hypothesis that fits the data best, without any concern for other criteria (such as hypothesis complexity), is generally recognized as a serious problem, and is known as "overfitting".

Comment author: MrMind 02 August 2014 09:05:00AM *  0 points [-]

When doing a Bayesian calculation to find out the posterior for a hypothesis, one does not compare the hypothesis against a particular competing hypothesis; one compares the hypothesis against the entirety of alternative hypotheses.

Yes, but in the case of "the totality of the theories of the mind", this is impossible. You cannot possibly calculate P("poorly conducted, unpublished case studies"| not CT) to get a likelihood ratio. Plus, if you admit more than two competing hypothesis, you lose the additivitiy of log-odds and make the calculations even more complicated.
If you want any hope to get a real Bayesian analysis of CT, the only possible way is to compare it against the best possible alternative.

It appears that you may have a skewed idea of how Bayesian reasoning works.

I think you underestimated my understanding of how Bayesian reasoning works or that I underestimated the inferential distances for Bayesian calculations done in a real case.

So, yes, if the only two hypotheses are CT and FT, and the evidence supports FT more than CT, then it's evidence against CT.

Which is exactly what I said.