Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

SilasBarta comments on Localized theories and conditional complexity - Less Wrong

7 Post author: jimmy 19 October 2009 07:29AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (5)

You are viewing a single comment's thread. Show more comments above.

Comment author: SilasBarta 19 October 2009 02:47:28PM 3 points [-]

Hm, I'm not sure what Quine's holism argument does for your point. Quine says that only "science" as a whole can be tested, since you can keep making arbitrary assumptions to salvage any theory. A canonical example might be: They didn't throw out Newton's Law of Gravitation when Saturn (or whatever) had an irregular orbit; they assumed a massive object behind it -- but why not reject the law of gravitation?

But far from being an example of the intractability of the Bayes Theorem, it's actually an example of how Bayesianism can resolve such problems: you build a belief network that connects the theories to the predicted observations. As you encounter new data, you update based on a well-defined rule (based on priors and likelihood ratios), which can be equivalently expressed as Jaynes's Maximum Entropy Method or the KL divergence minimization method.

This process allows you to determine whether a new observation requires you to believe you made an observation error, whether you should expect an additional observation (like the above case with Saturn), or whether you need to make fundamental revisions to the theory.

Of course, that's still just approximating an ideal Bayesian, and conditional independence isn't a required part of it, just a prior you can start from. But Quine's holism argument doesn't show any shortcoming of the use of Bayesian inference in science.

Comment author: taw 20 October 2009 01:17:49AM 1 point [-]

Holism means that for virtually every observation you have to update everything in your network, related or not. You cannot have nice networks with a bunch of small elegant compartments - it will be one huge mess.

My point is that actual Bayesianism is so insanely far beyond capabilities of any imaginable being, and Bayesianism with any assumed independence is no longer correct Bayesianism, that it's not really proper to pretend we're Bayesians in any practical sense.

Comment author: SilasBarta 20 October 2009 02:53:31PM 2 points [-]

Isn't that like saying that Newton's adherents weren't "actual Newtonians" because they assumed away enough bodies to make their computations tractable?

Comment author: jimmy 20 October 2009 02:18:11AM 0 points [-]

Agreed. I changed it to "wannabe" for you :p