= 27d567bc2d48d9f40e9ccc20ea370b15)
Qiaochu_Yuan comments on You only need faith in two things - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (86)
Promote it how? By ways of inductive reasoning, to which Bayesian inference belongs. It seems like there's a contradiction between the initially small prior of "induction works" (which is different from inductive reasoning, but still related) and "promote that low-probability hypothesis (that induction works) by ways of inductive reasoning".
If you see no tension there, wouldn't you still need to state the basis for "inductive reasoning works", at least such that its use can be justified (initially)?
Consider the following toy model. Suppose you are trying to predict a sequence of zeroes and ones. The stand-in for "induction works" here will be Solomonoff induction (the sequence is generated by an algorithm and you use the Solomonoff prior). The stand-in for "induction doesn't work" here will be the "binomial monkey" prior (the sequence is an i.i.d. sequence of Bernoulli random variables with p = 1/2, so it is not possible to learn anything about future values of the sequence from past observations). Suppose you initially assign some nonzero probability to Solomonoff induction working and the rest of your probability to the binomial monkey prior. If the sequence of zeroes and ones isn't completely random (in the sense of having high Kolmogorov complexity), Solomonoff induction will quickly be promoted as a hypothesis.
Not all Bayesian inference is inductive reasoning in the sense that not all priors allow induction.
To amplify on Qiaochu's answer, the part where you promote the Solomonoff prior is Bayesian deduction, a matter of logic - Bayes's Theorem follows from the axioms of probability theory. It doesn't proceed by saying "induction worked, and my priors say that if induction worked it should go on working" - that part is actually implicit in the Solomonoff prior itself, and the rest is pure Bayesian deduction.
Doesn't this add "the axioms of probability theory" ie "logic works" ie "the universe runs on math" to our list of articles of faith?
Edit: After further reading, it seems like this is entailed by the "Large ordinal" thing. I googled well orderedness, encountered the wikipedia article, and promptly shat a brick.
What sequence of maths do I need to study to get from Calculus I to set theory and what the hell well orderedness means?
Again, promoted how? All you know is "induction is very, very unlikely to work" (low prior, non 0), and "some single large ordinal is well-ordered". That's it. How can you deduce an inference system from that that would allow you to promote a hypothesis based on it being consistent with past observations?
It seems like putting the hoversled before the bantha (= assuming the explanandum).
Promoted by Bayesian inference. Again, not all Bayesian inference is inductive reasoning. Are you familiar with Cox's theorem?
Only in passing. However, why would you assume those postulates that Cox's theorem builds on?
You'd have to construct and argue for those postulates out of (sorry for repeating) "induction is very, very unlikely to work" (low prior, non 0), and "some single large ordinal is well-ordered". How?
Wouldn't it be: large ordinal -> ZFC consistent -> Cox's theorem?
Maybe you then doubt that consequences follow from valid arguments (like Carroll's Tortoise in his dialogue with Achilles). We could add a third premise that logic works, but I'm not sure it would help.
Can you elaborate on the first step?
I'm no expert in this -- my comment is based just on reading the post, but I take the above to mean that there's some large ordinal for ZFC whose existence implies that ZFC has a model. And if ZFC has a model, it's consistent.