= 27d567bc2d48d9f40e9ccc20ea370b15)
RobbBB comments on Welcome to Less Wrong! (5th thread, March 2013) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (1750)
It's more or less the same reason people call a variety of essentialist positions 'platonism' or 'aristotelianism'. Those aren't the only thinkers to have had views in this neighborhood, but they predated or helped inspire most of the others, and the concepts have become pretty firmly glued together. Similarly, the phrases 'Bayes' theorem' and 'Bayesian interpretation of probability' (whence, jointly, the idea of Bayesian inference) have firmly cemented the name Bayes to the idea of quantifying psychological uncertainty and correctly updating on the evidence. The Bayesian interpretation is what links these theorems to actual practice.
Bayes himself may not have been a 'Bayesian' in the modern sense, just as Plato wasn't a 'platonist' as most people use the term today. But the names have stuck, and 'Laplacian' or 'Ramseyan' wouldn't have quite the same ring.
I like Laplacian as a name better, but it's already a thing.
If I were to pretend that I'm a mainstream frequentist and consider "quantifying psychological uncertainty" to be subjective mumbo-jumbo with no place anywhere near real science :-D I would NOT have serious disagreements with e.g. Vaniver's list. Sure, I would quibble about accents, importances, and priorities, but there's nothing there that would be unacceptable from the mainstream point of view.
My biggest concern with the label 'Bayesianism' isn't that it's named after the Reverend, nor that it's too mainstream. It's that it's really ambiguous.
For example, when Yvain speaks of philosophical Bayesianism, he means something extremely modest -- the idea that we can successfully model the world without certainty. This view he contrasts, not with frequentism, but with Aristotelianism ('we need certainty to successfully model the world, but luckily we have certainty') and Anton-Wilsonism ('we need certainty to successfully model the world, but we lack certainty'). Frequentism isn't this view's foil, and this philosophical Bayesianism doesn't have any respectable rivals, though it certainly sees plenty of assaults from confused philosophers, anthropologists, and poets.
If frequentism and Bayesianism are just two ways of defining a word, then there's no substantive disagreement between them. Likewise, if they're just two different ways of doing statistics, then it's not clear that any philosophical disagreement is at work; I might not do Bayesian statistics because I lack skill with R, or because I've never heard about it, or because it's not the norm in my department.
There's a substantive disagreement if Bayesianism means 'it would be useful to use more Bayesian statistics in science', and if frequentism means 'no it wouldn't!'. But this methodological Bayesianism is distinct from Yvain's philosophical Bayesianism, and both of those are distinct from what we might call 'Bayesian rationalism', the suite of mantras, heuristics, and exercises rationalists use to improve their probabilistic reasoning. (Or the community that deems such practices useful.) Viewing the latter as an ideology or philosophy is probably a bad idea, since the question of which of these tricks are useful should be relatively easy to answer empirically.
Yes, it is my understanding that epistemologists usually call the set of ideas Yvain is referring to "probabilism" and indeed, it is far more vague and modest than what they call Bayesianism (which is more vague and modest still than the subjectively-objective Bayesianism that is affirmed often around these parts.).
BTW, I think this is precisely what Carnap was on about with his distinction between probability-1 and probability-2, neither of which did he think we should adopt to the exclusion of the other.
Err, actually, yes it is. The frequentist interpretation of probability makes the claim that probability theory can only be used in situations involving large numbers of repeatable trials, or selection from a large population. William Feller:
Or to quote from the essay coined the term frequentist:
Frequentism is only relevant to epistemological debates in a negative sense: unlike Aristotelianism and Anton-Wilsonism, which both present their own theories of epistemology, frequentism's relevance is almost only in claiming that Bayesianism is wrong. (Frequentism separately presents much more complicated and less obviously wrong claims within statistics and probability; these are not relevant, given that frequentism's sole relevance to epistemology is its claim that no theory of statistics and probability could be a suitable basis for an epistemology, since there are many events they simply don't apply to.)
(I agree that it would be useful to separate out the three versions of Bayesianism, whose claims, while related, do not need to all be true or false at the same time. However, all three are substantively opposed to one or both of the views labelled frequentist.)
Depends which frequentist you ask. From Aris Spanos's "A frequentist interpretation of probability for model-based inductive inference":
and
For those who can't access that through the paywall (I can), his presentation slides for it are here. I would hate to have been in the audience for the presentation, but the upside of that is that they pretty much make sense on their own, being just a compressed version of the paper.
While looking for those, I also found "Frequentists in Exile", which is Deborah Mayo's frequentist statistics blog.
I am not enough of a statistician to make any quick assessment of these, but they look like useful reading for anyone thinking about the foundations of uncertain inference.
I don't understand what this "probability theory can only be used..." claim means. Are they saying that if you try to use probability theory to model anything else, your pencil will catch fire? Are they saying that if you model beliefs probabilistically, Math breaks? I need this claim to be unpacked. What do frequentists think is true about non-linguistic reality, that Bayesians deny?
I think they would be most likely to describe it as a category error. If you try to use probability theory outside the constraints within which they consider it applicable, they'd attest that you'd produce no meaningful knowledge and accomplish nothing but confusing yourself.
Can you walk me through where this error arises? Suppose I have a function whose arguments are the elements of a set S, whose values are real numbers between 0 and 1, and whose values sum to 1. Is the idea that if I treat anything in the physical world other than objects' or events' memberships in physical sequences of events or heaps of objects as modeling such a set, the conclusions I draw will be useless noise? Or is there something about the word 'probability' that makes special errors occur independently of the formal features of sample spaces?
As best I can parse the question, I think the former option better describes the position.
IIRC a common claim was that modeling beliefs at all is "subjective" and therefore unscientific.
Do you have any links to this argument? I'm having a hard time seeing why any mainstream scientist who thinks beliefs exist at all would think they're ineffable....
Hmm, I thought I had read it in Jaynes' PT:TLoS, but I can't find it now. So take the above with a grain of salt, I guess.
Yes, but frequentists have zero problems with hypothetical trials or populations.
Do note that for most well-specified statistical problems the Bayesians and the frequentists will come to the same conclusions. Differently expressed, likely, but not contradicting each other.
I think they would have significant practical disagreement with #3, given the widespread use of NHST, but clever frequentists are as quick as anyone else to point out that NHST doesn't actually do what its users want it to do.
Hence the importance of the qualifier 'qualitative'; it seems to me that accents, importances, and priorities are worth discussing, especially if you're interested in changing System 1 thinking instead of System 2 thinking. The mainstream frequentist thinks that base rate neglect is a mistake, but the Bayesian both thinks that base rate neglect is a mistake and has organized his language to make that mistake obvious when it occurs. If you take revealed preferences seriously, it looks like the frequentist says base rate neglect is a mistake but the Bayesian lives that base rate neglect is a mistake.
Now, why Bayes specifically? I would be happy to point to Laplace instead of Bayes, personally, since Laplace seems to have been way smarter and a superior rationalist. But the trouble with naming methods of "thinking correctly" is that everyone wants to name their method "thinking correctly," and so you rapidly trip over each other. "Rationalism," for example, refers to a particular philosophical position which is very different from the modal position here at LW. Bayes is useful as a marker, but it is not necessary to come to those insights by way of Bayes.
(I will also note that not disagreeing with something and discovering something are very different thresholds. If someone has a perspective which allows them to generate novel, correct insights, that perspective is much more powerful than one which merely serves to verify that insights are correct.)
Yeah, I said if I were pretend to be a frequentist -- but that didn't involve suddenly becoming dumb :-)
I agree, but at this point context starts to matter a great deal. Are we talking about decision-making in regular life? Like, deciding which major to pick, who to date, what job offer to take? Or are we talking about some explicitly statistical environment where you try to build models, fit them, evaluate them, do out-of-sample forecasting, all that kind of things?
I think I would argue that recognizing biases (Tversky/Kahneman style) and trying to correct for them -- avoiding them altogether seems too high a threshold -- is different from what people call Bayesian approaches. The Bayesian way of updating on the evidence is part of "thinking correctly", but there is much, much more than just that.
At least one (and I think several) of biases identified by Tversky and Kahneman is "people do X, a Bayesian would do Y, thus people are wrong," so I think you're overstating the difference. (I don't know enough historical details to be sure, but I suspect Tversky and Kahneman might be an example of the Bayesian approach allowing someone to discover novel, correct insights.)
I agree, but it feels like we're disagreeing. It seems to me that a major Less Wrong project is "thinking correctly," and a major part of that project is "decision-making under uncertainty," and a major part of uncertainty is dealing with probabilities, and the Bayesian way of dealing with probabilities seems to be the best, especially if you want to use those probabilities for decision-making.
So it sounds to me like you're saying "we don't just need stats textbooks, we need Less Wrong." I agree; that's why I'm here as well as reading stats textbooks. But it also sounds to me like you're saying "why are you naming this Less Wrong stuff after a stats textbook?" The easy answer is that it's a historical accident, and it's too late to change it now. Another answer I like better is that much of the Less Wrong stuff comes from thinking about and taking seriously the stuff from the stats textbook, and so it makes sense to keep the name, even if we're moving to realms where the connection to stats isn't obvious.
Hm... Let me try to unpack my thinking, in particular my terminology which might not match exactly the usual LW conventions. I think of:
Bayes theorem as a simple, conventional, and an entirely uncontroversial statistical procedure. If you ask a dyed-in-the-wool rabid frequentist whether the Bayes theorem is true he'll say "Yes, of course".
Bayesian statistics as an approach to statistics with three main features. First is the philosophical interpretation of (some) probability as subjective belief. Second is the focus on conditional probabilities. Third is the strong preferences for full (posterior) distributions as answers instead of point estimates.
Cognitive biases (aka the Kahneman/Tversky stuff) as certain distortions in the way our wetware processes information about reality, as well as certain peculiarities in human decision-making. Yes, a lot of it it is concerned with dealing with uncertainty. Yes, there is some synergy with Bayesian statistics. No, I don't think this synergy is the defining factor here.
I understand that historically the in the LW community Bayesian statistics and cognitive biases were intertwined. But apart from historical reasons, it seems to me these are two different things and the degree of their, um, interpenetration is much overstated on LW.
Well, we need for which purpose? For real-life decision making? -- sure, but then no one is claiming that stats textbooks are sufficient for that.
Some, not much. I can argue that much of LW stuff comes from thinking logically and following chains of reasoning to their conclusion -- or actually just comes from thinking at all instead of reacting instinctively / on the basis of a gut feeling or whatever.
I agree that thinking in probabilities is a very big step and it *is* tied to Bayesian statistics. But still it's just one step.
I agree with your terminology.
When contrasting LW stuff and mainstream rationality, I think the reliance on thinking in probabilities is a big part of the difference. ("Thinking logically," for the mainstream, seems to be mostly about logic of certainty.) When labeling, it makes sense to emphasize contrasting features. I don't think that's the only large difference, but I see an argument (which I don't fully endorse) that it's the root difference.
(For example, consider evolutionary psychology, a moderately large part of LW. This seems like a field of science particularly prone to uncertainty, where "but you can't prove X!" would often be a conversation-stopper. For the Bayesian, though, it makes sense to update in the direction of evo psych, even though it can't be proven, which is then beneficial to the extent that evo psych is useful.)
Yes, I think you're right.
Um, I'm not so sure about that. The main accusation against evolutionary psychology is that it's nothing but a bunch of just-so stories, aka unfalsifiable post-hoc narratives. And a Bayesian update should be on the basis of evidence, not on the basis of an unverifiable explanation.
It seems to me that if you think in terms of likelihoods, you look at a story and say "but the converse of this story has high enough likelihood that we can't rule it out!" whereas if you think in terms of likelihood ratios, you say "it seems that this story is weakly more plausible than its converse."
I'm thinking primarily of comments like this. I think it is a reasonable conclusion that anger seems to be a basic universal emotion because ancestors who had the 'right' level of anger reproduced more than those who didn't. Boris just notes that it could be the case that anger is a byproduct of something else, but doesn't note anything about the likelihood of anger being universal in a world where it is helpful (very high) and the likelihood of anger being universal in a world where it is neutral or unhelpful (very low). We can't rule out anger being spurious, but asking to rule that out is mistaken, I think, because the likelihood ratio is so significant. It doesn't make sense to bet against anger being reproductively useful in the ancestral environment (but I think it makes sense to assign a probability to that bet, even if it's not obvious how one would resolve it).
I have several problems with this line of reasoning. First, I am unsure what it means for a story to be true. It's a story -- it arranges a set of facts in a pattern pleasing to the human brain. Not contradicting any known facts is a very low threshold (see the Russell's teapot), to call something "true" I'll need more than that and if a story makes no testable predictions I am not sure on which basis I should evaluate its truth and what does it even mean.
Second, it seems to me that in such situations the likelihoods and so, necessarily, their ratios are very very fuzzy. My meta uncertainty -- uncertainty about probabilities -- is quite high. I might say "story A is weakly more plausible than story B" but my confidence in my judgment about plausibility is very low. This judgment might not be worth anything.
Third, likelihood ratios are good when you know you have a complete set of potential explanations. And you generally don't. For open-ended problems the explanation "something else" frequently looks like the more plausible one, but again, the meta uncertainty is very high -- not only you don't know how uncertain you are, you don't even know what you are uncertain about! Nassim Taleb's black swans are precisely the beasties that appear out of "something else" to bite you in the ass.