= 27d567bc2d48d9f40e9ccc20ea370b15)
Vaniver comments on Many Weak Arguments vs. One Relatively Strong Argument - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (86)
Penrose is a worrisome case to bring as an example, since he is in fact wrong, and therefore you're giving an example where your reasoning leads to the wrong conclusion. If you can't easily find examples where your reasoning led you to a new correct conclusion instead of new sympathy toward a wrong conclusion, this is worrisome. In general, I tend to flag recounts of epistemological innovations which lead to new sympathy toward a wrong conclusion, as though the one were displaying compassion for a previously hated enemy, for in epistemology this is not virtue.
The Penrose example worries me for other reasons as well, namely it seems like it would be possible to generate hordes and hordes of weak arguments against Penrose; so it's as if because the argument against Penrose is strong, you aren't bothering to try to generate weak arguments; reading this feels like you now prefer weak arguments to strong arguments and don't try to find the many weak arguments once you see a strong argument, which is not good Bayesianism.
You also claim there's a strong argument for Penrose, namely his authority (? wasn't this the kind of reasoning you were arguing against trusting?) but either we have very different domain models here, or you're not using the Bayesian definition of strong evidence as "an argument you would be very unlikely to observe, in a world where the theory is false". What do you think is the probability of at least one famous physicist writing a widely panned book about the noncomputability of human consciousness, in a world where consciousness is computable? I should not call it very low, and that means that the pure argument from authority, if you don't believe the actual specifics of that argument, is Bayesian evidence with a low likelihood ratio or as it would be commonly termed a 'weak argument'.
Responses below. As a meta-remark, your comment doesn't steelman my argument, and I think that steelmanning arguments helps keep the conversation on track, so I'd appreciate it if you were to do so in the future.
The point of the example is that one shouldn't decisively conclude that Penrose is wrong — one should instead hedge.
Perhaps a relevant analogy is that of the using seat belts to guard against car accidents — one shouldn't say "The claim that I'm going to get into a potentially fatal car accident is in fact wrong, so I'm not going to wear seat belts." You may argue that the relevant probabilities are sufficiently different so that the analogy isn't a good one. If so, I disagree.
There are many such examples. My post extended to a length of eight pages without my going into them, and I wanted to keep the post to a reasonable length. I'm open to the possibility of writing another post with other examples. The reason that I chose the Penrose example is to vividly illustrate the shift in my epistemology.
One would expect this sort of thing to sometimes happen by chance in the course of updating based on incoming evidence. So I don't share your concern.
I can see how the example might seem disconsonant with my post, and will consider revising the post to clarify. [Edit: I did this.] The point that I intended to make is that I was previously unknowingly ignoring certain nontrivial weak lines of evidence, on the grounds that they weren't strong enough, and that I've recognized this, and have been working on modifying my epistemological framework accordingly.
I don't think that the hordes and hordes of weak arguments that you refer to are collectively strong enough to nullify the argument that one should trust Penrose because he's one of the greatest physicists of the second half of the 20'th century.
It's not at all clear to me why this is the case. The argument you give, as I understand it, is "weak arguments, if independent, add nonlinearly instead of linearly, and so we can't safely ignore weak arguments."* But in the case of Penrose, you have a weak argument in his favor (he's really clever), and many strong arguments against him, of which several are independent. The arrow of consilience points against Penrose, and so you should update against Penrose if you've gained a new respect for consilience.
*The argument that we shouldn't ignore arguments because they are below some evidence threshold, to me, falls under "proper epistemic hygiene" and so doesn't seem novel or need to be justified.
It appears that I didn't express myself clearly as well as I would have liked. Thanks for pointing this issue out.
My current epistemological framework is "give weight to all arguments, even the (non-negligibly) weak ones." My prior epistemological framework had been "give weight to all arguments that stand up to scrutiny." I agree that the arrow of consilience points against Penrose. My update is coming from the change "give weight to arguments that don't stand up to scrutiny."
I added an edit to my post explaining this.
I don't think that "Penrose is really clever" is an accurate description of my argument. Lots of people are really clever. I know hundreds of mathematicians who are really clever. Penrose is on a much higher level.
I'm not sure we're using 'scrutiny' in the same way. One potential usage is "if I can think of a counterargument, I can exclude that argument from my analysis," which is one I don't endorse and it sounds like you no longer endorse.
What I think scrutiny is useful for is determining the likelihood ratio of an argument. To use the first argument given in support for the quantitative major, you might estimate the likelihood ratio to be, say, 2:1 in support, and then after correcting for the counterargument of native ability, estimate the effect to be 3:2 in support. (Previously, this would look like revising the 2:1 estimate down to a 1:1 estimate.)
And so in the Penrose example, his suggestion that quantum effects might have something to do with consciousness is, say, 10:1 evidence in favor, because of your esteem for Penrose's ability to think. But when Tegmark comes along and runs the numbers, and finds that it doesn't pan out, I would revise that down to the neighborhood of 101:100. Lots of smart people speculate things could be the case, and then the math doesn't work out.
And so if you have a precise mathematical model of scrutiny, you can incorporate this evidence together without having to deal with rules of thumb like "give weight to arguments that don't stand up to scrutiny," which Eliezer is rightly complaining will often lead you astray.
We're using different standards for cleverness, but the reason I worded things that way is because everyone has access to the same logic. Penrose's intuitions are much more honed than yours in particular areas, and so it's reasonable to use his intuitions as evidence in those areas. But the degree that his intuitions are evidence depends on his skill in that particular area, and if he's able to articulate the argument, then you can evaluate the argument on its own, and then it doesn't matter who made it. I'm reminded of the student who wrote to Feynman complaining that she got a test question wrong because she followed his book, which contained a mistake. Feynman responded with "yep, I goofed, and you goofed by trusting me. You should have believed your teacher's argument, because it's correct."
Yes. I wasn't literally discarding arguments whenever I thought of counterarguments, but I strongly tended in that direction, and I don't endorse this.
I think that these likelihood ratios are too hard to determine with such high precision.
Metaphorically, I agree with this, my skepticism about determining precise numerical estimates not withstanding.
The confidence level in the range of ~ 0.5% sounds about right, up to an order of magnitude in either direction. The issue was that I was implicitly discarding that probability entirely, as if it it was sufficiently small so that it should play no role whatsoever in my thinking.
As far I know, Penrose hasn't fully retracted his position. If so, this should be given some weight.
I don't think that it's fruitful to numerically quantify things in this way, because I think that the initial estimates are poor, and that making up a number makes epistemology worse rather than better, because of anchoring biases. Certainly when I myself have tried to do this in the past, I've had this experience. But maybe I just haven't seen it done right.
My impression from Eliezer's comment is that he's implicitly reasoning in the same way that I was (discarding arguments that have ~ 1% probability of being true, as if they were too unlikely for it to be worthwhile to give any weight to.)
I think that the difference is significant. There's a dearth of public knowledge concerning the depth of the achievements of the best mathematicians and physicists (as well a sa dearth of public knowledge as to who the best mathematicians and physicists are). I think that the benefits to people's epistemology if they appreciated this would be nonnegligible.
Here again lies the key point of contention. The point is that there's a small but non-negligible probability that Penrose isn't able to articulate the argument despite attempting to do so, or that he communicates under bad implicit assumptions about the language that his readers think in, or there's another possibility that I haven't thought of that's consistent with his views being sound.
I'm certainly not saying that one should believe Penrose's views with 50+% probability (the level of confidence that the student in the story seems to have had). I'm saying that one should give the possibility enough credence so that one's world view isn't turned upside down if one learns that one of the hypotheticals that I give above prevails.
My claim is that "the chance that classical computers aren't capable of intelligence is negligible" is an inferior epistemic position to "it seems extremely likely that classical computers are capable of intelligence, but Roger Penrose is one of the greatest scientists of the 20th century, has thought about these things, and disagrees, so one could imagine believing otherwise in the future."