So, one more litany, hopefully someone else finds it as useful.

It's an understatement that humility is not a common virtue in online discussions, even, or especially when it's most needed.

I'll start with my own recent example. I thought up a clear and obvious objection to one of the assertions in Eliezer's critique of the FAI effort compared with the Pascal's Wager and started writing a witty reply. ...And then I stopped. In large part because I had just gone through the same situation, but on the other side, dealing with some of the comments to my post about time-turners and General Relativity by those who know next to nothing about General Relativity. It was irritating, yet here I was, falling into the same trap. And not for the first time, far from it. The following is the resulting thought process, distilled to one paragraph.

I have not spent 10,000+ hours thinking about this topic in a professional, all-out, do-the-impossible way. I probably have not spent even one hour seriously thinking about it. I probably do not have the prerequisites required to do so. I probably don't even know what prerequisites are required to think about this topic productively. In short, there are almost guaranteed to exist unknown unknowns which are bound to trip up a novice like me. The odds that I find a clever argument contradicting someone who works on this topic for a living, just by reading one or two popular explanations of it are minuscule. So if I think up such an argument, the odds of it being both new and correct are heavily stacked against me. It is true that they are non-zero, and there are popular examples of non-experts finding flaws in an established theory where there is a consensus among the experts. Some of them might even be true stories. No, Einstein was not one of these non-experts, and even if he were, I am not Einstein.

And so on. So I came up with the following, rather unpolished mantra:

If I think up what seems like an obvious objection, I will resist assuming that I have found a Weaksauce Weakness in the experts' logic. Instead I may ask politely whether my argument is a valid one, and if not, where the flaw lies.

If you think it useful, feel free to improve the wording.

New Comment
71 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]gwern300

A little Learning is a dang'rous Thing;
Drink deep, or taste not the Pierian Spring:
There shallow Draughts intoxicate the Brain,
And drinking largely sobers us again.

--Alexander Pope

2Shmi
That sounds just about perfect. Mind adding it to the wiki as a "Litany of Alexander Pope" or something, for easy reference?
2gwern
Let's see if it gets any uptake before we do any canonizing.
1Yitz
Consider me a fan

Over the last few years I've found that
1) I notice this problem in myself and others more and more
2) Reminding myself of it helps keep my identity small
3) People strongly resist not doing it, people are mistrustful of experts

My formulation: Oh, so you've overturned decades of investigation by professionals with your 30 seconds of poorly worded thoughts, watchout we have a badass over here. (note: I only use one this caustic on myself)

OTOH: there are areas where a cursory investigation reveals that no one has been doing much rigorous work and it's easy to disregard established opinion.

The odds that I find a clever argument contradicting someone who works on this topic for a living, just by reading one or two popular explanations of it are minuscule.

There's different kinds of "works for a living", e.g. I write software for a living and employ statistical methods for a living, and it has to actually work, not merely look plausible, or I will not get paid. I also do art for a living, and here it has to look plausible, and if I do very plausible looking clouds and mountains that doesn't make me much of a meteorology expert or a geophysicist. (It does make me a bit of an expert because human visual system is amazingly sensitive to things being wrong, but only a bit, and its very superficial expertise).

Ohh, and also: actually, my expertise in photo-realistic CGI should actually lower the strength of any photographic evidence I present.

Are you saying the clouds did not spell "FAI is scam"?

1maia
On a completely off-topic note: I find this interesting, because I've toyed with the idea of trying to do both a technical job and some sort of artistic thing on the side (but am not sure how I'd go about the latter). How is that working out for you?
8private_messaging
I have it rather mixed, actually. I write computer graphics software (which often employs some quite advanced Monte-Carlo method, or tries to approximate one). And I use it too. There's the site . Works out fine, I ended up making quite good money making a videogame.
1jsteinhardt
When I try to go to the site it says the certificate is untrusted.
7private_messaging
Fixed, I accidentally used https because i got there from admin interface.
[-]gwern240

"Somebody says, 'You know, you people always say that space is continuous. How do you know when you get to a small enough dimension that there really are enough points in between, that it isn't just a lot of dots separated by little distances?'

Or they say, 'You know those quantum mechanical amplitudes you told me about, they're so complicated and absurd, what makes you think those are right? Maybe they are not right.'

Such remarks are obvious and are perfectly clear to anybody who is working on this problem. It does not do any good to point this out."

--Richard Feynman, The Character of Physical Law (1965)

-3Shmi
I assume this is in support of the OP, though I have trouble gleaning your position from just the quote. Well, it probably works as a status-raising trick, "if you are sufficiently smart, you ought to know what I meant".
4wedrifid
What the? That is a candidate for the most dumbfounding accusation of "status-raising trick" that I have ever seen. Most people would understand the kind of reasons why gwern is providing an on topic quote to the thread. That being the case gwern would not anticipating raising his status via refraining from stating the obvious (or at least what is believed to be obvious). There is no trick here. Just a projection of personal confusion onto the intention of another.

dealing with some of the comments to my post about time-turners and General Relativity

Next up: The Litany of the Déformation professionnelle.

One of the most robust findings in studies of expertise is that unless the expertise is in callibration itself (eg. gambling) experts are systematically overconfident. In particular they overestimate the relevence of their expertise to things that are not central to their actual professional or academic practice. For example a neurobiologist with no training in information theory may tend to place inordinate trust in their intuition "it is theoretically impossible to recover information from that brain" without realising how different that question is from the fact that they actually do know, "it isn't possible for us to repair that brain to functioning condition in place".

In the case of the time turner thread that motivates this post we have an example of believing that some knowledge of General Relativity is sufficient to make one an expert at solving engineering problems in a counterfactual physical reality rather than merely being necessary. The most insightful comments in that thread don't do anything to challenge ... (read more)

5buybuydandavis
This. Expertise is not integrated instantaneously across fields, and to the extent one field relies on another, it can be making basic mistakes corrected elsewhere decades ago.
7IlyaShpitser
"Send in the clergy, they can move diagonally!" http://www.smbc-comics.com/index.php?db=comics&id=1776
[-][anonymous]150

Dear myself from several years ago: Please read this.

6Kawoomba
Careful with the space-time-continuum!
5DaFranker
Remember that one time when you found a post-it on your fridge in your handwriting saying "DON'T TRY THAT AGAIN" that you didn't remember putting there? You 'just' put it there.

Good post. Experts on X are usually right about X. What's worse, if an expert in X and an expert in a closely related Y disagree about Y, the latter is usually right (or has a stronger position).

It's not easy being a small fish in a big intellectual sea, but that's how it is for us all, even the most brilliant.

Experts on X are usually right about X

That's true in many fields but not others. Experts in "New Testament studies" are usually wrong about whether Jesus had magical powers.

6IlyaShpitser
Here is a series of papers from the journal of new testament studies: http://journals.cambridge.org/action/displayJournal?jid=NTS Not a lot of mention of actual magical powers in here. Note the language: "Drawing on the book of Job, we show that the disciplinary practice Paul advocates in 1 Corinthians 5 is a spiritual practice that aims to remove the spiritual protection enjoyed by the incestuous man while he remained in the body of Christ, thereby exposing him to Satan's attacks. Paul's hope was that the affliction suffered by the man at the hands of Satan as a result of this exposure would lead to his repentance and ultimate salvation." Paul's hope.
2lukeprog
I'm saying that if you took a poll of everyone at one of the major New Testament Studies conferences, the majority would say that Jesus had magical powers, and they'd be wrong.
[-]elharo110
  1. I'm not sure it's actually true that the majority of such scholars would say that Jesus had magical powers. Now admittedly my priors on this are mostly Catholic, and most of those Jesuit, and I don't know a lot of evangelical new testament scholars; but it still appears to me that serious Biblical research is one of the best ways of convincing believing Christians that the religious world is not as they thought. Case in point: Bart Ehrman.

  2. The poll would have to be anonymous to have any validity. There are a lot of priests, reverends, and theology professors who know how much they can and cannot say. Pay very careful attention to subjects an individual scholar does not talk about. Notice how many of them never mention the resurrection or the virgin birth, for example. OTOH they will talk about loaves and fishes and walking on water and Lazarus because they can get away with explaining those parts of the New Testament as understood by modern scholarship without being fired.

5Kawoomba
All the magic events get explained away as symbolical, typical embellishments for their time, allegorical or something other. Except the bodily resurrection. There is no large Christian tradition I know of which does not insist that the bodily resurrection did actually happen. Which is why all the backpaddling on all the other magical events doesn't matter. If you need to explain one ghost, you're in as bad a spot as having to explain all the ghosts.
1MugaSofer
You know, Kawoomba, for once I'm going to actually ask you to provide your source for this sweeping generalization/attack on theists. Seriously. What do you think you know and how do you think you know it?
3Kawoomba
(The comment format breaks when a link isn't properly formatted, corrected now.) Let's go through the list of Christian denominations by members, which would be, from the top: Catholicism - 1.2 billion Protestantism - 600–800 million: mainly Baptist, Lutherian, Methodists Eastern Orthodoxy - 230 million: mainly the Russian Orthodox Church Anglicanism - 85 million Oriental Orthodoxy - 82 million Restorationism - 49 million: e.g. Mormons, Jehova's Witnesses I'll go through them, if any one of them in their doctrine believe that Christ did not actually live and did not get bodily resurrected, I'd concede my "sweeping generalization/attack on theists". What's the largest bet you're willing to make? For full disclosure, "The belief in Jesus' physical resurrection remains the single doctrine most accepted by Christians of all denominational backgrounds." If you want to stand by your charge of "sweeping generalization", let's bet. Edit: Also, nothing special against (personal) theists. I abhor motivated cognition in favor of ludicrously contrived stupidity equally whereever I see it. (I myself think there is a case only for general theism, and only if counting simulationism in general in that category, which would be a non-traditional interpretation of general theism.)
3elharo
Luke's claim was about "everyone at one of the major New Testament Studies conferences", not every self-avowed Christian. I don't doubt that most Christians believe Jesus had magical powers. I'm a lot less certain that even a majority of attendees "at one of the major New Testament Studies conferences" believes Jesus had magical powers. In fact, thinking back on all the New Testament Scholars I've known (more than a few, though not necessarily representative of the entire field) I can't come up with a single name of one I'm confident of saying, "yes, this person believes Jesus had magical powers." I'm sure they're out there, but I'm not sure they even make up a majority of New Testament scholars. I can think of one such scholar who professed a belief in the resurrection but when he was asked to explain what exactly he meant by that, it became clear that he had "faith" that Jesus was there and resurrected even though no one could actually see him post-resurrection. That's pretty flimsy. Serious Bible scholarship (or, for that matter, Church history) is really, really faith-killing. Many scholars went into school committed Christians and came out the other end, well, not. I already mentioned Bart Ehrman. Karen Armstrong is another well-known example. I think they're both still theists, but neither is close to a conventional Christian. I'm not sure either would actually call themselves Christian any more. Recall Bismarck's famous quote that "Laws, like sausages, cease to inspire respect in proportion as we know how they are made." I'd add religion to that list. The most effective way to deconvert a Christian may not be to teach them science. It may well be to teach them more about their own religion.
2Kawoomba
(My claim: There is no large Christian tradition which does not insist that the bodily resurrection did actually happen. I agree with your comment in general, although there are plenty of Schools of Divinity at respected universities whose staff still believes in the usual fairytale. Many atheists know their bible quite well, yet I wouldn't count that as Christians not believing in the bodily resurrection.)
2elharo
Believing in no bodily resurrection was the heresy of Docetism and a common belief of various Gnostic sects. Since these sects were rather effectively stamped out in the first Millennium, it's certainly true today that this isn't a common belief of orthodox, Nicean Creed Christians. Almost by definition if you don't believe in the bodily resurrection, you're not a Christian; or at least not a Christian as defined by the Nicean Creed. Within schools of divinity and theology departments, there are indeed likely many faculty who believe in Jesus's magical powers. My claim is that amidst the subset of those faculty who specialize in the New Testament and the history of the early church, you will find a much smaller percentage who believe in Jesus's magical powers than in the general population of Christians, or even than in the general population of divinity school faculty.
-2MugaSofer
Kawoomba, I'm a Christian, I don't see accusing Christians of Christianity as an attack. (Should I?) I was referring to this: That said, I think I pattern-matched your comment to "Kawoomba attacking theists again", probably because I was primed by "PawnOfFaith" and "Silent M". So I'm sorry for that. Pretty damn hypocritical on my part, too. Unless of course you meant it as an attack, I guess.
1Kawoomba
Aww, no bet then? Apology accepted. I myself hold contradictory, irrational beliefs. I like many of them, even though part of me knows of the contradictions. I also know that if I streamlined my values to be coherent, I wouldn't be myself, and it's not a realistic endeavor anyways, psychologically. I very much doubt that my beliefs are especially contradictory, if a supposed rationalist were telling me he/she held very few contradictory beliefs/aliefs that would be mostly amusing. My problem is taking a clearly irrational belief (and I suspect many smart theists deep down know this) and abusing all of one's wits to put lipstick on that pig. It's such a waste, and such an unnecessary self-delusion ("trying to make it seem rational", not even "believing in it"). That's what gets me going, the waste of potential. I'm not concerned with the Christian version of the Categorical Imperative. Not with Christian ethics, there certainly are worse kinds. Not with having a group identity, so do fans of Vernor Vinge novels (just finished A Fire Upon The Deep). Not even with the First Cause musings, although those often get more into the motivated cognition area. (I also think Krauss is trivially wrong when overstating his "nothing" in "a universe from nothing".) My problem is with the absurd epistemic claims such as "a bodily resurrection took place", "Jesus was tortured to death so I can be saved from my original sin". In a way, it's as bad as Creationism. There isn't all that much difference between saying "the devil planted the dinosaur skeletons", and saying "eyewitness testimony of a few dozen shepherds and partly biased people thousands of years ago, in a book full of allegories and symbolisms suffices to establish that it in fact Jesus was bodily resurrected". Sure, Creationists make many more such claims, but really, does it matter how many risen dead you believe in, as long as the number is greater than 0?
-1MugaSofer
You know, it does. Making less mistakes = rationality. Not much more for me to reply to here.
0Kawoomba
Point. However, I was referring to "using my cognition to defend indefensible claims", as a binary attribute denoting a very, very bad habit.
-1MugaSofer
Ah, right. Still, as you point out, I seriously doubt anyone on this site is that well integrated.
4Richard_Kennaway
Is that based on observation of people at major New Testament Studies conferences? Because as expressed, that gives me the impression of being a prior, rather than a posterior.
2IlyaShpitser
I am sure there exist New Testament scholars who are religious, and I think some who are not. Serious study of the New Testament as a historic text has a way of destroying faith in at least some people. The point though is that experts on 'New Testament studies' aren't experts on Jesus' magical powers, they are experts on the history of the text and the people described in the text. On those topics (those on which papers in journals are actually written), their beliefs are more accurate than yours or mine or a typical Christian faithful's. On Jesus' magical powers they aren't speaking as experts, regardless of what they say. No one can be an expert on that because it's not a scientific topic.
1Alex Vermillion
I am annotating this, even though it is old as hell, because new readers go through stuff like this (or at least my n=1 says so), and I want to highlight something. As I understand the below quote, this is the clearest "No True Scottsman" [1] I have seen on LessWrong. I highlight this so that I can be corrected if I am wrong (as the positive comment score would usually indicate this mistake isn't being made), or so that, if I am right, we can see how easily a conversational slip like this can happen, even when debating "good arguing stuff".
-2MugaSofer
I don't think experts in "New Testament studies" are studying whether Jesus had magical powers. More along the lines of textual analysis.

Man, I didn't even follow the rest of that discussion until now.

I'm going through and systematically upvoting you to bring your accurate, on-point posts back up towards 0. Like, in the thread where we were disagreeing, you were telling me the conventional approach, which is an important thing to know. And now I know where this charge without charge idea is coming from, which is nice! But half of it was at -1. And it's worse with the higher-rated, less comprehensible objections.

3Shmi
Thank you for voicing this :)

I notice that people (including me) are especially prone to this in economics. Everyone feels that they can figure out the correct tax/foreign/monetary/regulatory policy simply by using some trivial demand-supply arguments. Further, they always use a moral axis as well (arguing about fairness/equality/social justice), which always dilutes whatever little insight they might have had.

Worse: economists are highly distrusted as experts. One friend told me that economics as a field has a huge "conservative bias", and you can't trust their methods bec... (read more)

2TitaniumDragon
You assume that economists are actually an expert on the economy. They aren't. That's the problem. Economics only really has a good understanding of very low level effects, and even there things are very difficult to truly deal with. The law of supply and demand, for instance, is really less of a law and more of a guideline - the only way to actually determine real world behavior is experimentation, as there is no single equation you can plug things into to get a result out of. And that's something SIMPLE. Ask them how to fix the economy? They have no ability to do that because they don't actually understand the economy, nor how it works. A great deal of economics is down to ideology rather than actual reality. There are a lot of economic models but they either only represent very narrow areas or they don't actually match up with reality. Ask ten economists what happens when you raise minimum wage and you'll get ten answers - three will agree one way, three will agree the other, and the other four will give you two answers each. The problem is that the economy is such a hideously complex system that economists have no way to actually simulate it or experiment with it (unless you argue what is actually done is an experiment, but if it is, it is a very poorly controlled one with a ridiculous number of variables). Indeed, if you look at reality, it is obvious that economists are NOT experts on the economy. How can you tell this? Because economists make their money by being paid to be economists. The best economist in the world is probably Warren Buffet, because he makes his money by actually making predictions about the economy and then makes money based on how good they are (as well as how well he manages his business). But ask him about tax policy and he'll still likely not have any better answer than you can get via thought and intuition. Empirical evidence suggests that economists are not experts on the economy.
2Jayson_Virissimo
Which is strange, since (according to The Making of an Economist, Redux by David Colander) only 16% of economists are conservative (while 47% are liberal, 24% are moderate, and 6% are radical). Perhaps he meant that the field of economics has less of a non-conservative bias than other fields (like sociology and political science), which I'm pretty sure is true.
2[anonymous]
For yourself? Study economics until you can properly trust or distrust the experts. For other people, ignore or study, but don't engage.
9wedrifid
And, to expand on your point, study economics until you can properly identify the experts. In a field with such strong political implications status is not necessarily (or even often) assigned according to object level competence.
7buybuydandavis
Study the field until you determine whether there are experts. Can anyone successfully predict outcomes?
-2Shmi
First, no area is as bad as sports :) Now, I don't know much about economics, but let us assume for the moment that it is a real natural science and it is impossible to get to the leading edge without investing thousands of hours of hard work, then repeating and alieving the mantra I have suggested in the OP would be my recommendation. One way to convert this belief into alief could be looking through the relevant textbooks and realizing that you cannot possibly understand the advanced stuff without learning the basics first. And real-life economic policy is probably as advanced as it gets. Sort of an economic version of this. Now, it is not necessarily true that economics is like other natural sciences, or like law or medicine. Maybe it is more like alchemy and what the subject matter experts learn over a decade or two does not help at all with giving sound economic advice. Maybe one or two terms is enough, and the rest is just fluff. I don't know. But this point ought to be addressed first, somehow...
4Desrtopa
I'm confused by this categorization of law. I would suggest that economics is not best categorized as a science, since models are rarely tested and then discarded on falsification (real life conditions are rarely good enough at isolating variables to convince proponents that a hypothesis has seriously been falsified,) but that good economists probably do have expertise that an interested amateur couldn't duplicate with a few key insights. On the other hand, there's plenty of room in the field for economists whose learning amounts to indoctrination in models which offer no meaningful predictive advantage over ignorance. I do not claim sufficient expertise to say who falls into what category.
0Shmi
I don't know much about law, except that one needs to know a lot about it to pass the bar exam and to successfully navigate through many legal issues. I share your suspicions about economics, but I have not read any definitive analysis on the matter.
0Desrtopa
Now that I look back at your comment, I see that I misread it in the first place; I missed the "or" before "like law or medicine."

If I come up with $flaw in $argument, how I endorse presenting $flaw depends on my goal.

If my goal is to learn more about $argument, I endorse asking a question which prompts others to provide me the information I want. (E.g., "How does $argument address $flaw?") I also endorse doing this if I think others might benefit from that information, even if I won't.

If my goal is to expose weaknesses in $argument for the benefit of others, I endorse asserting that $flaw is a flaw in $argument. I am of course aware that there exists a social pattern where... (read more)

0Kindly
This may make you less effective at showing off.
3TheOtherDave
Yeah, sometimes that's true. Sometimes it's very false. I have waffled about this a fair bit over the years and concluded somewhat-more-than-tentatively that on balance knowing I'm engaging in status-management makes me more effective at status-management than not knowing. But I don't claim that's universal.
2TimS
If nothing else, being conscious of status-management helps you allocate effort. There are lots of communities in which you or I could become high status, but we don't because we don't care about those communities. Any activity that enhances one's status in those communities is worthless (unless the activity also has other benefits that one does desire).

For any given assertion by an expert on a situation you are not an expert on, the probability that your criticism is correct is not small. However,

1) this does not mean that the expected value of the criticism is negative, even to the expert. If the expert receives 100 comments, 99 of which are confused and one of which blows apart their argument, then they are probably collectively valuable.

2) if the expert is unusually patient, your comment can present her with an opportunity to correct your confusion.

I would say that the important thing is more humility of presentation than humility of willingness to speak at all.

6Shmi
I agree, it's OK to ask what you think could be a stupid question. It's better than not asking, as you lose a chance to learn. It's not OK to insist that you are right and she is wrong once an explanation has been given, even if it does not make sense to you. Though, given the usual inferential distance problems, it's perfectly fine to ask for clarification. From my experience, I think that your estimate of the odds of encountering a comment "which blows apart their argument" as about 1% is overly optimistic. Maybe in some other fields it's different. At best you can expect a minor correction or a qualification. If the expert is any good, they probably have heard it all before, and if they aren't, their ego would likely prevent them from admitting that they are wrong, anyway.
0Oligopsony
That's probably a more accurate way of phrasing things, yeah.

Instead I may ask politely whether my argument is a valid one, and if not, where the flaw lies.

One should always have respect for one's own ignorance and limitations, and yes, it's highly unlikely you'll come up with an objection in a minute that experts haven't already considered, but I don't know why groveling to your betters is required. Why do I have to preface every statement with "I could be wrong, but...."? Can't we take that as assumed, and discuss the facts?

"You're so much wiser and more experienced than I am, but ...." Why... (read more)

7NoSignalNoNoise
Because an unfortunately high percentage of people don't alieve that they could be wrong. Therefore pointing out that you are aware of that fact provides useful information.
4jsteinhardt
It also makes you more likely to alieve that you could be wrong. I.e. the words place you in the right mindset.
0Skeeve
A quick search or two hasn't provided me with a definition of 'alieve', but if multiple people are using it it's probably safe to assume that the word's not a typo. How does it differ from 'believe', which is what I expected to see in that part of the sentence?
6Shmi
This Wikipedia article) is the top google hit for alief and for "alieve vs believe", once you tell it to not helpfully replace your search terms.
2Skeeve
That is a useful word. Thanks for the heads up!

We must test our own ideas and arguments. Just because we don't know how to do so doesn't make our ideas any better, but it can make them seem better to the careless.

It's part of why I don't post very often on this site. Even though I know more QM than most people here, I know I don't know enough to argue the validity of the sequence.

[-]tgb20

Hmm. I'm not sure what to think about this post given that I just made remarkably similar comments on your general relativity/time turner thread in part accusing you of being on the wrong side of this in a way that you don't seem to be acknowledging. I would appreciate your comments on this, shminux, and also to know whether or not you are in a position of expertise regarding GR.

(I made that comment before seeing this thread and would probably have worded things differently if I had.)

8Shmi
Yes, I have spent some 10 years studying the subject and wrote two theses on it. I'll reply to your other comment separately.
2tgb
Thanks.