I was recently re-reading a piece by Yvain/Scott Alexander called Epistemic Learned Helplessness. It's a very insightful post, as is typical for Scott, and I recommend giving it a read if you haven't already. In it he writes:

When I was young I used to read pseudohistory books; Immanuel Velikovsky's Ages in Chaos is a good example of the best this genre has to offer. I read it and it seemed so obviously correct, so perfect, that I could barely bring myself to bother to search out rebuttals.

And then I read the rebuttals, and they were so obviously correct, so devastating, that I couldn't believe I had ever been so dumb as to believe Velikovsky.

And then I read the rebuttals to the rebuttals, and they were so obviously correct that I felt silly for ever doubting.

And so on for several more iterations, until the labyrinth of doubt seemed inescapable.

He goes on to conclude that the skill of taking ideas seriously - often considered one of the most important traits a rationalist can have - is a dangerous one. After all, it's very easy for arguments to sound convincing even when they're not, and if you're too easily swayed by argument you can end up with some very absurd beliefs (like that Venus is a comet, say).

This post really resonated with me. I've had several experiences similar to what Scott describes, of being trapped between two debaters who both had a convincingness that exceeded my ability to discern truth. And my reaction in those situations was similar to his: eventually, after going through the endless chain of rebuttals and counter-rebuttals, changing my mind at each turn, I was forced to throw up my hands and admit that I probably wasn't going to be able to determine the truth of the matter - at least, not without spending a lot more time investigating the different claims than I was willing to. And so in many cases I ended up adopting a sort of semi-principled stance of agnosticism: unless it was a really really important question (in which case I was sort of obligated to do the hard work of investigating the matter to actually figure out the truth), I would just say I don't know when asked for my opinion.

[Non-exhaustive list of areas in which I am currently epistemically helpless: geopolitics (in particular the Israel/Palestine situation), anthropics, nutrition science, population ethics]

All of which is to say: I think Scott is basically right here, in many cases we shouldn't have too strong of an opinion on complicated matters. But when I re-read the piece recently I was struck by the fact that his whole argument could be summed up much more succinctly (albeit much more pithily) as:

"Don't be gullible."

Huh. Sounds a lot more obvious that way.

Now, don't get me wrong: this is still good advice. I think people should endeavour to not be gullible if at all possible. But it makes you wonder: why did Scott feel the need to write a post denouncing gullibility? After all, most people kind of already think being gullible is bad - who exactly is he arguing against here?

Well, recall that he wrote the post in response to the notion that people should believe arguments and take ideas seriously. These sound like good, LW-approved ideas, but note that unless you're already exceptionally smart or exceptionally well-informed, believing arguments and taking ideas seriously is tantamount to...well, to being gullible. In fact, you could probably think of gullibility as a kind of extreme and pathological form of lightness; a willingness to be swept away by the winds of evidence, no matter how strong (or weak) they may be.

There seems to be some tension here. On the one hand we have an intuitive belief that gullibility is bad; that the proper response to any new claim should be skepticism. But on the other hand we also have some epistemic norms here at LW that are - well, maybe they don't endorse being gullible, but they don't exactly not endorse it either. I'd say the LW memeplex is at least mildly friendly towards the notion that one should believe conclusions that come from convincing-sounding arguments, even if they seem absurd. A core tenet of LW is that we change our mind too little, not too much, and we're certainly all in favour of lightness as a virtue.

Anyway, I thought about this tension for a while and came to the conclusion that I had probably just lost sight of my purpose. The goal of (epistemic) rationality isn't to not be gullible or not be skeptical - the goal is to form correct beliefs, full stop. Terms like gullibility and skepticism are useful to the extent that people tend to be systematically overly accepting or dismissive of new arguments - individual beliefs themselves are simply either right or wrong. So, for example, if we do studies and find out that people tend to accept new ideas too easily on average, then we can write posts explaining why we should all be less gullible, and give tips on how to accomplish this. And if on the other hand it turns out that people actually accept far too few new ideas on average, then we can start talking about how we're all much too skeptical and how we can combat that. But in the end, in terms of becoming less wrong, there's no sense in which gullibility would be intrinsically better or worse than skepticism - they're both just words we use to describe deviations from the ideal, which is accepting only true ideas and rejecting only false ones.

This answer basically wrapped the matter up to my satisfaction, and resolved the sense of tension I was feeling. But afterwards I was left with an additional interesting thought: might gullibility be, if not a desirable end point, then an easier starting point on the path to rationality?

That is: no one should aspire to be gullible, obviously. That would be aspiring towards imperfection. But if you were setting out on a journey to become more rational, and you were forced to choose between starting off too gullible or too skeptical, could gullibility be an easier initial condition?

I think it might be. It strikes me that if you start off too gullible you begin with an important skill: you already know how to change your mind. In fact, changing your mind is in some ways your default setting if you're gullible. And considering that like half the freakin sequences were devoted to learning how to actually change your mind, starting off with some practice in that department could be a very good thing.

I consider myself to be...well, maybe not more gullible than average in absolute terms - I don't get sucked into pyramid scams or send money to Nigerian princes or anything like that. But I'm probably more gullible than average for my intelligence level. There's an old discussion post I wrote a few years back that serves as a perfect demonstration of this (I won't link to it out of embarrassment, but I'm sure you could find it if you looked). And again, this isn't a good thing - to the extent that I'm overly gullible, I aspire to become less gullible (Tsuyoku Naritai!). I'm not trying to excuse any of my past behaviour. But when I look back on my still-ongoing journey towards rationality, I can see that my ability to abandon old ideas at the (relative) drop of a hat has been tremendously useful so far, and I do attribute that ability in part to years of practice at...well, at believing things that people told me, and sometimes gullibly believing things that people told me. Call it epistemic deferentiality, or something - the tacit belief that other people know better than you (especially if they're speaking confidently) and that you should listen to them. It's certainly not a character trait you're going to want to keep as a rationalist, and I'm still trying to do what I can to get rid of it - but as a starting point? You could do worse I think.

Now, I don't pretend that the above is anything more than a plausibility argument, and maybe not a strong one at that. For one I'm not sure how well this idea carves reality at its joints - after all, gullibility isn't quite the same thing as lightness, even if they're closely related. For another, if the above were true, you would probably expect LWer's to be more gullible than average. But that doesn't seem quite right - while LW is admirably willing to engage with new ideas, no matter how absurd they might seem, the default attitude towards a new idea on this site is still one of intense skepticism. Post something half-baked on LW and you will be torn to shreds. Which is great, of course, and I wouldn't have it any other way - but it doesn't really sound like the behaviour of a website full of gullible people.

(Of course, on the other hand it could be that LWer's really are more gullible than average, but they're just smart enough to compensate for it)

Anyway, I'm not sure what to make of this idea, but it seemed interesting and worth a discussion post at least. I'm curious to hear what people think: does any of the above ring true to you? How helpful do you think gullibility is, if it is at all? Can you be "light" without being gullible? And for the sake of collecting information: do you consider yourself to be more or less gullible than average for someone of your intelligence level?

New Comment
106 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-][anonymous]160

Post something half-baked on LW and you will be torn to shreds. Which is great, of course, and I wouldn't have it any other way

I would have it, and I don't find it great. Why should baking be an individual effort? Teamwork is better. It should be seen as "here, if you like it, help me bake it". That is why it is Discussion, not Main. I think a good way to use this site setup would be to throw half-baked things into Discussion, if it sounds interesting cooperate on baking it, then when done promote to Main. Really, why don't we do this?

All the great articles in the past, LW 2007-2010 look a lot like individual effort. Why should it be so?

Is this a bit Silicon Valley Culture? Because those guys do the same - they have a software idea and work on it individually or with 1-2 co-founders. Why? Why not start an open source project and invite contributors from Step 1? Why not throw half-made ideas out in the wild and encourage others to work on them to finish them? Assuming you are not after the money but after a solution you yourself would use, of course - "scratch your own itch" is a good idea in open source.

This kind of individual-effort culture sounds a lot like a culture where insights are in abundance but working on them is scarce, so people don't value much insights from others as long as they are not properly worked out. Well, I should say I am pretty much used to the opposite, most folks I know just work routine and hardly any reflection at all...

[-][anonymous]180

I disagree with the premise that LW tears half-baked ideas to shreds. My experience (which, admittedly is limited to open threads) is that you'll be fine if you're clear that what you're presenting a work in progress, and you don't overreach with your ideas.

By overreach, I mean something like this:

This is an attempt to solve happiness. Several factors, such as health, genetics, and social environment, affect happiness. So happiness = healthgeneticssocial environment.

You can see what's wrong with the post above. It's usually not this blatant, but I see this sort of thing too often, and they are invariably ripped to shreds. On the other hand, something like this:

This is an attempt to solve happiness. First, I'd like to identify the factors that affect happiness. I can think of health, genetics, and social environment. Can we break this down further? Am I missing any important factors?

Probably won't be ripped to shreds. It has it's fair share of problems, so I wouldn't expect an enthusiastic response from the community, but it won't be piled upon either.

Frankly speaking, the first type of post reeks of cargo cult science (big equations, formal style (often badly executed), and references that may or may not help the reader). I'm not too unhappy to see those posts being ripped to shreds.

5ahbwramc
I agree with this. "Half-baked" was probably the wrong phrase to use - I didn't mean "idea that's not fully formed or just a work in progress," although in retrospect that's exactly what half-baked would convey. I just meant an idea that's seriously flawed in one way or another.
2ChristianKl
Because ideas are cheap. There an abundance of ideas but not enough people to execute ideas well. Executing ideas well needs focused effort which is easier when you have a company that can pay developers. That doesn't mean that there aren't cases where the open source model makes sense, but quite often it's easier with a different model.
1Lumifer
Ideas are cheap and plentiful. Good ideas are precious and rare.
3ChristianKl
The problem is that you don't know whether an idea is good if you don't try to execute on it. The way you show that an idea is good is to actually execute on it.
1Lumifer
I don't think it's true. Take the reverse case: can you tell that an idea is bad without executing it? Yes, most of the times you can. Obviously, there is uncertainty, but usually you can get a decent estimate of the "quality" of an idea before you start to act on it. There are, of course, nuances and exceptions.
0ChristianKl
I agree that there are idea for which there are obvious reasons that the idea is bad but most of the time there isn't that certainty. Many successful companies such as AirBnB or PInterest had a hard time raising money because investors thought those were bad ideas. On element of a good startup idea is that there's little direct competition. If the idea is obvious there's usually competition.
0[anonymous]
But are you sure in this? I for example have zero even remotely actionable startup ideas right now. By actionable I mean something looking very simple on the outside, such as hipmunk or reddit, is also a huge amount of work. So all the ideas I would have already look complex on the outside, that is impossibly much work probably :) So what I would call actionable startup idea is something that does not look more complex than hipmunk.
0ChristianKl
If you look at Reddit, Reddit wasn't the first idea of the guys. The got to Y Combinator and Paul Graham basically said that their original idea was crap but that Paul Graham really liked the guys so they should still enter Y Combinator. Then the come up with Reddit. The kind of people who have good startup ideas usually can come up with more than one idea.
0maxikov
For one thing, because open source community isn't terribly likely to embark on a random poster's new project, and you'll end up developing it mostly by yourself anyway. Furthermore, there's this aspect of hacker culture, and especially open source culture, where it's actively anti-evangelistic, and dislikes developing user-friendly things like Ubuntu, preferring Slackware or Gentoo.

The aspect of taking ideas seriously that you are talking about seems orthogonal to forming beliefs. It's about initiative in investigating ideas and considering their general applicability, as opposed to stopping at a few superficial observations or failing to notice their relevance in unusual contexts. You don't need to believe an idea to investigate it in detail, the belief may come eventually or not at all. Considering an idea in many contexts may also blur the line with believing it. (Another aspect is taking action based on a belief.)

The process of investigating ideas in detail might get triggered by believing them for no good reason, but there is no need.

[-]Shmi110

A way to deal with this is to learn to notice the situation where you are likely being gullible. To quote the classic poker proverb,

If after ten minutes at the poker table you do not know who the patsy is—you are the patsy.

Maybe you feel out of your depth, maybe a great guru gives a very convincing sermon or writes something controversial but convincing on their blog/forum and wants you to follow/donate/carry the word. Or maybe a car salesperson suggests this great deal only available today. What are the symptoms of being out of your depth and likely to be taken advantage of?

[-][anonymous]50

I've had several experiences similar to what Scott describes, of being trapped between two debaters who both had a convincingness that exceeded my ability to discern truth.

I always feel so.

I see a lot of rational sounding arguments from red-pillers, manosphericals, conservatives, reactionaries, libertarians, the ilk. And then I see the counter-arguments from liberals, feminists, leftists and the ilk that pretty much boil down to the other side just being uncompassionate assholes and desperately rationalizing it with arguments. Well, rationalizing is a ... (read more)

3ChristianKl
The way climate science is done is much more complex than that, and nobody did predict boiling oceans.
-3[anonymous]
I mean, I have read blog posts people acquiring and trying the source code and it was the result they got. Of course such results were not published.
4Lumifer
The source code is of a model. The model has many parameters to tune it (that's an issue, but a separate one) -- you probably can tune it to boil the oceans by 2000, but nothing requires you to be that stupid :-/
4gjm
These people took NASA's GISTEMP code and translated it into Python, cleaning it up and clarifying it as they went. They didn't get boiling oceans. (They did find some minor bugs. These didn't make much difference to the results.) Can you tell us more about the people who said they tried to use climate scientists' code and got predictions of boiling oceans? Is it at all possible that they had some motivation to get bad results out of the code?
127chaos
How do meteorologists predict the weather? By using computer models. Weather is more chaotic and short term than climate so there are obviously differences between the fields, but this should illustrate that you're being a little harsh.
0VoiceOfRa
So one side is giving rational arguments for their position, and the other side is dismissing them with a universal counterargument. Seriously, how is this even a tough call?
4philh
It seems like pretty much the same dynamic would occur with paperclip maximizers. Clippy can argue as rationally and correctly as ve likes that terrible thing will increase the quantity of paperclips made, and the counterargument would be "you're an uncompassionate asshole".
1VoiceOfRa
No, the counter argument would be "we don't care about paperclips". Furthermore in the case of the SJW/NRx debate, most of the "terrible things" in question are things that no one had previously considered terrible until the SJW (and their predecessors) started loudly insisting that these things were terrible(tm) and that the only possible reason anyone would disagree was lack of compassion.
3[anonymous]
Because the discussion is not about a fact of nature but human behavior! And the rules are different there. Basically a smart asshole can make up a ton of excellent rationalizations of why each and every asshole move of his makes sense, but they are still just rationalizations and the real reason of the moves are still his personality (disorders...). When discussing human behavior you cannot really separate facts from values, and thus you need a certain kind of agreement in values. You also cannot separate subject from object, the object being observed and analyzed and the subject doing the studying, the observation, the analysis. Okay there are some partial wins to be made - some aspects of human behavior can be nailed down 100% objectively. But you just can't expect it being a general rule. For this reason, usually it works so that you can discuss it meaningfully with people you are on the same page with, so to speak, i..e. people with broadly similar values to yours and people you consider more or less mentally healthy. For example, the guy who wrote The Misandry Bubble looks like some alien from an alien planet to me. And I am saying it as a guy who hardly had any action until about 30 or so. We are very seriously not on any sort of a similar page, I hardly understand the hidden assumptions and "values" behind the whole thing. I sort of halfway get it that he thinks a man should be some kind of a sex machine and a woman some sort of a vending machine handing it out, but I have no idea even why. The point is, when discussing a law of physics, or, say, climate change, you can set yourself and other people aside and try to look at it from a truly neutral, objective angle. But when discussing human behavior not! The inputs to your computation are basically everything inside you! Because the object to be observed is the human mind, the same thing that does the observation. This is really the issue there - because it is not about strictly defined concepts but a
1Journeyman
I liked your description of certain unconventional schools of thought as "tough-minded" and "creative." Tough-minded, creative thought processes will often involve concepts and metaphors that make people uncomfortable, including the people who think them up. Sometimes, understanding the behavior of large groups of people involves concepts or metaphors that would be unhealthy to apply at the individual level. For instance, you can learn a lot about human behavior by thinking about game theory and the Prisoner's Dilemma. This does not mean that you need to think about other people as "prisoners," or think about your interactions with them as a "game" or as a "dilemma." I think you probably do have a lot of differences in values from people who are “red-pillers, manosphericals, conservatives, reactionaries, libertarians,” but I think this case is really just about inferential distance on the object-level. Although “sexual access” has potential problematic connotations, it actually accurately describes situations where some people’s dating challenges are so great that they are effectively excluded. I apologize for the length this post will be, but I want to drop down to the object-level for a while to give you sufficient evidence to chew on: * Demographics: sex ratio and operational sex ratio have a gigantic influence on society. Exhibit A: China has a surplus of men. Exhibit B: The shortage of black men due to imprisonment turns dating upside-down in the black community and causes black women to compete fiercely for black men. Exhibit C: In virtually all US cities (not just the West Coast), there are more single men than women below age 35 (scroll down for the age breakdown or use the sliders). Young men face a level of competition than young women do not. * If something like 120 men are competing for 100 women, in the system if monogamous, then 20 of those men are going to be excluded from marriage. Yes, in some sense, all 120 have an "opportunity," but we know
3[anonymous]
The issue is that the Prisoner's Dilemma doesn't seem to predict human behavior in modern society well.Partially because it is the kind of tough situation that is uncommon now - this is a bit similar to the SSC's thrive-vs-survive spectrum. All this tough-minded right-wing stuff is essentially survivalist, and every time I am back in Eastern Europe I too switch back to a survivalist mode which is familiar to me, but as usually I am sitting fat and happy in the comfortable West, I am simply not in a survivalist mode nor is anyone else I see. People focus on thriving - and that includes that they are not really in this kind of me-first selfish mood but more interested in satisfying social standards about being empathic and nice. I totally accept the dating market is an uphill battle for most young men - I too was in these shoes, perhaps I would still be if not by sheer luck finding an awesome wife. This is not the issue at all. Rather it is simply what follows from it. This is a good, research-based summary of the opposing view here: http://www.artofmanliness.com/2014/07/07/the-myth-of-the-alpha-male/ This isn't really that. I care very little about being PC except when it is about love. That is, if some kids gaming on Xbox call each other faggots the implied homophobia does not really bother some kind of inner social justice warrior in me, I don't really feel this need to stick to a progressivism-approved list of okay words. But I have this notion that relationships and dating are not simply a brutal dog-eat-dog market competing for meat. There must be something we may call love there, something that goes beyond the merely personal and selfish level, a sense that one would if need be sacrifice for the other. And love is really incompatible with hate or harboring hidden ressentiment or anything even remotely similar, such as objectification. For all I care people may hate whoever they want to, maybe they have good reasons for doing so, but when people seem to hate t
0Journeyman
I think your “mental muscle” analogy is interesting: you are suggesting that exercising mental grievance or ressentiment is unhealthy for relationships, and is part of why men red pill men have an “uphill battle.” You argue that love is incompatible with resentment. You also argue that certain terms “demonstrate” particular unhealthy and resentful mindsets, or lead to “objectification” which is tantamount to not viewing others as people. I share your concern that some red pill men have toxic attitudes towards women which hamper their relationships. I disagree that language like “sexual access” is sufficient to demonstrate resentment of women, and I explained other reasoning behind that language in my previous comment where I discussed operational sex ratio, polygyny, and other impersonal forces. My other argument is that views of relationships operate at different levels of explanation. There are least 3 levels: the macro level of society, the local level of your peers and dating pool, and the dyadic level of your interpersonal relationships. Why can’t someone believe that dating is a brutal, unfair, dog-eat-dog competition at the macro or local level, but once they succeed in getting into a relationship, they fall in love and belief in sacrifice, like you want? It’s also possible to have a grievance towards a group of people, like bankers, but still respect your personal banker as a human being. A metaphor that is useful for understanding the mating market at the societal or local level can be emotionally toxic if you apply it at the dyadic level. If you believe that the current mating market results in some men lacking sexual access at the macro level, that’s a totally correct and neutral description of what happens under a skewed operational sex ratio and polygyny. If you tell your partner “honey, you’ve been denying me sexual access for the past week,” then you’re being an asshole. In the past, men and women of the past held beliefs about gender roles and se
1VoiceOfRa
Of course, any idiot who doesn't like the conclusion of some argument can accuse the person making it of being a smart asshole. I don't see what this has to do with the "smart asshole" problem. A "smart asshole" (or a boxed AI, or the devil) can just as easily create a plausible sounding argument about physics as about human behavior. Is the term somehow ambiguous? Maybe your English isn't that good but it seems pretty self-explanatory. To the extent there is a different culture, it's probably caused by the social situation in Hungary being much less dysfunctional than the social situation in the US.
7[anonymous]
I haven't lived in Eastern Europe for about 10 years now. When I did it felt a lot like a "gangsta" culture, like in GTA: San Andreas esp. in the nightlife / club scene, big buff aggressive guys and stripperish girls with infantile Hello Kitty accessories - does that come accross as functional? I have lived in the UK which is probably the closest to the US culture around here - I must admit I did not like much the music pubs with the fat girls being drunk and cussing and even fighting as if they were male sailors, but as my expertise was in manufacturing software, I lived in a really industrial, read, PROLE area, near Dudley, so that is not really a good sample. It is just prole culture for the most part. Now living in Vienna the only serious social dysfunction I see is everybody being fscking old - it has a retirement home vibe. Demographics screwed up. But what does it have to do with the problem I raised with the word access? The problem I raised is that it is a dehumanizing term that ignores the romantic and loving aspects of relationships, even ignores how sex is a mutual pleasing participating act, it objectifies women as something passive and handing out sex as rewards, basically it has something akin to a prostitution vibe. This is not how a healthy relationship works. Not even how a healthy one night stand - it is based on mutual desire and mutual escalation. It feels incredibly transactional at best and objectifying at worst. But I am not trying to raise a moral finger here. The issue is not that this is morally wrong, the issue is the inferential distance, that there is not one objectively examinable set of human behaviors but the author and me think/talk about entirely differently behaving humans. How the heck to find a rational conclusion in that? There is hardly a shared set of experience because there is hardly a shared value or goal or motive. Yes, but the motives would be entirely different - and yes, they matter. The human mind is apparently too
-3VoiceOfRa
The point is that from what I heard Hungary is a culture where someone whose "interest in women is loving them, being loved by them, and making love, in that order" has a chance of winding up with a woman. What do you mean by "objectifies". I've yet to see a coherent explanation of the concept that doesn't boil down to "applying Baysian (or any) reasoning to humans is evil". Now you're just resembling the semi-marxist/semi-aristocratic "how dare you reduce what I do to something as banal as trade!" Care to explain what you think the two sets of motives are? Rather you have to be running good epistomology rather than anti-epistomology.
3[anonymous]
This IMHO works in every culture, Anglo ones including, you just have to ignore the party b...es and go for the intelligent and non-crazy. Usually it means training yourself to be not too focused on cover-girl looks and be okay with stuff like no makeup. As a theoretical example, consider how would you pick up Megan McArdle - she writes, sounds and looks a lot like my past girlfriends, and Suderman looks and sounds broadly like the same kind of guy I am. This just a hunch, though. However I fully agree that my dating experience in the UK was worse than in Germany, Austria, Hungary, Slovakia or Serbia. (Lived in some places and went to all kinds of meditation camps in the others.) And perhaps it would be worse in the US too. This is largely because I can tolerate things like no make-up, no heels, body hair etc. but I cannot really deal with obesity, and that means playing in a shrinking and increasingly competitive market. Yet, on the whole, my UK experience was not so bad either. On speed dating events in Birmingham, there was a non-fat, intelligent, friendly, considerate 15-20% always. This is that simple basic Kantian thinking that got deeply incorporated into the cultural DNA of the West centuries ago, this why I don't understand what is in not to understand about. It is about primarily treating people as ends and only secondarily and cautiously as means. It is about understanding humans have a faculty of reason and thus autonomy. What follows from this? Autonomy means people can decide to be different from each other, and thus be really cautious with generalizations and stereotypes - perhaps, cultural ones are still okay, because socialization is a powerful thing, but gender is not a culture. Second, and more important, the ends not means stuff means not seeing sex as a prize to be won by an active, driven men and women just passively hand it out as a reward for the effort, but as an mutually initiated, mutually desired interaction between two autonomous being
1VoiceOfRa
Just a hunch but I suspect Megan McArdle would not be doing speed dating. Except the generalizations are frequently correct and have enormous predictive power. Why? Yes, socialization is powerful, but so is genetics, including the difference between XX and XY. In particular the SRY gene has much more influence than a typical gene. You see to be confusing is and ought there. However, you think sex ought to be obtained, being active and driven (among other things) makes a man more likely to get it. Whether, you consider the women's behavior here "passive" or "actively seeking driven men" is irrelevant, and probably doesn't correspond to any actual distinction in reality. So you're saying its not just SJW because it was also used by their leftist predecessors? If you mean that humans are game-theoretic agents, I agree. However, I don't see how "therefore we can't or shouldn't apply probability theory to them" follows. Doesn't this seem to contradict your earlier claim that anti-objectification was responsible for the abolition of slavery? Well, in this case the social theory in question is indeed about a verifiable thing outside the person, namely the dynamics of human romantic interaction. Quote please. I'm guessing you're badly misinterpreting what they wrote. Probably something about how since people respond to incentives, empirically observed behavior will change when the incentives change. Something like a proto-version of Goodhart's law. This is not the same thing as the claim that the laws of probability don't apply to humans, which is the claim you seem to be making. If you mean there is a lot of variance among humans, I agree. However, you seem to be arguing that we should worship and/or ignore this variance rather then studying it.
0gjm
I know what you mean, but I think there is a coherent notion in there, along the following lines: 1. Human beings are people, with hopes and fears and plans and preferences and ideas and so forth. 2. Inevitably, some of our thoughts about, and actions toward, other human beings involve more attention to these features of them than others. 3. Something is "objectification" to the extent that we would change it if we attended more to the specifically person-ish features of the other people involved: their hopes, fears, plans, preferences, ideas, etc. (Or: that a decent person would, or that we should. These framings make the value-ladenness of the notion more explicit. Or, and actually this may be a better version than the other three, that they would prefer you to. The fact that on my account there are these different notions of "objectification" isn't, I think, a weakness; words have ranges of meaning.) So, e.g., consider "treating someone as a sex object", which for present purposes we may take to mean ignoring aspects of them not relevant to sex. If you are currently engaged in having sex with them, this is probably a good thing; on careful consideration of their wants and needs as a person you would probably conclude that when having sex they would prefer you to focus on those aspects of them that are relevant to having sex. On the other hand, if you are in the audience of a seminar they are presenting, you should probably be attending to their ideas about fruit fly genetics or whatever rather than to how they'd look right now with no clothes on; at any rate, that would probably be their preference.
2VoiceOfRa
I *would prefer it" if you sent me a million dollars. By this definition it would seem that you're objectifying me by not sending me the money?
2gjm
Only in so far as the reason why I don't is that I'm not paying attention to the fact that you have preferences. If I'm perfectly well aware of that but don't give you the money because I don't have it, because I think you would waste it, because I would rather spend it on enlarging my house, or because I have promised my gods that I will never give anything to someone who uses the name of their rival, then I may or may not be acting rightly but it's got nothing to do with "objectification" in the sense I described.
0VoiceOfRa
Did you think of the fact that I wanted a million dollars until I told you? OK, if you allow excuses like that, i.e., "I know your preferences and don't care", then I don't see how PUA stuff counts as "objectification".
2gjm
Explicitly? No, but I don't think that's relevant. I'm aware that people generally prefer having more money, and giving someone else $1M would be difficult enough for me that it seems vanishingly unlikely that explicitly generating the thought "X would be better off with an extra $1M" for everyone I interact with would change my behaviour in any useful way. If in the course of talking to you it became apparent that you had a need so extraordinary as to give a near-stranger reason for mortgaging his house and liquidating a big chunk of his retirement savings, then I'm pretty sure I would explicitly generate that thought. (I still might not act on it, of course.) The borderline between objectification and mere selfishness is sometimes fuzzy, no doubt. On reflection, I think "nothing to do with objectification" in my earlier comment was an overstatement; if A treats B just as he would if he were largely ignoring the fact that B has preferences and opinions and skills and hopes and fears and so forth, then that has something to do with objectification, namely the fact that it generates the same behaviours. Let's introduce some ugly terminology: "cobjectification" (c for cognitive) is thinking about someone in a way that neglects their personhood; "bobjectification" (b for behaviour, and also for broad) is treating them in the same sort of way as you would if you were cobjectifying them. I am very far from being an expert on PUA and was not commenting on PUA. But if you are approaching an encounter with someone and the only thing on your mind is what you can do that maximizes the probability that they will have sex with you tonight, that's a clear instance of bobjectification. It's probably easier to do if you cobjectify them too, but I don't know whether doing so is an actual technique adopted by PUA folks. And I guess that when anti-PUA folks say "PUA is objectifying" they are making two separate claims: (1) that PUA behaviour is bobjectifying, which is harmful to th
0VoiceOfRa
Ok, let's apply these terms to the million dollar example. You didn't know or care whether I wanted the money (cobjectification) and once you found out you wouldn't send it to me (bobjectification). So it appears your new terminology applies just as well to the refusing to send money example.
3gjm
Incorrect. I didn't know whether you wanted the money, but not because I was thinking of you as an object without preferences; simply because the question "should I send VoR a million dollars?" never occurs to me. Just as the parallel questions never occur to me in day-to-day interactions with friends, colleagues, family, etc. It's got nothing to do with cobjectification, and everything to do with the fact that for obvious reasons giving someone $1M isn't the kind of thing there's much point in contemplating unless some very obvious and cogent reason has arisen. It is, indeed, true that not sending you $1M is a thing I might do if I didn't think of you as a person with preferences and all the other paraphernalia of personhood. But it's also a thing I might do (indeed, almost certainly would do) if I did think of you as a person. Therefore, it is not a good example of bobjectification. (We could say, in the sort of terms the LW tradition might approve of, that something is bobjectification precisely in so far as it constitutes (Bayesian) evidence of cobjectification. In this case, perhaps Pr(not send $1M | cobjectify) might be 1-10^-9 and Pr(not send $1M | not cobjectify) might be 1-10^-8, or something. So the log of the odds ratio is something like 10^-8: very little bobjectification
-2VoiceOfRa
So you're actual definition of "cobjectification" amounts to "ignoring people's preferences except where there's a gjm!'obvious reason' to ignore them". BTW, I'm not making fun of you. I seriously can't see how this case is different from the case of PUA. Except you weren't thinking of me as a person with preferences. You were thinking of me, if at all, as "just another person I interact with". Note: I'm not saying there is anything wrong with this, but I don't see how it's different from a PUA thinging of a girl as "just another girl I banged" or "just another girl I can't get".
2gjm
Nope. (Nor do I see how what I wrote leads to that conclusion. As an aside, I have this problem quite frequently in discussions with you, and I have the impression that some other people do too. My impression is that you are adopting a sort of opposite of the "principle of charity": when there are two interpretations of what someone else has said, pick whichever is less sensible. Perhaps that's not what's going on, but in any case it doesn't make for constructive discussion.) By "cobjectification" I mean, as I have already said, not thinking of someone else as a person with preferences etc. This is not at all the same thing as thinking of them as a person with preferences etc., but not being at all times consciously aware of all their preferences. If I am talking to someone, then -- as I already said -- the question of whether they would like me to give them $1M generally doesn't cross my mind, perhaps because there'd be no point in its doing so. And also because there are countless different things someone might want me to do, and I am several orders of magnitude short of enough brainpower to think about them all explicitly. Which is to say that not considering whether to send VoR $1M is simply business as usual, it's about equally likely whoever I'm talking to and however I think about them, and none of that applies to thinking about someone only in terms of how I can get them to have sex with me. What makes you think that?
-2VoiceOfRa
So in it's not cobjectification if you abstractly know the person has preferences? Well, the PUA certainly abstractly knows the women has preferences. I don't see how this is different from say only thinking of a Batista in terms of getting coffee.
2gjm
No, the point isn't abstractly knowing, it's how (if at all) those preferences (and other distinctly "personal" features of the person in question) affect your thinking and speaking and action. There's a lot of interaction where the answer is "scarcely at all, for anyone" and such interaction is therefore not a very good measure of objectification. (Though your example is an interesting one; if A and B but coffee from the same barista, and A notices that she looks harassed, takes extra trouble to be polite to her, and maybe remarks "you look rushed off your feet -- has it been a long day?" while B is brusque and rude, that might in fact reflect a difference in the extent to which A and B see her as a person. But this is a very noisy signal.) It's not (in the usage I'm proposing) cobjectification if the way in which you are thinking about the person does not pay markedly less attention to their preferences, personality, hopes, fears, etc., than some baseline expectation. Exactly where that baseline is will change what counts as cobjectification (and hence indirectly what counts as bobjectification) for a given person: objectification is an expectation-dependent notion just like "stupid", "strong", or "beautiful". In the case of PUA, I suppose a reasonable baseline might be "other somewhat-personal face-to-face conversations between two people in a social setting". And if someone claims that PUA commonly involves objectifying women, they mean some combination of (1) would-be pickup artists are attending less to the personhood of their interlocutors than they would to that of other people (especially other men) in other contexts and (2) they behave as if #1 were true. Perhaps an analogy might be helpful. Suppose that instead of "personhood-neglect" we think about "danger-neglect". You might claim that sometimes people fail to recognize others as dangerous when they should, or behave as if they do. An objection exactly parallel to your million-dollar objection to "ob
-2VoiceOfRa
So if we're going by social baseline, that means blacks weren't cobjectified in the ante-bellum south since treating them as property was the baseline. Except by that standard PUA isn't objectifying. Robin Hanson analyzes all kinds of personal interactions in terms of status games and no one calls that objectification unless it involves gender (or race or some other protected category). Except this analogy doesn't work. Most people aren't carrying around TNT, but most people would in fact like a million dollars.
6gjm
No, it means typical antebellum Southerners, if they'd had the word "objectified" and used it roughly as I describe, might well not have considered that black people were being objectified. (Although if you're asking "is group X being objectified by group Y?" then surely the relevant baseline has to involve victims not in group X, or perpetrators not in group Y, or both. So an antebellum Southerner aware that they treated black people differently from white people, or that the dirty race-traitors up north treated black people differently from how they did, might instead say: Yeah, sure, we objectify them, but that's because they're not persons in the full sense, any more than little children or animals are.) I'm not sure which of two arguments you're making. (Maybe neither. My probabilities: 70% #2, 20% #1, 10% something else.) (1) "Robin Hanson does all this dispassionate analysis and no one claims he's objectifying anyone. So dispassionate analysis is OK and what PUAs do is no different." (2) "Robin Hanson's analysis shows that most of us, most of the time, treat people as means rather than ends and ignore their preferences and hopes and fears and personalities and beliefs and so forth. So if PUAs do that too, they're doing nothing different from anyone else." To #1, I say: scientific and economic analysis of people's behaviour is a context in which we expect some aspects of their personhood to get neglected; when we study things we can't attend to everything. And if Robin Hanson analyses behaviour like mine in a particular way, that neither picks my pocket nor breaks my leg; there's no actual personal interaction in which I could be harmed or annoyed or anything. This is all very different from the PUA situation. To #2, I say: Robin Hanson certainly makes a lot of claims about how people think and feel and act that suggest we're less "nice" than we like to think we are. I don't think he's given very good evidence for those claims, and taking a leaf from his b
-2VoiceOfRa
So objectification is a 2-place word now. So why should I care about gjm!objectification? I was asking about individual actions, not groups of people. Yes, I meant (1). The same applies to the book about dating behavior DVH was talking about. And PUA's don't pick anyone's pocket or break anyone's leg either. A closer analogy to PUA would be if someone reads Hanson's (or someone else's) analysis and started applying it in his day-to-day interactions. Do you just automatically write that phrase now without regard to whether it's actually true? It sure seems that way. Well, assuming your rich enough to afford $1M, there is a genuine opportunity for you to help me.
2gjm
Always has been, and I thought I already said so fairly explicitly. (... Yup, I did.) I don't say that you should. The question I thought we were discussing was whether any useful meaning can be attached to "objectification". I say it can; I have described how I would do it; the fact that the word has some subjectivity to it is (so far as I can see) no more damning than the fact that "clever" and "beautiful" and "extravagant" have subjectivity to them. (So can a PUA accused of objectifying women just say: Not according to my notion of objectification? Yeah, in the same way as a sociopath accused of being callous and selfish can say something parallel. That doesn't make it useless for other people with different notions of callousness and selfishness from his to describe his behaviour that way.) But the complaint that I thought formed the context for this whole discussion is that PUA, or some particular version of PUA, is objectifying. That's a group-level claim. (First, just to be clear, I wasn't only referring to literal pocket-picking and leg-breaking but alluding to this. I'm going to assume that was understood, but if not then we may be at cross purposes and I apologize.) I think those who complain that PUA is objectifying would say that its practitioners are picking pockets and breaking legs: that they are manipulating women in ways the women would be very unhappy about if they knew, and (if successful) getting them to do things that they are likely to regret later. If the way they applied it was to try to manipulate me using their understanding of my low-level cognitive processes into doing things that I would not want to do if I considered the matter at my leisure without their ongoing manipulations, and that I would likely regret later -- then I would have a problem with that, and what-I'm-calling-objectification would be part of my analysis of the problem. (The actual primary harm would be getting me to make bad decisions. Objectification is a vice r
4ChristianKl
It's not ambiguous. It's just that it communicates certain values that are foreign to DeVliegendeHollander.
0gjm
And, to be quite clear about it, DVH at no point suggested that he doesn't understand what the term means (despite VoR's respose which seems to presuppose that he did). He understands what it means, he just thinks it implies a strange and unpleasant attitude.
1VoiceOfRa
And yet here he claims that he's "not trying to raise a moral finger here". So is his problem that this "strange and unpleasant attitude", represents a flaw in the argument that would render its conclusions false.
0gjm
Calling something unpleasant is perfectly consistent with "not trying to raise a moral finger". (For the avoidance of doubt, the word "unpleasant" here is mine, not DVH's, but I don't think I've misrepresented his meaning.) I am not entirely convinced that he really isn't trying to raise a moral finger, at least a little bit. I don't think I see how the attitude DVH thinks he perceives via the idea of "sexual access to women" could represent a flaw in any argument, nor is it quite clear to me what argument you have in mind or which conclusions would be being invalidated. Could you be a bit more explicit?
0VoiceOfRa
I have no idea either but if you look up thread, you'll see that DVH seems to think it does.
0gjm
Oh, OK, I'd misunderstood what you were saying. But I don't think I agree; I don't see that DVH is claiming that any argument is invalidated, exactly. I'm not sure to what extent there are even actual arguments under discussion. Isn't he rather saying: look, there's all this stuff that's been written, but its basic premises are so far removed from mine that there's no engaging with it? I expect that, e.g., the book he mentions has some arguments in it, and I expect he does disagree with some of the conclusions because of disagreeing with this premise, by it looks to me as if that's a side-effect rather than the main issue. Imagine reading a lot of material by, let's say, ancient Egyptians, that just take for granted throughout that your primary goal is to please the Egyptian gods. You might disagree with some conclusions because of this. You might agree with some conclusions despite it (e.g., if the goods are held to want a stable and efficiently run state, and you want that too). But disagreement with the conclusions of some arguments wouldn't be your main difficulty, so much as finding that practically every sentence is somehow pointing in a weird direction. I think that's how DVH feels about the stuff he's referring to.
0VoiceOfRa
Except he didn't object to a premise, he objected to the term "sexual access to women". In which case I could point to a specific false premise, namely the existence of the Egyptian gods. Neither you not DVH have pointed to any false premises. You've objected to terms used, but have not claimed that the terms don't point to anything in reality.
1gjm
Here's the most relevant bit of what he actually wrote: "Not about strictly defined concepts". "Your own light which can be utterly different from the light of other people". "For example". "What kind of a life could this come from". The point isn't that there's something uniquely terrible about this particular term, it's that if someone finds it natural to write in such terms then they're looking at the world in a way DVH finds foreign and unpleasant and confusing. Falsity isn't (AIUI) the point. Neither is whether the term in question points to anything in reality. The point is that the whole approach -- values, underlying assumptions, etc. -- is far enough removed from DVH's that he sees no useful way of engaging with it. "When discussing human behavior you cannot really separate facts from values, and thus you need a certain kind of agreement in values." Anyway, I'm getting rather bored of all the gratuitous downvotes so I think I'll stop now. By the way, you've missed a couple of my comments in this discussion. But I expect you'll get around to them soon, and in any case I see you've made up for it by downvoting a bunch of my old comments again.
0lfghjkl
Reversed Stupidity Is Not Intelligence
5[anonymous]
Instinct != stupidity. This is a different thing here. Leaning towards an idea comes both from finding it true and liking it. If you equally lean towards two ideas, but like one more, that suggests you subconsciously find that less true. So if you go for the one you dislike, you probably go for an idea you find subconsciously more true.Leaning towards an idea you dislike suggests you found so much truth in it, subconsciously, that it even overcame the ugh-field that came from disliking it. And that is a remarkably lot of truth. Reversed stupidity is a different thing. That is a lot like "Since there is no such thing as Adam and Eve's original sin, human nature cannot have any factory bugs and must be infinitely perfectible." (Age of Enlightenment philosophy.) That is reversed stupidity. It is a different thing. It is reversed affect.
1lfghjkl
And it could also mean that you just think the evidence for that proposition is better. Your argument looks more like post-hoc reasoning for a preferred conclusion rather than something that is empirically true. I'm sorry, but if you subconsciously like a false idea more often than chance then this quote still applies: You cannot determine the truth of a proposition from whether you like it or not, you have to look at the evidence itself. There are no short-cuts here.
0TheAncientGeek
So what's the right way to predict the future?
1[anonymous]
What exactly do you mean by that? Because the obvious answer is to figure out the causal structure of things, but I don't think that helps here.
0TheAncientGeek
The causal structure is basically a chaotic system, which means that NewtonIan style differential equations aren't much use, and big computerized models are. Ordinary weather forecasting uses big models, and I don't see why climate change, which is essentially very long term forecasting would different.
0Vaniver
Climatological models and meteorological models are very different. If they weren't, then "we can't predict whether it will rain or not ten days from now" (which is mostly true) would be a slam-dunk argument against our ability to predict temperatures ten years from now. One underlying technical issue is that floating point arithmetic is only so precise, and this gives you an upper bound on the amount of precision you can expect from your simulation given the number of steps you run the model for. Thus climatological models have larger cells, larger step times, and so on, so that you can run the model for 50 model-years and still think the result that comes out might be reasonable. (I also don't think it's right to say that Newtonian-style diffeqs aren't much use; the underlying update rules for the cells are diffeqs like that.)

I'm not sure if I'm understanding you correctly, but the reason why climate forecasts and meterological forecasts have different temporal ranges of validity is not that the climate models are coarser, it's that they're asking different questions.

Climate is (roughly speaking) the attractor on which the weather chaotically meanders on short (e.g. weekly) timescales. On much longer (1-100+ years) this attractor itself shifts. Weather forecasts want to determine the future state of the system itself as it evolves chaotically, which is impossible in principle after ~14 days because the system is chaotic. Climate forecasts want to track the slow shifts of the attractor. To do this, they run ensembles with slightly different initial conditions and observe the statistics of the ensemble at some future date, which is taken (via an ergodic assumption) to reflect the attractor at that date. None of the ensemble members are useful as "weather predictions" for 2050 or whatever, but their overall statistics are (it is argued) reliable predictions about the attractor on which the weather will be constrained to move in 2050 (i.e. "the climate in 2050").

It's analogous to th... (read more)

1Vaniver
I did know this, but thanks for spelling it out! One of the troubles with making short comments on this is that it doesn't work, and adding detail can be problematic if you add details in the wrong order. Your description is much better at getting the order of details right than my description has been. I will point out also that my non-expert understanding is that some suspect that the attractor dynamics are themselves chaotic, because it looks like it's determined by a huge number of positive and negative feedback loops whose strength is dependent on the state of the system in possibly non-obvious ways. My impression is that informed people are optimistic or pessimistic about climate change based on whether the feedback loops that they think about are on net positive or negative. (As extremes, consider people who reason by analogy from Venus representing the positive feedback loop view and people who think geoengineering will be sufficient to avoid disaster representing the negative feedback loop view.)
5btrettel
There are a number of different mechanisms which can trigger bifurcations. Finite precision is one of them. Another is that the measurements used to initialize the simulation have much more limited precision and accuracy, and that they do not sample the entire globe (so further approximations must be made to fill in the gaps). There also are numerical errors from the approximations used in converting differential equations to algebraic equations and algebraic errors whenever approximations to the solution of a large linear algebraic system are made. Etc. Any these can trigger bifurcations and make prediction of a certain realization (say, what happens in reality) impossible beyond a certain time. The good news is that none of these models try to solve for a particular realization. Usually they try to solve for the ensemble mean or some other statistic. Basically, let's say you have a collection of nominally equivalent initial conditions for the system*. Let's say you evolve these fields in time, and average the results overall realizations at each time. That's your ensemble average. If you decompose the fields to be solved into an ensemble mean and a fluctuation, you can then apply an averaging operator and get differential equations which are better behaved (in terms of resolution requirements; I assume they are less chaotic as well), but have unclosed terms which require models. This is turbulence modeling. (To be absolutely clear, what I've written is somewhat inaccurate, as from what I understand most climate and weather models use large eddy simulation which is a spatial filtering rather than ensemble averaging. You can ignore this for now.) One could argue that the ensemble mean is more useful in some areas than others. Certainly, if you just want to calculate drag on a wing (a time-averaged quantity), the ensemble mean is great in that it allows you to jump directly to that. But if you want something which varies in time (as climate and weather models do) t
4Lumifer
I don't believe that in reality the precision of floats is a meaningful limit on the accuracy of climate forecasts. I would probably say that people who think so drastically underestimate the amount of uncertainty they have in their simulation.
0TheAncientGeek
Yeah, you can get arbitrary precision libraries.
3Lumifer
You can, and what you'll discover is that they are abysmally slow.
0Vaniver
How much experience do you have with scientific computation? Disagreed. The more uncertainty you incorporate into your model (i.e., tracking distributions over temperatures in cells instead of tracking point estimates of temperatures in cells), the more arithmetic you need to do, and thus the sooner calculation noise raises its ugly head.
4Lumifer
Enough to worry about the precision of floats when inverting certain matrices, for example. We continue to disagree :-) Doing arithmetic is not a problem (if your values are scaled properly and that's an easy thing to do). What you probably mean is that if you run a very large number of cycles feeding the output of the previous into the next, your calculation noise accumulates and starts to cause problems. I would suggest that as your calculation noise accumulates, so do does the uncertainty you have about the starting values (and your model uncertainty accumulates with cycling, too), and by the time you start to care about the precision of floats, all the rest of the accumulated uncertainty makes the output garbage anyway. Things are somewhat different in hard physics where the uncertainty can get very very very small, but climate science is not that.
1Vaniver
To return to my original point, the numerical precision limits due to floating-point arithmetic was an illustrative example that upper bounds the fidelity of climate models. Climate isn't my field (but numerical methods, broadly speaking, is), and so I expect my impressions to often be half-formed and/or out of date. While I've read discussions and papers about the impact of numerical precision on the reproducibility and fidelity of climate models, I don't have those archived anywhere I can find them easily (and even if I did remember where to find them, there would be 'beware the man of one study' concerns). I called it an upper bound specifically to avoid the claim that it's the binding constraint on climate modeling; my impression is that cells are the volume they are because of the computational costs (in both time and energy) involved. So why focus on a constraint that's not material? Because it might be easier to explain or understand, and knowing that there is an upper bound, and that it's low enough that it might be relevant, can be enough to guide action. As an example of that sort of reasoning, I'm thinking here of the various semiconductor people who predicted that CPUs would stop getting faster because of speed of light and chip size concerns--that turned out to not be the constraint that actually killed increasing CPU speed (energy consumption / heat dissipation was), but someone planning around that constraint would have had a much better time than someone who wasn't. (Among other things, it helps you predict that parallel processing will become increasingly critical once speed gains can no longer be attained by doing things serially faster.) I don't agree, but my views may be idiosyncratic. There's a research area called "uncertainty propagation," which deals with the challenge of creating good posterior distributions over model outputs given model inputs. I might have some distribution over the parameters of my model, some distribution over the bo
0Lumifer
Yes, this is technically correct but I struggle to find this meaningful. Any kind of model or even of a calculation which uses real numbers (and therefore floating-point values) is subject to the same upper bounds. Well, of course there is an upper bound. What I contest is that the bound imposed by the floating-point precision is relevant here. I am also not sure what kind of guide do you expect it to be. In reality things are considerably more complicated. First, you assume that you can arbitrarily reduce the input uncertainty by sufficient sampling from the input distribution. The problem is that you don't know the true input distribution. Instead you have an estimate which itself is a model and as such is different from the underlying reality. Repeated sampling from this estimated distribution can get you arbitrarily close to your estimate, but it won't get you arbitrarily close to the underlying true values because you don't know what they are. Second, there are many sources of uncertainty. Let me list some. * The process stability. When you model some process you typically assume that certain characteristcs of it are stable, that is, they do not change over either your fit period or your forecasting period. That is not necessarily true but is a necessary assumption to build a reasonable model. * The sample. Normally you don't have exhaustive data over the lifetime of the process you're trying to model. You have a sample and then you estimate things (like distributions) from the sample that you have. The estimates are, of course, subject to some error. * The model uncertainty. All models are wrong in that they are not a 1:1 match to reality. The goal of modeling is to make the "wrongness" of the model acceptably low, but it will never go away completely. This is actually a biggie when you cycle your model -- the model error accumulates at each iteration. * Black swan events. The fact something didn't occur in the history visible to you is not a guaran
2btrettel
The more I think about this, the less sure I am about how true this is. I was initially thinking that the input and model uncertainties are very large. But I think Vaniver is right and this depends on the particulars of the implementation. The differences between different simulation codes for nominally identical inputs can be surprising. Both are large. (I am thinking in particular about fluid dynamics here, but it's basically the same equations as in weather and climate modeling, so I assume my conclusions carry over as well.) One weird idea that comes from this: You could use an approach like MILES in fluid dynamics where you treat the numerical error as a model, which could reduce uncertainty. This only makes sense in turbulence modeling and would take more time than I have to explain.
0Lumifer
I am not a climatologist, but I have a hard time imagining how the input and model uncertainties in a climate model can be driven down to the magnitudes where floating-point precision starts to matter.
2btrettel
If I'm reading Vaniver correctly (or possibly I'm steelmanning his argument without realizing it), he's using round-off error (as it's called in scientific computing) as an example of one of several numerical errors, e.g., discretization and truncation. There are further subcategories like dispersion and dissipation (the latter is the sort of "model" MILES provides for turbulent dissipation). I don't think round-off error usually is the dominant factor, but the other numerical errors can be, and this might often be the case in fluid flow simulations on more modest hardware. Round-off error can accumulate to dominate the numerical error if you do things wrong. See figure 38.5 for a representative illustration of the total numerical error as a function of time step. If the time step becomes very small, total numerical error actually increases due to build-up of round-off error. As I said, this only happens if you do things wrong, but it can happen.
0Lumifer
Yes, I understand all that, but this isn't the issue. The issue is how much all the assorted calculation errors matter in comparison to the rest of the uncertainty in the model.
2btrettel
I don't think we disagree too much. If I had to pick one, I'd agree with you that the rest of the uncertainty is likely larger in most cases, but I think you substantially underestimate how inaccurate these numerical methods can be. Many commercial computational fluid dynamics codes use quite bad numerical methods along with large grid cells and time steps, so it seems possible to me that those errors can exceed the uncertainties in the other parameters. I can think of one case in particular in my own work where the numerical errors likely exceed the other uncertainties.
4Richard_Kennaway
Even single-precision floating point gives you around 7 decimal digits of accuracy. If (as is the case for both weather and climate modelling) the inputs are not known with anything like that amount of precision, surely input uncertainty will overwhelm calculation noise? Calculation noise enters at every step, of course, but even so, there must be diminishing returns from increased precision.
0Vaniver
See the second half of this cousin comment. But a short summary (with a bit of additional info): First, I see a philosophical difference between input uncertainty and calculation noise; the mathematical tools you need to attack each problem are different. The first can be solved through sampling (or a number of other different ways); the second can be solved with increased precision (or a number of other different ways). Importantly, sampling does not seem to me to be a promising approach to solving the calculation noise problem, because the errors may be systematic instead of random. In chaotic systems, this problem seems especially important. Second, it seems common for both weather models and climate models to use simulation time steps of about 10 minutes. If you want to predict 6 days ahead, that's 864 time steps. If you want to predict 60 years ahead, that's over 3 million time steps.
2btrettel
Many combustion modeling approaches do precisely this. Look into prescribed PDF methods, for example. You can see the necessity of this by recognizing that ignition can occur if the temperature anywhere in a cell is above the ignition temperature. (There is also the slightly confusing issue that these distributions are not the same thing as the distribution of possible realizations.)
127chaos
The differences between climate and meteorological models are reasons that should only increase someone's confidence in the relative capabilities of climate science, so the analogy seems apt despite these differences.
0TheAncientGeek
What's that got to do with causal structure?
0Vaniver
I am not sure what you mean by "causal structure" in this context. I was attempting to provide some intuition as to why ordinary weather forecasting and climate change modeling would be different, since you stated that you didn't see what the essential difference between them is. But it was a short comment, and so many things were only left as implications. For example, the cell update laws (i.e. the differential equations guiding the system) will naturally be different for weather forecasting and climate forecasting because the cells are physically different beasts. You'll model cloud dynamics very differently depending on whether or not clouds are bigger or smaller than a model cell, and it's not necessarily the case that a fine-grained model will be more accurate than a coarse-grained model, for many reasons.
0TheAncientGeek
Understanding causal structure seems to be something that is kind of shiny and impressive sounding, connotationally, but doesn't mean much, at least not much that is new, denotationally. And it comes up because I thought was replying to DVH, who brought it up. I don't think CC modelling and weather forecasting are all that essentially different, or at least not as different as Causal Structure is supposed to be from either. The pattern "the experts in X are actually incompetent fools, because they are not doing Y" is frequent in LessWrong Classic, even if it hasn't been applied to climate change before.
1IlyaShpitser
I think one reasonable complaint is that you should not use predictive models to guide policy because of the usual issues with confounding.
-1TheAncientGeek
Unguided policy is better?
3IlyaShpitser
No? What do you think my position is? ---------------------------------------- Model bias is not a joke. If your model is severely biased, it is giving you garbage. I am not sure in what sense a model that outputs garbage is better than no model at all. The former just gives you a false sense of confidence, because math was used. If you think there are [reasons] where the model isn't completely garbage, or we can put bounds on garbage, or something, then that is a useful conversation to have. If you set up the conversation where it's the garbage model or no science at all, then you are engaged in rhetoric, not science.
1TheAncientGeek
I don't suppose public policy is based on a single model. If you read back, nothing has been said about any specific model, so no such claim needs defending. If you read back, it has been suggested that there is a much better way of doing climate .science than modelling of any kind....but details are lacking.
3IlyaShpitser
If I read back I also read things like this: No, it means a whole lot. You need to get the causal structure right, or at least reasonably close, or your model is garbage for policy. See also: "irrational policy of managing the news." I fight this fight, along with my colleagues, in much simpler settings than weather. And it is still difficult.
3Lumifer
Related. Sample:
1TheAncientGeek
Getting causal structure right in that sense is not an alternative to modelling, it is part of getting modelling right.
-3Lumifer
If you don't want to talk in binary black-or-white terms, perhaps you shouldn't lead with a set-up where a model outputs either truth or garbage ;-)

Michael Smith touched on this in his keynote talk at LWCW last weekend. Don't believe something just because you've heard a good argument for it, he said (I think, reconstructing from memory, and possibly extrapolating as well). If you do that, you'll just change your mind as soon as you encounter a really good argument for the opposite (the process Yvain described). You don't really know something until you've reached the state where the knowledge would grow back if it was deleted from your mind.

Post something half-baked on LW and you will be torn to

... (read more)
5passive_fist
"Don't be gullible" itself has gotchas. If I adjust my skepticism-meter too far upward, then manufactured controversy can seem like real controversy to me -- this fact was used by tobacco companies in the 1960's to sway public opinion and continues to be used today by many other groups. Just because some seemingly-smart people seem to be having a debate about a subject does not mean that the truth is beyond your understanding. I think the better advice would be "There is no easy path to find the truth." That is, there is no formula or template you can always follow to find the truth. Finding the truth requires continuously challenging your beliefs and questioning authority.
0satt
I always liked the more alliterative formulation of that: "no royal road to rationality".
2ChristianKl
I don't think that's the case. If you look at the LW census you find that people think UFAI isn't the biggest Xrisk for most people on LW, even through it's the Xrisk that's most prominently discussed on LW.

Another good post of Scott's comes to mind, where he writes:

Suppose there are two sides to an issue. Be more or less selfish. Post more or less offensive atheist memes. Be more or less willing to blame and criticize yourself.

There are some people who need to hear both sides of the issue. Some people really need to hear the advice “It’s okay to be selfish sometimes!” Other people really need to hear the advice “You are being way too selfish and it’s not okay.”

It’s really hard to target advice at exactly the people who need it. You can’t go around giving e

... (read more)
1Vaniver
Consider parallels to the "openness" personality trait--for the intelligent and conscientious, high openness seems good (they're able to rapidly adapt to changing conditions, in a healthy way because of their intelligence and conscientiousness) but for those lacking those traits, high openness seems bad (they'll fall into pits because they wander without being able to see what to avoid).

This is a great and very precise elaboration of lightness and how to deal with evidence and I think the idea to start gullible is very practical advice - esp. for aspiring rationalists. I do think that it needs some kind of disclaimer, some limitiation or sanity check to avoid going into some affective death spiral around some convincing looking but actually self sealing mind trap.

For me I can very much relate to your exposition. Apparently I'm also kind of advanced gullible. I'm told that I accept new ideas and concepts (new to me) too easily. I'm by now ... (read more)

You might like this: http://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1004&context=eng_faculty_pubs

Someone else posted it to this site originally, I have no recollection who, but we are all indebted to them.

Most people are binary about beliefs. Either they believe X is true or they believe X is false. When talking with LW people you find people saying: "I think X is likely but I don't think it's certain".

If your goal is to get to the right shade of gray, then you need to change your beliefs a lot.

It's likely easier to convince me that P(X)~0.10 instead of P(X)~0.001 while at the same time it's harder to convince me to go from P(X)~0.90 to P(X)~0.999

4DTX
This seems like a decent explanation of why I change my own mind as frequently as I do. If you're just tracking my history of Internet comments, I probably sound all over the place, but it's really me going from 54% certain of position X to 52% certain of not X, and it's hard to properly express that in an environment prone to rhetorical flourish and a debate atmosphere where you feel like you really really can't back down or you'll look weak. Most of the interesting things out there are very hard to legitimately be certain of. Factor in availability bias and it's easy to find yourself arguing for something you're really on the fence about just because you read a good argument for it a few hours ago (but not really any better than the argument for the opposite position a few days ago), then you make a good argument because you're good at arguing, and you just convinced yourself without actually introducing any new evidence. And now I'm trapped in an infinite meta-regress wondering if I actually believe what I just wrote or it just sounds plausible.

yeah being open to ideas is sometimes the only way to go. its okay to take up contrary ideas if you trust in some process of resolution that will happen. at first you wouldn't trust this to happen, and you may want to force the resolution. but even just working on math problems this can be the wrong route. sometimes you need to give it time and have patience. ill take the math problem analogy a little further. just like there can be different takes on issues, you could see different methods to approach a problem. they might both seem promising, but neithe... (read more)

I think there's always been something misleading about the connection between knowledge and belief. In the sense that you're updating a model of the world, yes, "belief" is an ok way of describing what you're updating. But in the sense of "belief" as trust, that's misleading. Whether one trusts one's model or not is irrelevant to its truth or falsity, so any sort of investment one way or another is a side-issue.

IOW, knowledge is not a modification of a psychological state, it's the actual, objective status of an "aperiodic cry... (read more)

The way I've been framing this in my head is that there is a tendency towards having either universally too strong priors or universally too weak priors. It seems almost like strength with which to believe things is a personality trait.

0ChristianKl
I think that you make a mistake if you model the thinking process of the average person with priors. Priors are elements of Bayesian models that are far awah from how normal humans reason.

I'm gullible. Or at least that's what I'm told...