Keith_Coffman comments on Reverse engineering of belief structures - Less Wrong

5 Post author: Stefan_Schubert 26 August 2014 06:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (34)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stefan_Schubert 01 September 2014 11:26:02AM 0 points [-]

There's a whole bunch of information out there - literally more than any one person could/cares to know - and we simply don't have the time (or often the background) to fully understand certain fields and more importantly to evaluate which claims are true and which aren't.

In other words, reality is objective and claims should be evaluated based on their evidence, not the person who proposes them.

It would seem to me that these claims aren't consistent. I agree with the first claim, not with the second. It's true that experts' claims are objectively and directly verifiable, but lots of the time checking that direct evidence is not an optimal use of our time. Instead we're better off deferring to experts (which we actually also do, as you say, on a massive scale).

I wrote a very long post on a related theme - "genetic arguments" - some time ago, by the way.

that bit about the 50% chance of being true is ridiculous even if you don't have any knowledge going into it - you would simply say "I don't know if these claims are true"

Well according to the betting interpretation of degrees of belief, this just means that you would, if rational, be willing to accept bets that are based on the claim in question having a 50 % chance of being true (but not bets based on the claim that it has, say, a 51 % chance of being true). But sure, sometimes it can seem a bit contrived to assign a definite probability to claims you know little about.

I guess the main thing I am trying to say that directly ties into your post is that we shouldn't really care how someone formed their beliefs when evaluating the veracity of a claim; when we should care is:

I don't agree with that. We use others' statements as a source of evidence on a massive scale (i.e. we defer to them. Indeed, experiments show that we do this automatically. But if these statements express beliefs that were produced by unreliable processes - e.g. bias - then that's clearly not a good strategy. Hence we should care very much of whether someone is biased when evaluating the veracity of many claims, for that reason.

Also, as I said, if we find out that someone is biased, then we have little reason to use that person as a source of knowledge.

What I want to stress is the need for cognitive economy. We don't have time to check the direct evidence for different claims lots of the time (as you yourself admit above) and therefore have to use assessments of others' reliability. Knowledge about bias is a vital (but not the only) ingredient in our assessments of reliability, and are hence extremely useful.

Comment author: Keith_Coffman 01 September 2014 04:40:13PM 2 points [-]

I'm making a separate reply for the betting thing, only to try to keep the two conversations clean/simple.

Let's muddle through it: If I have a box containing an unknown (to you) number of gumballs and I claim that there are an odd number of gumballs, you would actually be quite reasonable in assigning a 50% chance to my claim being true.

If I claim that the gumballs in the box are blue, would you say there is a 50% chance of my claim being true?

What if I claimed that I ate pizza last night?

You might have a certain level of confidence in my accuracy and my reliability as a person to not lie to you; and, if someone was taking bets, you would probably bet on how likely I am to tell the truth, rather than assuming there was a 50% chance that I ate pizza last night.

If you you then notice that my friend, who was with me last night, claims that I in fact ate pasta, then you have to weigh their reliability against mine, and more importantly now you have to start looking for reasons that we came to different conclusions about the same dinner. And finally, you have to weigh the effort it takes to vet our claims against how much you really care what I ate last night.

So, assuming you are rational, would you bet 50/50 that I ate pizza? Or would you just say "I don't know" and refuse to bet in the first place?

Comment author: Stefan_Schubert 02 September 2014 07:06:31PM *  1 point [-]

This is a bit of a side-track. For the Bayesian interpretation of probability, it's important to be able to assign a prior probability to any event (since otherwise you can't calculate the posterior probability, given some piece of evidence that makes the event more or less probable). They do this using, e.g. the much contested principle of indifference. Some people object to this, and argue along your lines that it's just silly to ascribe probabilities to events we know nothing about. Indeed, the frequentists define an event's probability as the limit of its relative frequency in a large number of trials. Hence, to them, we can't ascribe a probability to a one-off event at all.

Hence there is a huge discussion on this already and I don't think that it's meaningful for us to address it here. Anyway, you do have a point that one should be a bit cautious ascribing definite probabilities to events we know very little about. An alternative can be to say that the probability is somewhere in the interval from x to y, where x and y are some real numbers betwen 0 and 1.

Comment author: Keith_Coffman 02 September 2014 08:43:38PM 1 point [-]

I agree that it is largely off-topic and don't feel like discussing it further here - I would like to point out that the principle of indifference specifies that your list of possibilities must be mutually exclusive and exhaustive. In practice, when dealing with multifaceted things such as claims about the effects of changing the minimum wage, an exhaustive list of possible outcomes would result in an assignment of an arbitrarily small probability according to the principle of indifference. The end effect is that it's a meaningless assignment and you may as well ignore it.