This is our monthly thread for collecting these little gems and pearls of wisdom, rationality-related quotes you've seen recently, or had stored in your quotesfile for ages, and which might be handy to link to in one of our discussions.
- Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
- Do not quote yourself.
- Do not quote comments/posts on LW/OB.
- No more than 5 quotes per person per monthly thread, please.
I was worried by my own conclusion, so I built a mathematical model to check it*:
Suppose that there is an urn with 100 balls. You're 99% sure that there are 99 white balls and 1 black, but there's a 1% chance that there are 99 black balls and 1 white.
You're about to be scored on the probability you assign to the correct state of the urn, using a logarithmic scoring method, but before that happens your friend takes a ball from the urn, looks at it, and puts it back. Your friend then tells you what colour it was.
Your prior that your friend would lie is 10%.
Suppose you are given the chance to check the colour of the ball your friend drew. How much are you willing to pay for this knowledge? Will you pay more or less if your friend said that the ball was black?
By my calculations** the expected utility
if your friend said "white", and you don't check is -0.013581774
if your friend said "white", and you check is -0.0037359
if your friend said "black", and you don't check is -0.391529169
if your friend said "black", and you check is -0.155101993
So your you will pay 0.0098 to check if your friend said "white" but 0.2364 if they said "black".
You try harder to investigate unexpected evidence!
If you're really sure of your conclusion, my maths is probably wrong somewhere. If you think the model itself is inappropriate, please point out how.
EDIT: Of course, if your friend lies 50% of the time, you care just as much about checking confirmatory evidence as disconfirmatory evidence, but then your friend isn't really "offering you a fact".
*And if I wasn't worried, I wouldn't have built a mathematical model!
**Using base 2 logs.
jimrandomh's point might have been that you don't try at all to investigate expected evidence. So you'd pay 0.2364 to check if they said 'black', but if your friend said 'white' you wouldn't pay to check anything, you'd scoff at the suggestion.
Your result is important, though. Equal checking is wrong, lopsided checking is wrong. Only checking exactly as much as is required by the mathematics is okay.