= 27d567bc2d48d9f40e9ccc20ea370b15)
Warrigal comments on Deciding on our rationality focus - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (48)
(If this post is too long, read only the last paragraph.)
Evidence that regards statements. I guess the "regarding statements" bit was redundant. Anyway, let me try to give some examples.
First, let me postulate a guy named Delta. Delta is an extremely rational robot who, given the evidence, always comes up with the best possible conclusion.
Andy the Apathetic is presented with a court case. Before he ever looks at the case, he decides that the probability the defendant is guilty is 50%. In fact, he never looks at the case; he tosses it aside and gives that 50% as his final judgement. Andy is rational-neutral, as he discarded evidence regardless of its direction; his probability is useless, but if I told Delta how Andy works and Andy's final judgement, Delta would agree with it.
Barney the Biased is presented with the same court case. Before he ever looks at the case, he decides that the probability that the defendant is guilty is 50%. Looking through the evidence, he decides to discard everything suggesting that the defendant is innocent; he concludes that the defendant has a 99.99% chance of being guilty and gives that as his final judgement. Barney is not rational-neutral, as he discarded evidence with regard to its direction; his probability is almost useless (but not as useless as Andy's), and if I told Delta how Barney works and Barney's final judgement, Delta might give a probability of only 45%.
Finally, Charlie the Careful is presented with the same court case. Before he ever looks at the case, he decides that the probability that the defendant is guilty is 50%. Looking through the evidence, he takes absolutely everything into account, running the numbers and keeping Bayes' law between his eyes at all times; eventually, after running a complete analysis, he decides that the probability that the defendant is guilty is 23.14159265%. Charlie is rational-neutral, as he discarded evidence regardless of its direction (in fact, he discarded no evidence); if I told Delta how Charliie works and Charlie's final judgement, Delta would agree with it.
So, here's another definition of rational neutrality I came up with by writing this: you are rational-neutral if, given only your source code, it's impossible to come up with a function that takes one of your probability estimates and returns a better probability estimate.
It might be useful to revise this concept to account for computational resources (see AI work on 'limited rationality', e.g. Russell and Wefald's "Do the Right Thing" book).
I'll try my best to get my hands on a copy of that book.
Upon thinking about that second definition of rational neutrality, I find myself thinking that that can't be right. It's identical to calibration. And even a rational-neutral agent that's been "repaired" by applying the best possible probability estimate adjustment function will still return the same ordinal probabilities: Barney the Biased, even after adjustment, will return higher probabilities for statements he is biased toward than statements he is biased against.
I would have said this:
...but that definition doesn't rule out the possibility that an agent would look at your probability estimates, figure out what the problem is, and come up with a better solution on its own. In the extreme case, no agent would be considered rational-neutral unless it had a full knowledge of all mathematical results. That's not what I want; therefore, I stick by my original definition.
It took two read-throughs to get this, but I'm fairly sure that's the concept and not your handling.Thanks for the explanation!