There's probably a lot of low hanging fruit, for example use of correct priors, e.g. given Gaussian prior distribution, a quite strong proof should be needed before you should believe someone (including yourself) has very high intelligence or expertise on a task.
Furthermore, many ways of evaluating people are to some extent self reinforcing as the people being evaluated are aware of evaluation. A smart person or expert can relatively cheaply demonstrate intelligence and/or expertise, in some way that provides very strong evidence, and will do so even for relatively moderate payoffs. Likewise, very selfless people can cheaply (in terms of their utilons) help strangers, and will do so for lower payoffs than people with more selfish utilities.
Other issue is gradual updates on huge amounts of weak, possibly non-independent evidence.
(copied and edited from my post in the Facebook LW group)
I suspect that it would be obvious to most rationalists that the way people judge other people is flawed. Typically for a heuristic approach, it's correct to a degree, but with many faults. And it's wasting a big amount of information and a potential for a more planned approach where you can ask questions that assess certain qualities and exchange information about people's personalities by giving their "parameters".
I needn't think of it in this way, it was natural for me to take this approach as soon as I learnt my first measurable parameter and its implications (it was IQ). Then I explored more of them and researched them some more.
So far, I know about IQ, rationality (Keith Stanovich's), Big Five personality traits, executive functions, intuition for social situations and a few more things. However, I can't seem to find any literature that helps describe them (how do I detect them in people and what are their implications?) and their implications (how *exactly* is someone with a higher IQ different from someone with a lower IQ?). Also, I can't find literature on other traits.
Any literature on any of that would be greatly appreciated. I wonder if there is a book that deals with the whole issue. Also, I need literature about IQ and Big Five, but anything else would still be useful.
Is that sort of thing popular on LessWrong?