Googling around phrases like 'perception of intelligence' seems to be a keyword for a relevant literature. On a very cursory skim (i.e. no more than what you see here) it seems to suggest "people can estimate intelligence of strangers better than chance (but with plenty of room for error and bias), even with limited exposure". E.g.:
Perceived Intelligence Is Associated with Measured Intelligence in Men but Not Women (Note in this study the assessment was done purely on looking at a photograph of someone's face)
Accurate Intelligence Assessments in Social Interactions: Mediators and Gender Effects (Abstract starts with: "Research indicates that people can assess a stranger's measured intelligence more accurately than expected by chance, based on minimal information involving appearance and behavior.")
Thin Slices of Behavior as Cues of Personality and Intelligence. (Short 1-2min slices of behaviour in a variety of contexts leads to assessments by strangers that positively correlate with administered test scores for IQ and big 5)
"Better than chance" doesn't really mean much. The question is how well, quantitatively. Just briefly scanning over your studies:
The amount of effort people invest in seeming smarter than they are, for example in situations like hiring or sales, suggests to me that people's gut judgments are easy enough to fool to be worth it.
I expect that what people are actually doing is using a few heuristics to make the judgment. I would also expect that courtesy of things like Dunning-Kruger, people towards the bottom will be as bad at estimating IQ as they are competence at any particular thing.
If we just stick with the intuition provided by fact that some people are really terrible at guessing this sort of thing and some people are not, I further expect that what people use as a heuristic changes with ability level. In example: people at the bottom might base it on whether someone agrees with them about stuff; those in the middle might focus heavily on whether someone has a smart-person job like doctor or lawyer; people towards the top might look more at the way someone does things, like some sign that they chose to think carefully about it or apply some technique known to them.
I would also expect that courtesy of things like Dunning-Kruger, people towards the bottom will be as bad at estimating IQ as they are competence at any particular thing.
FWIW, the original Dunning-Kruger study did not show the effect that it's become known for. See: https://danluu.com/dunning-kruger/
In particular:
In two of the four cases, there's an obvious positive correlation between perceived skill and actual skill, which is the opposite of the pop-sci conception of Dunning-Kruger.
Faception is a startup which claims to have developed machine learning algorithms that can classify IQ and other things (including "academic researcher", "professional poker player", and "terrorist").
If it is the case that AI can actually do this well, then I'd take that as evidence that humans might be capable of it too.
(From a quick look it's unclear how successful they are. They seemed to be focused on military/security applications, i.e. detecting terrorists.)
" Does anyone know of any similar experiments that have been run? "
I can say with certainty that it has been done on a small scale, because I once saw a German TV documentary for which they had also gotten a small group (6-10) of both males and females and gotten them tested and then shown them the pictures of the others. Later they also gave them some facts, showed videos and asked them again.
The outcome was that there was a clear correlation between the guesses and the IQ-Test results and it got better the more information people had.
But the people were not representative but of above average education. In my own experience it is not the case that people always have a good intuition. I would guess that people are bad as soon as the person judged is much more intelligent than the judge. I think very smart people are much better at recognizing each other.
Research on how humans attribute intelligence to each other suggests that most people attribute higher intelligence to efficient agents, i.e., agents whose actions most resemble what we believe is the best course of action in an uncertain situation. A minority attributes intelligence to agents based on the outcome of their actions.
People attribute intelligence to efficiency in proportion to their own ability to plan. Less competent planners attribute intelligence to outcome. To elaborate, a competent planner would evaluate the behavior of an agent and understand whether it acted rationally, regardless of its success. On the other hand, a bad planner cannot infer the reasoning of an agent; since he himself cannot foresee the best course of action, he would not understand why an agent made a particular decision, and would likely attribute high intelligence to an irrational but lucky agent since its behavior would result in a good outcome.
My interpretation of these results is that intelligent people are better at evaluating the intelligence of others. I believe that given enough time (and thus enough information to evaluate) an intelligent person can adequately estimate a peer's intelligence. It could be argued that "seeming" smart requires a sufficient degree of awareness and intelligence, although of a different kind; perhaps IQ tests do not have a wide enough scope to accurately measure different intelligence in its different forms.
Source:
Kryven, Marta. "Attributed Intelligence." (2018).
Look into the history of EQ and specifically the Mayer Salovey-Caruso work. Truly observed and assessed IQ was random and compounded by the observers own sense of IQ. Primary school teachers were the most accurate at assessing IQ but that was a low correlation and still less than coin toss. The Johari window also comes into play. The more educated you are AND the higher your IQ, the lower you assess your own IQ...i.e. you actually know what you don't know...you are smart but you know you are not the smartest...make sense?
So who is anyone to judge another person's IQ? No body. Test it, don't ask others to judge it.
Peoples' experiences with this are majorly confounded, and your experiment risks also being confounded, by the Conservation of Virtue effect. While it's true that mental attributes, health, height, wealth, and so on all positively correlate, within any filtered social group, those things tend to *negatively* correlate instead, because someone whose composite-trait-score is too low or high for a group will tend to end up in a different one. So, estimating IQ based on a conversation will tend to work well when the person whose IQ you're estimating comes from an unfiltered population, and poorly when it comes from a filtered one.
Wow this is just another instance of talking past each other without even noticing. I'll try to keep this in mind the next time I argue with someone whether IQ and social skills are negatively correlated.
Whatever ability IQ tests and math tests measure, I believe that lacking that ability doesn’t have any effect on one’s ability to make a good social impression or even to “seem smart” in conversation.
That section of Sarah's post jumped out at me too, because it seemed to be the opposite of my experience. In my (limited, subject-to-confirmation-bias) experience, how smart someone seems to me in conversation seems to match pretty well with how they did on standardized tests (or other measures of academic achievement). Obviously not perfectly, but way way better than chance.
Thought something similar, and then I noticed that I met most the people recently through board-game nights, which makes it rather easy to observe these traits first hand. This now makes me wonder to which degree you can deliberately optimize for this by sneakily throwing verbal Puzzles (Did they get the joke?) at people, with or without taking social hit points. My guess would be that it would be hard to Tell apart Motivation (your joke wasn't funny (to them) to begin with).
It's been a few years, so I don't have sources to cite, but I remember looking into this at one point and finding that immune health during developmental years is a common major underlying cause for intelligence, attractiveness, physical fitness, and so forth. This makes a lot of sense from an evolutionary standpoint: infectious disease used to be the #1 killer, so immune health would be the thing which everything else traded off against.
One consequence is that things like e.g. attractiveness and intelligence actually do positively correlate, so peoples' halo-effect estimates actually do work, to some extent.
Are you sure that the correlation between immune health during childhood and all of those positive attributes is causal?
Superficially, it seems more plausible that the causation starts much earlier, via mutational load or specific conditions during the gestation period. Though I suppose that it is also possible that health in the womb and health during early childhood could both effect adult outcomes.
Yeah, sorry, I should have been more clear there. The mechanisms which trade off immune strength against other things are the underlying cause. Testosterone levels in males are a good example - higher testosterone increases attractiveness, physical strength, and spatial-visual reasoning, but it's an immune suppressor.
I'm a little concerned that this could lead to operationalization of the fundamental attribution error. If you're in a conversation with someone who says intelligent things, does it matter if they are an intelligent person?
A record of transcripts might be useful (to make sure people aren't figuring it out based on correlation (ACT/SAT scores, level of education, etc.).
In her latest post, Sarah Constantin writes:
I remember that I made the opposite prediction a few months ago, while I was studying psychometrics more actively. In particular, I had the sense that because of the positive manifold (i.e. almost all positive mental attributes quite strongly correlate), it seems like judging someone's IQ should be relatively easy, and also quite valuable in assessing a large number of other positive qualities, which should make it quite evolutionarily advantageous to be able to assess it.
Two concrete experiments I've thought about:
1. Take a group of people whose IQ you now, then just take pictures of their faces (or short videos of them saying something), and then have a group of participants rate them on their intelligence (probably in order). See how strong they correlate.
2. Throw a bunch of participants into a group, have them talk to each other, then afterwards ask them about the relative judgement of the intelligence of the other members of the group, see how predictive they are.
These seem like very straightforward experiments, that I can imagine someone having already run, and where I would be pretty interested in the results. If only so that I can be better calibrated on how much to trust my gut judgement of someone's intelligence.
Does anyone know of any similar experiments that have been run?