Googling around phrases like 'perception of intelligence' seems to be a keyword for a relevant literature. On a very cursory skim (i.e. no more than what you see here) it seems to suggest "people can estimate intelligence of strangers better than chance (but with plenty of room for error and bias), even with limited exposure". E.g.:

Perceived Intelligence Is Associated with Measured Intelligence in Men but Not Women (Note in this study the assessment was done purely on looking at a photograph of someone's face)

Accurate Intelligence Assessments in Social Interactions: Mediators and Gender Effects (Abstract starts with: "Research indicates that people can assess a stranger's measured intelligence more accurately than expected by chance, based on minimal information involving appearance and behavior.")

Thin Slices of Behavior as Cues of Personality and Intelligence. (Short 1-2min slices of behaviour in a variety of contexts leads to assessments by strangers that positively correlate with administered test scores for IQ and big 5)

The amount of effort people invest in seeming smarter than they are, for example in situations like hiring or sales, suggests to me that people's gut judgments are easy enough to fool to be worth it.

I expect that what people are actually doing is using a few heuristics to make the judgment. I would also expect that courtesy of things like Dunning-Kruger, people towards the bottom will be as bad at estimating IQ as they are competence at any particular thing.

If we just stick with the intuition provided by fact that some people are really terrible at guessing this sort of thing and some people are not, I further expect that what people use as a heuristic changes with ability level. In example: people at the bottom might base it on whether someone agrees with them about stuff; those in the middle might focus heavily on whether someone has a smart-person job like doctor or lawyer; people towards the top might look more at the way someone does things, like some sign that they chose to think carefully about it or apply some technique known to them.

Faception is a startup which claims to have developed machine learning algorithms that can classify IQ and other things (including "academic researcher", "professional poker player", and "terrorist").

If it is the case that AI can actually do this well, then I'd take that as evidence that humans might be capable of it too.

(From a quick look it's unclear how successful they are. They seemed to be focused on military/security applications, i.e. detecting terrorists.)

" Does anyone know of any similar experiments that have been run? "

I can say with certainty that it has been done on a small scale, because I once saw a German TV documentary for which they had also gotten a small group (6-10) of both males and females and gotten them tested and then shown them the pictures of the others. Later they also gave them some facts, showed videos and asked them again.

The outcome was that there was a clear correlation between the guesses and the IQ-Test results and it got better the more information people had.

But the people were not representative but of above average education. In my own experience it is not the case that people always have a good intuition. I would guess that people are bad as soon as the person judged is much more intelligent than the judge. I think very smart people are much better at recognizing each other.

Research on how humans attribute intelligence to each other suggests that most people attribute higher intelligence to efficient agents, i.e., agents whose actions most resemble what we believe is the best course of action in an uncertain situation. A minority attributes intelligence to agents based on the outcome of their actions.

People attribute intelligence to efficiency in proportion to their own ability to plan. Less competent planners attribute intelligence to outcome. To elaborate, a competent planner would evaluate the behavior of an agent and understand whether it acted rationally, regardless of its success. On the other hand, a bad planner cannot infer the reasoning of an agent; since he himself cannot foresee the best course of action, he would not understand why an agent made a particular decision, and would likely attribute high intelligence to an irrational but lucky agent since its behavior would result in a good outcome.

My interpretation of these results is that intelligent people are better at evaluating the intelligence of others. I believe that given enough time (and thus enough information to evaluate) an intelligent person can adequately estimate a peer's intelligence. It could be argued that "seeming" smart requires a sufficient degree of awareness and intelligence, although of a different kind; perhaps IQ tests do not have a wide enough scope to accurately measure different intelligence in its different forms.

Source:

Kryven, Marta. "Attributed Intelligence." (2018).