I don't understand why and how the authors reached the conclusion that IQ is not that important from this study on Swedish CEOs. They are 1.5 standard deviations above the mean. This is a huge effect.
I got the impression that I wouldn't enjoy being interviewed by Tyler Cowen. If “I believe that if you go to the beach, but you do not give the ocean a chance to taste you, she will come take her taste when she chooses.” is his favorite answer to the crazy belief question, how is he shooting for anything except the ability to spout BS?
And I'm not saying there isn't anything in that answer. Jordan Peterson says "if you see a cat you should pet it". That seems to gesture at exactly the same thing. Do the little nice and relaxing things when the opportunity presents itself as a matter of habit. But of course she can't give that explanation, because then it wouldn't be a "crazy belief" anymore.
I would like to answer that question with something like "I checked the chimp-pig-hybrid theory and I believe it to be correct.". But I can't give that answer because I didn't check it. And if I did check it, I fully expect that to not result in positive evidence. So what facts back her "ocean taste theory"? How is this even an answer to a question about beliefs? Is a belief just "I like the sound of these words"?
I fully expect her to be a lovely and creative and competent person with a poetic bent. But I hate the idea that VCs allocate money (ok, I guess status) based on the ability to not actually answer very hard questions, but to use them as jumping-off point to sell yourself as interesting.
Full disclaimer: I applied for Cowen's AI Venture grant and answered their current "trick question" "which mainstream opinion do you agree with" with "I agree with the mainstream's disdain for trick questions" and I didn't get funding. (Yes, not smart, I know.)
And just as I was writing this I came across another good example of the ‘you think you’re in competition with others like you but mostly you’re simply trying to be good enough’
I'm straight, so possibly unreliable, but I remember Michael Curzi as a very good-looking guy with a deep sexy voice. I believe him when he says other dudes are not competition for him 95% of the time. ;-)
Fair, but my experience says this is true even for Area Man, although Area Man will have a harder time meeting the bar.
Then there’s the actual most courageous things I’ve done, which I am absolutely not going to talk about here and no I wouldn’t tell (most of) you if you asked. I will only say that it worked out. Have fun guessing.
Most of us aren't going to war. Most of the times we're allowed to show courage is by breaking social norms for either the Greater Good^TM or neutral reasons, and involves some kind of subterfuge. This is how I'd expect most people would respond.
The even bigger danger is that at some point you risk crossing from Goodhart’s Law problems to 2nd level Goodhart’s Law problems, where you are effectively optimizing a metric that was originally predicting who would be able to successfully game the original metric, and so on.
This feels maybe worthy of a post.
Maybe. I am torn between 'this does seem like an important thing no one has noticed or at least no one has pointed out explicitly, that I understand and could explain to them' and 'but that's the whole post, the rest is obvious, no?'
I think in the same way that part of the point of books (that could be distilled into one blogpost), is to give you more time to meditate on them and reflect on them, a blogpost that could be one sentence is useful to give the sentence more hooks into the rest of your thinking.
Like, maybe just listing a few different real examples.
Anyone have good real life examples of 2nd Level Goodhart to throw out there? (e.g. where you have T as a measure of U which is supposed to be a measure of V, but you can't measure U directly either, so you end up optimizing for T). Can be either 'it works out mostly OK' or where it works out totally not OK.
State average altitude as a proxy for individual altitude, as a proxy for altitude of water source, as a proxy for water contamination. Jury is out on correctness.
https://slimemoldtimemold.com/2021/07/13/a-chemical-hunger-part-iii-environmental-contaminants/
(I'm broadly on board with contaminant theory of obesity but state is never the right level of data to look at except for laws).
state is never the right level of data to look at except for laws
County-level obesity datasets are mostly based on educated guesses that vary widely rather than actual measurements. I have found several of those datasets that correlate very poorly with one another. Variables such as median household income often correlate more strongly with obesity in some of those datasets than different obesity estimates correlate with each other.
See this Google Colab notebook for a few comparisons.
AFAICT, state-level obesity estimates are way more reliable. The estimates generated with BRFSS data seem to be based on large sample sizes in each state, which is something that we do not have for each individual county. So I think it makes sense to look at obesity at the state level.
That's only true if people within states are more similar to each other on the relevant axes than to people in other states, right? If the real divide is rural/urban or education, then comparing states isn't very useful even if some states are more rural or educated than others.
The fact that the county-level data is bad is unfortunate and makes the county-level analysis less useful, but doesn't fix any of the problems with state-level data.
Interesting writeup, not sure if the book is making any novel claims or is solely a summation of existing knowledge. Can you clarify?
Also, it seems to switch the meaning of the word intelligence back and forth between IQ test measured ‘intelligence’ and g factor general intelligence. Even though it’s well understand IQ tests rapidly lose predictive power in measuring general intelligence more than two or three standard deviations beyond or below the mean/median.
A bunch of it was novel to me, and it represents thinking that is clearly their own, but I am not familiar with the literature in question.
The last bit about US presidents seems especially off. Nixon for example was highly liked and respected among his equals, i.e. other leaders of major countries. Because people at that level can see through all charades, facades, and personas that subordinates put on to try and please superiors, and what political aspirants try to put on to get votes. So Nixon rightly didn’t spend any energy to try and maintain such.
And he has by far the best foreign policy track record of any president since FDR. He actually was smart enough to not just take every suggestion from the State Department or CIA, etc., at face value. But humble enough to realize he may have been totally misled by his own briefers when engaging foreign leaders so he was always open minded. Also smart enough to realize he wasn’t that knowledgeable about domestic politics which he mostly delegated away to others, until he made a mistake.
Which is why the other seemingly novel points from the author may in fact not be. Gell-Mann amnesia and so on.
Tyler Cowen will often praise a book by saying it is interesting on every page. Together with Daniel Gross, he has written a book that very much matches this description.
The topic is talent: How to search for it, how to identify it, interview and otherwise evaluate it, how to recruit it, how to buy it low and sell it high. The book asks, in various ways, what matters? What makes one person succeed or provide lots of value, while another fails and provides little? How can we put together a portfolio of the first type of people at (in all senses) a good price?
There is a unifying philosophy and way of looking at people and the world underlying all of it. I mostly subscribe to the bulk of it, and especially to a core kind of economic thinking.
As part of this way of thinking, the goal is not to interview or evaluate people in an adversarial framework. You don’t want to ‘get’ them by tricking them and making them look foolish. What does that accomplish? You want to ‘get’ them by understanding who they are, and you learn by asking in a situation where they feel they can trust you, about things you both care about.
I strongly agree with this. You’re still going to want to test them and provide challenges so you can see how they react, but there’s essentially never any call for an actually adversarial situation, which only poisons the well and muddies the waters.
Another of Tyler’s principles, in general and that is applied frequently in Talent, is to attempt to crack as many cultural codes and modes of thinking as possible. I again strongly agree. I get great value out of the ones I have and wish I had the ability and time to acquire more. Yet it is also important that these codes are not created equal. The more I explore and model the world, the more desperately I want to spread understanding of the basic economic cultural code, to think in terms of value, supply and demand and compound interest, above any other.
That economic code is the core language of this book, and the book could not have been written without it. The core hypothesis is that you want to look for good deals and good investments with high expected returns. That means finding people who:
Among other things. That’s the biggest macro idea.
In this mode, the whole point of an interview is to truly get to know the person on a deep level. How they think. What they care about. Where they are strong and weak and how smart they are, sure, but it’s more about the quirky stuff that makes us unique.
That of course assumes you can make good use of such information. A lot of the interview questions Talent suggests are the kinds that don’t have right or wrong answers, but if that’s true how do you know which candidate is right or wrong? You need to be able to turn that understanding of the candidate into an analysis into whether or not they would make a good hire, including avoiding equating ‘good hire’ with ‘would then make money and get promoted/funded going forward’ since your goals are different with (although also correlated with) that.
Part of this is that when you discover something ‘bad’ in some sense, that can be good because this will make them underpriced in the market, and something ‘good’ can likewise be bad if it is overly legible and overpriced.
My worry is that most people, myself included, would if they did this well ask a bunch of questions, have an interesting conversation where they get to know someone, then be confused what to do with all of this information. Are these attributes overpriced in various senses? Would they lead to good results in this endeavor? What actually matters?
Tyler and Daniel are true ‘power users’ of talent. Who else would be writing such a book? They both devoted tons of time to evaluating, following, cultivating and promoting talent in various forms. Someone like me has done more of it than most people, but much much less than Tyler or Daniel. So I have enough experience and world modeling to talk extensively about the micro of these questions and explore them, but remain highly uncertain about being able to benefit from the approach.
Having thought and written about a bunch of the micro ideas, I notice I worry that the thing being thought about as ‘talent’ is often ‘this person will be Successful™' and thus becomes talent for navigating moral mazes and selling oneself, or having successful future funding rounds or simply getting higher pay.
If you say you want talent that is idea based instead of status based and that focuses on production, but you measure production by markers of success, what you’re actually optimizing for is the production of the markers of success. Certainly all these are correlated with more generally useful talent, but the more the world is rewarding the wrong things, the more you will reward those same wrong things too. The even bigger danger is that at some point you risk crossing from Goodhart’s Law problems to 2nd level Goodhart’s Law problems, where you are effectively optimizing a metric that was originally predicting who would be able to successfully game the original metric, and so on.
I think about this a lot.
You absolutely want extreme forms of ambition and self-improvement, but in a sufficiently broken and hyper-competitive world this starts to select against caring about anything else, especially if those around you are running forms of meta-Goodhart and punishing caring about anything else. This seems like a big worry for the approach being taken, especially by Daniel.
There’s a lot of emphasis on compounding returns to working on self-improvement, looking for an obsession with improvement and one’s work, and being suspicious of those who spend much time on other things via ‘free time’ and ‘social life.’
I certainly see the argument. I’ve had jobs that were all-consuming, where I was obsessed with putting everything into them that I could, and in many cases I was very good. Whereas the times that I wasn’t doing that, it was a bad sign on multiple levels – I wasn’t putting in the work, and I wasn’t sufficiently motivated to be putting it in.
Tyler Cowen thinks about talent through the lens of Emergent Ventures, while Daniel Gross thinks about talent through the lens of venture capital. They notice.
Thus both are on the lookout for big hits that lead to big wins even if there are a lot of misses. It is important to keep this in mind when thinking about your situation, whether you are searching for talent or you are the talent. Most situations have more limited upside and more serious downside. It is not usually so easy to walk away from or limit the damage from one’s bad talent choices. Also I suspect the bad VC choices matter more than VCs realize due to incentive effects, both globally for all VCs and in terms of selection effects and how candidates pitch themselves and select among possible funders for specific VCs.
As talent seekers and writers of a book called Talent, they of course talk their book.
I had a chance to be CEO, and I did indeed spend a lot of that time obsessed with finding, recruiting, retaining and utilizing talent. It was most certainly not the number one threat to the business, which was lack of money and our inability to raise it (along with our inability to sell our product for a lot of money). I then had a chance at a different company to essentially be the technical cofounder, and yes we have a huge problem finding great engineers, but also once again the number one threat to our business involves some combination of raising money and being able to sell products.
The last few years have made the talent claim more plausible, with a lot more talk (in EA circles, and in my other circles more generally) of being ‘talent constrained’ and the primary problem being lack of people capable of running projects rather than a lack of the necessary funds. Finding people capable of Doing Thing and even more than that of Leading the Doing of Thing is plausibly the limiting factor on many things now.
They also point out that if you are the talent people are searching for, that makes it important to know how other people are searching, so you can help the right ones to find you.
This does not imply that you should be reading Talent for this. That’s a harder question. My hunch is that depends on how ambitious you want to be.
Talent is focused on the best ways to find talent. It has some things to say about how most people looking for it actually find and evaluate talent, but mostly to help you avoid their mistakes and know what things will be undervalued and thus underpriced. There is information there that can be applied when you are the talent, but in practice if one is looking to play it relatively straight because one is facing talent seekers playing it straight, one is better off with a book that plays it more straight.
If you are an exceptional talent looking for an exceptional match, that advice might not apply, but it also might apply anyway. My experience with several (very good) talent evaluators who were seeking extraordinary talent were doing rather different things than are talked about in Talent. In the case where I failed, I still failed for ordinary-interview-failure reasons.
The core message the book has for talented folks trying to be found is: Mostly don’t waste time playing signaling games. Instead cultivate your talent, do real things, be open and be aggressive about seeking out other talent and talent seekers when it would be useful. Which as always implies its own signaling games, and knowing when to set aside that advice for the moment – but it is the right mistake to be making.
That’s the macro-level overview and synthesis as I see it, as opposed to the micro.
Micro and Macro
What is interesting on every page of Talent is the micro, the dive into particular questions and strategies and details. How should we think about a particular personality trait, what does it predict or have to offer? When is this general disadvantage actually an advantage? What are good and bad answers to this job interview question, and how should you follow up tactically to ensure the interview goes places that will inform you?
What excites me about writing this review is the opportunity to dive into those micro questions. Which then allows one to figure out the important macro things they are pointing at, and incorporate them into the first section above.
In the longer sections that follow, I’ll be focusing on the micro. By all means pick and choose the parts of that that seem most interesting to you.
I only managed to get about halfway through my highlights this week, and I’m declaring myself out of time, so I’m going to (at least for now) stop there. If I feel sufficiently motivated, I will do a Part 2.
For all quotes from the book, the number in parenthesis at the end is the quote’s location on the Kindle.
Interview Questions
The key point of the whole interview approach is to get people ‘off script’ which means putting them at ease and asking questions they did not prepare for, ideally that they could not have prepared for. Which is more anti-inductive than the book wants to think, especially if you’re using questions from this book, but that move is likely mostly safe for now.
There’s at least two ways to think about each question. You can ask: What is my answer? What should be my answer versus what is my real answer? And you can also ask: How would I evaluate answers? Do I agree with the frame and thinking behind the question?
Tell Me a Story
Before getting to the strict questions, there’s the concept that asking people to tell stories can be good for eliciting more truth.
My answer:
I woke up around 5:30, then I fed the baby while watching the Mets game, made myself French Toast and caught up on emails, Twitter and a work project until the nanny got here. Then I played Rogue Legacy 2 a bit until my wife Laura was fully awake, after which I started writing, with a pause to take Gideon to preschool.
On the question:
I can see how that gives you a lot to go on, while also being a very typical morning and my choosing not to have embellished or gone into extra detail anywhere. The decision of what level of detail to use is of course part of the question. Still, I can’t help but think it would be a little weird and unfair/off-putting? As if I was preparing for a test and then the test wanted to know how I was studying for it? That’s kind of weird, right?
The coworker thing doesn’t seem right to answer on a blog, but also I notice that when I thought about answering this in an interview I got a very strong ‘I really don’t want to answer this, this is making me stressed and uncomfortable’ response. I simultaneously get the feeling ‘I’d be bragging’ and ‘I’m going to be judged based on my ability to find the best possible brag and they’ll assume this is the best I could do’ and ‘my memory sucks I can’t think of anything oh no’ and also ‘that’s none of your damn business.’ I wonder if that’s common.
The ultimate pure version of all this is the question “How Did You Prepare For This Interview?”
When I hear that last question, my reaction is something like – that’s cheating. That’s not allowed. That’s profane, you’re fighting on holy ground. Which is, of course, a very informative answer in its own way.
My guess is that in many cases, my answer real would be something like ‘aside from at most learning who you are and what you’re about I simply showed up’ and I would then get highly positive selection based on the reaction. There are two ways one can view that answer. One can say that’s a terrible answer, almost the worst possible, you’re not taking this seriously and you don’t work to game systems so you won’t be able to game systems, you’re worthless, go away. Or you can say the opposite, that this is the best possible answer, because the whole point of an interview is to see if there’s a good fit and it’s not a competitive process, so actions designed to ‘prepare’ for an interview distort that process and are wasteful. It’s great to find someone who is focused on other things instead and also you can evaluate the rest of the interview in light of this.
I want my life to draw from that second group of thinkers.
What’s a story one of your references might tell me when I call them? How do you feel you are different from the people at your current company?
These question from the above list stand out because they seems straightforwardly quite bad? If you are preparing and gaming the system via your references you’ll pick your references based on what you want them to say and how they are going to pitch you, so you’ll have a good idea what stories you’d want them to say on your behalf. If you’re preparing for a standard interview you’ll be all over the ‘why are you different’ type of question.
Whereas if the target didn’t do that and played it the way the system wants to work, this is still probably the kind of freeform request for selling oneself and if you resist that temptation then it’s kind of a request to tell a story at all or make a self-pitch at all, whereas a more specific request would go better.
So I wouldn’t ask these.
I also wouldn’t ask:
What are ten words your spouse or partner or friend would use to describe you? (590)
I’m trying to put my finger on exactly what feels wrong here, why it feels like a ‘standard interview question’ and also makes me want to not answer it. Is it that it feels like something that’s about to be graded, where there’s ‘better’ and ‘worse’ responses? That’s obviously not strictly true, you want to present an accurate picture in many senses. There has to be more to the self-reported-impression thing somewhere, as a demand for you to be strategic and choose a signaling strategy, where it feels like there is strong social pressure to not give a true ‘honest’ answer, where you’re being kind of ordered to sell and brag, and all that, and it all again feels like a strategic move in a game in the bad sense.
And the ‘ten words’ thing seems like some sort of trigger to push all that stuff up a notch or three? As if it wants to be certain you notice you’re playing a game and being quizzed.
I have a more mixed attitude towards this next one:
What’s the most courageous thing you’ve done?
My answer:
Among the things I could post online, I had to think for a long time. I’ve done a bunch of stuff that looked courageous. I gambled for a living, I founded multiple companies, stuff like that, but it always seemed like the natural thing to do at the time. Maybe that’s a lot of how courageous people do what they do, they set it up to seem natural.
Then there’s the actual most courageous things I’ve done, which I am absolutely not going to talk about here and no I wouldn’t tell (most of) you if you asked. I will only say that it worked out. Have fun guessing.
On the Question:
There’s a few things going on here, beyond simply ‘this should be a good story.’ First, you’re seeing if they can point to a concrete particular courageous thing, because a lot of people haven’t done any such thing, and therefore can’t, and therefore probably won’t do any such thing in the future. Similarly you’re checking to see what they think it means to be courageous, what they think takes courage. These kind of go opposite ways in some sense, you want someone who will be courageous but you also for any given action worth doing would prefer they not think it required courage – someone who needs to be ‘so brave’ or ‘so courageous’ to get through the day is in that state because something is less than ideal.
Then there’s the ‘what they think would sound impressive’ kind of angle, which gives me some pause. I’d definitely worry this is a question someone would prepare for, and to me it gives off the ‘sell me on you’ vibe for sure.
If I was the perfect Netflix, what type of movies would I recommend for you and why?
My Answer:
Mostly I’m putting this in because I want to say out loud: If you were the perfect Netflix you’d go back to doing what you used to do when you had access to everything ever made and also I had star ratings and predict everything based on whether or not you think I’m going to like it, which you actually used to do a great job doing and if you wanted to you could do so much better now, except also I’d extend that to allow ‘power users’ to basically give the system any and all information they wanted to, and in general wow there’s so much you can do I really want to do this some day but there’s so much other stuff to do. I even think it would be useful for AI Alignment to be able to solve recommendation system alignment in this way. And for now I’ll stop there but yeah I find the whole thing fascinating and someone needs to start the open-source fully-power-usable-and-customizable universal-recommendation-engine project outside of the platforms themselves.
What kind of movies would this recommend to me? I presume a wide variety of actually-good movies, but wouldn’t push me towards particular types because that’s something I would tell it and then it would optimize based on that – I’d essentially ‘tell it the mood’ and then it would offer me the choices.
On the Question:
To me the interesting part of the question is the rabbit hole of ‘what would a perfect Netflix be like?’ so much so that I get fully nerd sniped.
There’s also value in learning what types of things people want to be watching, but I’m not sure what one does with that information exactly?
Practicing Scales
My answer:
It’s changed over the years.
When I was a Magic player I would shuffle cards and take test draws, and of course you play a ton of matches under various conditions. And you analyze decks, strategies, metagames and particular plays in detail, you post-mortem your games.
When I was gambling on sports, my business partner and I would make odds on anything and everything, whether or not it had anything to do with sports. If we weren’t sure if we’d arrive on time, one person would set odds and the other would bet.
As a rationalist, one assigns probabilities to things, which is very similar, but also importantly different. A gambling line is not only about what you believe, it is about what you believe others believe, what they are likely to do, how you will adjust the odds and protect your risk and what degree of uncertainty you have about the right probability and how much your answer changes if someone disagrees with you strongly. A probability on its own contains none of that, it is pure and non-adversarial instead.
As a trader it was trading, as various people on the desk would make markets. This is different from setting odds because others can undercut your market, and also it uses a different set of language and convention. But it’s not a coincidence that all three had very similar kinds of scales.
Now, I haven’t stopped thinking in terms of probability, but that’s not my scales anymore. Instead, I model and optimize every little thing I see, more explicitly and consciously than I used to, and look for surprise. Anything I encounter is a puzzle to be reasoned out and solved for its own sake. What knobs could be turned? What does this provide evidence for and against?
On the question:
I like this question, as the answer will tell you a lot about how someone thinks about self improvement, what skills they want to develop and how they view the core thing that they are doing or attempting to do.
What bothers me somewhat are what I see as the assumptions that give the impression they are embedded in the suggested interpretation a given answer, even if strictly speaking they are not. Indeed, one could say that many of the best answers defy those assumptions explicitly and are all the better for it.
In particular, there is the assumption that you can and should extensively goal factor, self-improving sometimes and working at other times, as opposed to constantly doing both interchangeably in a way very different from piano scales.
There are certainly great benefits to deliberate drilling and to dedicating time explicitly to practice and self-improvement that does not produce any other useful output. I’ve done plenty of it in the past, and still do when preparing for what gaming competitions I still enter.
Yet I consider this to be a second best solution to the problem, the same way school is at best a second best solution to the problem of learning.
The first best solution is, whenever possible, to do things that are useful in their own right but that may have also been chosen with an eye towards giving you an opportunity to learn and improve. As a Magic player, you enter a tournament with an experimental deck and a new sideboard strategy, then extensively analyze the games between rounds. Rather than making markets or setting probabilities on arbitrary things, focus on questions where you want to know the fair price, while also paying attention to your refining your methods. Figure out how to optimize your day, then figure out how to optimize figuring out how to optimize it, and consider not stopping there.
And so on. Instead of going to school, set out to do something real, and learn what you need to learn along the way, with a mindset of deliberate practice and continual self-improvement in all things – there are sufficiently urgent situations where one would suspend that until after things are over, but they’re rare.
That’s my real answer. The problem with the question is that it doesn’t seem likely to elicit that answer, and also I worry it is looking for the ‘wrong’ answer in some sense.
The exception is when one is engaging in a finite game of absolute competition, where you can absolutely describe exactly what matters. That happens to describe sports perfectly.
Magic: The Gathering wasn’t quite as good a fit, but was a much better match to ‘you had to be obsessional about explicit improvement and doing the work to win’ than when I was doing other things.
As for the part about not pursuing typical social pursuits, I do think you want to see at least some of this.
Another question raised here is: If you do see someone who doesn’t have an obvious drive or focus on self-improvement, does that mean they won’t improve, or won’t improve that much? Does what you see therefore equal (in the best case) what you get?
The chances are obviously lower, but I think no because it is very easy to miss kind of improvement that comes from treating work as at least something of a deliberate practice or otherwise learning by doing. In some sense, as long as someone is striving to do a good job and is given tasks that are challenging it is hard not to learn.
I think this is one reason that good interviews involve doing some amount of work or problem solving. That gives you a chance to see whether someone uses a challenge as an opportunity to learn.
What Are Your Open Browser Tabs Right Now?
My answer:
One good part about doing this at home where you can actually look at the tabs is it gives you incentive to close some of them and reminds you of others. Having 22 open tabs is not so many these days, but I still consider it ‘too many’ and think it would be better to get that down to 5-10.
On the question:
As long as no one sees the question coming, I love the question because everyone is unique when it comes to what they keep open unless they are running a very tight ship. As the book suggests, you learn a lot about a person, how they think and organize, what they are interested in.
The natural question then is: What are ‘good’ answers versus ‘bad’ answers? In some sense this is of course a wrong question. You’re not looking to classify people as good/bad or better/worse, you’re looking to figure them out, so in that sense there aren’t wrong answers. In another sense of course there are wrong answers, and of course there are some things you’re pretty much always looking for.
In that second sense, as Tyler said in an interview with Megyn Kelly, what he’s looking for more than anything is a sense that the person is attempting to solve problems and figure things out. Megyn thought her tabs were bad, and Tyler reassured her that he fully approved because she had specific real world problems and was attempting to solve them.
He also looks for signs the person is working on their skills in their spare time.
It makes sense that working on whatever skills are relevant to anything is still an excellent sign, where the thing you want to most measure is ‘do you work on real skills in general’ rather than exactly which skills are being worked upon.
It is the absence of these things that Tyler thinks is a poor answer. In particular, he thinks not having open browser tabs at all is a bad answer.
Which to me is odd. Open browser tabs tell you a lot, but one thing they tell you is a person is using open browser tabs as bookmakers/placeholders of a sort. Is that actually a good idea? I read an article on Bloomberg recently about the Arc browser that intentionally tries to prevent open tabs, among other things auto-closing most tabs after 12 hours into an archive. The founder argues that Chrome and Safari are stagnant for fear of alienating existing users but that dozens of open tabs are an actively bad sign that something is very wrong, and instead they want to build a new one. They’re still in ‘join the waitlist’ mode so I won’t be trying it out yet but if they randomly sent me a copy after seeing this I’d give it a whirl.
I spend a lot of time organizing stuff, web-based and otherwise. I still end up using my inbox and open browser tabs as de facto to-do lists of a sort reasonably often, and it works but at least I have the decency to feel bad about it and like I should be doing something better, you know?
What counts as a good or bad answer, in such cases, I think a lot comes down in large part (far from all, but a large part) to what the candidate thinks is a good or bad answer. What parts of this are they proud of and why, versus which parts do they think are embarrassing or shouldn’t have been there? If they knew there was an interview with this question today, what tabs would they have chosen to open or close?
What Subreddits or Blogs Do You Read?
This is similar to the tabs question, getting at where someone’s true interests lie.
My Answer:
I don’t read subreddits, as I don’t find the format useful – it’s either Twitter where I follow a variety of lists and people, or it’s blogs and podcasts.
For blogs, I took this opportunity to update my list of links on my Substack main page. For podcasts, I want to mostly save that for another time, but the big two I make sure to listen to are Odd Lots and Conversations with Tyler.
I remember ‘online communities’ but that all seems so old school now. One might be useful though. I’m considering starting a Discord server.
On the Question:
Together with the browser tabs I have no doubt these questions are highly useful in understanding someone or finding things to talk about, and mostly it seems good, but I also get a sense that there’s something wrong with all this? It’s an invasion of privacy, crossing the streams, threatening to mess people up or something?
Seems worth unpacking a bit. I know I’d find it completely unacceptable if someone were to actually invade my privacy and extract this kind of information without my consent – and that if I thought that was likely I’d probably change some of my choices in ways that would make my life and the world worse.
There’s a sense that some things are ‘fair game’ while other things are private, in any given context. Some things are subject to signaling games and are done in light of others’ potential Bayesian updates, while others need a safe space where that isn’t true. Open browser tabs seem like they are somewhat muddled, since you are indeed ‘leaving them open’ to some extent, and the (ideally) temporary nature of them prevents other issues from getting too severe, but then in other ways things get murkier.
When I’m choosing which things to read or do ‘on my own time’ it is all for the good if I am thinking about how this helps me be better at my job. I worry more if I’m thinking about whether someone knowing what I am up to in my spare time, or what I am reading, even if I am not worried about cancel-style guilt by association.
One acts so differently when others are looking or might be looking, bringing in a whole new set of considerations. It opens up new realms to more intense signaling games, And That’s Terrible. In some sense you want people to be in a totalizing game of self-improvement and accomplishment, with its increasing returns to investment, even if it eats into other pursuits, and you want to measure that. What you don’t want is even a hint of Goodhart’s Law along similar lines, which will rapidly eat away at absolutely everything and destroy people’s lives as humans along with the actual investments that generate the increasing returns. If one has to worry about looking like you’re investing, there aren’t resources left to actually invest.
The obvious response to this is that so long as such questions are a surprise, so long as they are rare, that all is fine. You will do what you would have done anyway. It takes a reasonably large probability of being asked about your browser tabs or blogroll – in ways you can choose to dodge or fudge if necessary – before your behaviors start getting seriously bent out of shape.
This then gets back to the question of underused approaches, finding what is overlooked and underpriced. Talent is full of strategies whose entire approach is built upon the assumption that you are doing something completely different, because almost everyone else is doing something completely different. When people are doing completely the same things instead, as they usually are, they have to be bounded within a finite game where they can be optimized for the inevitable adversarial dances.
What’s Wrong with Pioneer’s Tournament System?
Daniel wants to know.
Ask for actual useful information? Don’t mind if I do. Alas, the book doesn’t talk about the tournaments so it’s hard to offer useful feedback. I do like the question both as valuable for its own sake and also as a check to see if someone is willing and able to offer specific criticism.
What Is Your Most Absurd Belief? (Or: Questions Whose Utility Have Expired)
I would note that I would give different answers to the different forms of the question, and most others likely should too – the whole idea of ‘thing I think is true that people around me think is crazy’ is largely about believing that the people around you being crazy in a particular way, so I think it would be right to answer with something that most other people find relatively normal, rather than my most absurd belief (and I’m going to keep you guessing on that for now, sorry).
Tyler says once word got out, the canned answers were no good.
I think this underestimates the dynamics, and assumes the answer to the question and that there is a ‘right’ response and the right answer is to seek the right response. In some cases the right answer is the wrong response, because it shows you weren’t intentionally choosing the right response. If you were a true contrarian, would you alter your level of contrarianism to your expectations of the expectations of the interviewer? I say no, you would not.
Also, one could say that the good answers are things you hadn’t previously even considered, so even if you don’t get such great info on the person you’re still getting an interesting new belief, or if you don’t get one you’re getting info that they don’t have one to offer, of any level of contrarianism.
“Which of your beliefs are you least rational about?” “What views do you hold almost irrationally?” (731) and “Which of your beliefs are you most likely wrong about?” (742)
My Answer:
I mean, if I knew that I wouldn’t believe them. Or would I? Hard to say. In the context of founding a company, the real answer would of course be ‘that this company will succeed’ because it is well known that this belief is usually wrong. Or perhaps it’s something related to rationality by way of AGI or immortality or some such. Or in another sense maybe it’s simple stuff, that I think my family is the absolute best and the Mets are great and so on? That could be it too.
On the Question:
In some sense I see this not as checking to what extent the person is keeping multiple sets of books, and where the person sees this as useful, along with the question of whether the person thinks one should strive to hold accurate (rational) beliefs at all. I find keeping some sense of multiple sets of books more useful than many others I know, but far less (yet far more intentionally and self-aware about it) than most others.
My gut expectation is that this is mostly going to elicit BS, but also tell you what the candidate thinks is the good kind of BS to go around with, so I suppose that is at least somewhat useful.
I also don’t remember the book mentioning it, but often the obviously correct response will be a religious belief, or religious belief in general, to which I assume replying ‘aside from that sort of thing’ is the next move, but they score points for self-awareness.
How Do You Think Is This Interview Going?
How should you respond to this? It’s complicated.
As the interviewee, you want to maximize the chance you get hired for jobs you want to get hired for, and ideally to avoid jobs you don’t want to get hired for, and you don’t know what you will get punished or rewarded for saying. The intention may have been to see if you’re honest and self-aware, and to see what you think they’re thinking about, but that doesn’t mean highlighting your potential weaknesses won’t reinforce and draw focus to them. And you often have little or no idea how things are actually going, because you don’t know what they are looking for. If you are doing your job properly as an interviewee, of course, there will be a correlation between your performance and the prize – the more you should want the job, the better you’ll be doing.
If the prize isn’t worth having, then it doesn’t much matter what you answer, and in fact you want to get turned down (although it does feel good to get a yes anyway, so maybe not if you know not to accept the offer, and also you might want to use the offer as leverage). So you should presumably start by assuming the job is indeed good and you want it, which means the company wants someone like you for the role, which means you’re probably doing pretty well. And given that most interviews end without an offer, if you’re not doing pretty well then it likely matters little what you say, so you might as well be relatively optimistic.
Stonewalling on the negative could be thought of as the risk-averse choice, but that isn’t obvious. Usually there will be something that obviously didn’t go great, and also you can use this as an opportunity to show high self-expectations. There are always things you could have done better. If you don’t see anything that went wrong, that means they should evaluate you as if you’re doing about as well as you can do. That’s probably not an impression you want to give. Whereas if you hold yourself to a higher standard, and frame your performance as below your expectations for yourself, that seems like a positive.
It also seems to me like expressing uncertainty is almost always good, unless you want to get hired for a position where they punish uncertainty, which seems terrible?
I’d also expect ‘always only be in adversarial mode and only say positive things’ to be a mode that’s on or off across the interview, and this is a place where they are checking to see which state that switch is in.
So I’d say even more than it being a ‘risk-averse’ answer to stonewall, stonewalling tells me you think that stonewalling is risk-averse, which tells me what you think I am looking for, and also that you’re willing to be somewhat dishonest to avoid risk, and yes I think it seems reasonable to hold all that against you.
I wouldn’t plan on asking this, but there are times when it should seem like the logical next question. In both of the great interviews I’ve done, that question would have been insightful at the right time.
How Ambitious Are You?
One common theme is that good questions to ask are questions where honest answers are ideally rewarded, or at least seem like the thing to do at the time. One way to do that is to ask a question where the process that chooses the true answer is the same process, or similar to, the process that chooses what you think is a good thing to say in an interview. Thus, if asked how ambitious you are, you’ll probably think that the true answer is the ‘best’ answer. The person who said ‘I want to get tenure’ probably thinks the proper goal of life is tenure, and that anything more would sound grandiose. If someone is telling you about their grand ambition, it does seem rather unlikely that they are lying about having one.
If At First
I notice I am confused about this idea. Sometimes it is absolutely the right thing. Sometimes you need to know. Sometimes repeating the question until they ‘get it right’ is all they need to work through it. Perhaps their refusal to answer tells you the answer is all the more important.
Yet sometimes them avoiding the question is answer enough? Or alternatively, if you wish to get and keep them relaxed and in a non-hostile interaction, perhaps you should accept their refusal to answer.
So I do endorse considering the option to ask again and refusing to accept a non-answer, but I think the majority of the time it’s right not to do that. Them avoiding the question is some combination of answer enough and a signal you asked the wrong question.
Then there’s the situation where you get an answer, but you keep asking anyway.
To what extent is this a test of memory and the keeping of records of achievement, especially specific stories one can use in such spots, versus a test of how often one actually goes above and beyond the call of duty?
It’s hard to avoid the ‘what would happen if someone did this to you personally’ lens when thinking about interview techniques, so my lousy memory for such things is focusing my worries there.
One could think of this as, you ask once you are asking for basic prep. When you ask more than once, what kind of prep is being tested? You are hoping to blow past anyone’s ability to prep answers, and you will likely do so for interview purposes but what about those who track their achievements for other purposes, which seems like a reasonable reason to stack up tangible examples? If someone comes up with answers two through ten ‘too quickly’ in some sense, or they are too organized or too polished, what does that tell you? If they have to think a bit, is that a good or bad sign?
I also believe some people think of ‘above and beyond’ as what others think of as ‘the basics of the job.’ Thus, there is a job I had where I could claim I went above and beyond every day, or I could kind of say I never did because that’s simply the job. When you are a CEO of a startup, is it even possible in theory to go above and beyond? The job is supposed to be all-encompassing. That’s about this specific example question, but a similar version likely applies to many others.
There is this angle as well. You’re interested in how they respond to the challenge as much or more than you are interested in the content, although you might (or might not) also care about the content for its own sake. To some extent half the book’s suggestions seem to boil down to ‘the point of an interview is to figure out who you’re dealing with so get them reacting to things and see what happens.’
Change of Venue
The book suggests taking candidates to different locations.
This certainly seems like ‘part of a balanced interview process.’ If I was doing one interview and had an hour or two, I usually wouldn’t want to do this, I’d be ‘spending my time’ inefficiently in terms of information gathering. Yes, you get to see (as they mention elsewhere) how this person interacts with the world in various ways, how they treat waiters and so on, but is that the most important thing to learn? How much of your time will be ‘wasted’?
If you have a whole day or more to evaluate someone, or at least the majority of one, then that is different. Everyone has to eat and you get more benefit from taking a break of sorts and changing the overall tone and setting. So I like this for later stages of high-level interviews, but I’d only do it once someone had my full attention.
The Premortem
Everyone remembers ‘what is your greatest weakness?’ This is the better variation of that, asking what might make things not work out.
I’m more excited by the few months timeline than the few years. For a few years, the question feels like a loyalty or endurance test of some kind. It feels like it’s telling me to come up with a false story. Whereas if the timeline is only a few months, clearly it didn’t work out so the question is why things could fail that quickly, and you can give a more honest answer that gives insight into what the person cares about and worries about. Might they prove not up to the task? Uninterested? A bad fit? What would get them to pull the plug so quickly?
What Did You Like to Do as a Child?
My answer:
I played a lot of games. Card games, board games, computer games.
But also…
On the question:
Before the world started bossing them around? Wait, WHAT?
Children are bossed around the most of anyone, by far.
The bulk of the day you are told exactly where to physically be, where you must do exactly what you are told, otherwise keeping still and saying nothing. If you do not spend a lot of the rest of your time doing the arbitrary work they assign you, they punish you further as will your parents. The rest of the time, your parents largely tell you what you can and must do. Where you can go isn’t up to. Who your friends are often isn’t up to you. What you eat isn’t up to you. Where you sleep isn’t up to you. What television you can watch or games you can play often isn’t up to you.
Meanwhile, you are told that everything you do bears upon whether you get into a good next school, which will determine your entire future – you are constantly threatened that failing at one of thousands of things will ruin your entire life. And then that you’ll be judged based on arbitrary hidden and random criteria, so you should fill your life with things that ‘look good on an application.’
I already went on too long, but basically I cannot fathom someone thinking children aren’t being bossed around.
I mean, I get the intention behind the statement. Kids, one hopes at least, are still spending some amount of time ‘being kids’ although I increasingly worry that they are increasingly not being allowed to do this. And what people look back upon fondly, whether or not they had any say in its selection, gives you insight into the person, sure.
What is an example of a mistake you have committed but did not come to regret for a long time? (1,079)
My Answer:
Oh man I have at least two really good answers to this one.
The first is that when I founded MetaMed, I was running a startup, but I got confused and thought I was running a business. Thus, I tried to make the company be profitable and work as a business, rather than realizing that such concerns were irrelevant and I should be focused on creating proper hockey stick graphs that would enable additional fundraising rounds. By the time we realized that wasn’t going to work, it was too late.
The other, which predates that, is that the owners of a company told me that I was the future of the business and that if I kept going around making the place profitable and working my ass off they would transition ownership and leadership of the company over time, and I could do this remotely.
And I believed them.
Whoops. Yeah, no. That’s not how the world works. That’s on me.
I mean, it worked out fine, and I don’t even know I would have really wanted to move onsite so I could have had a shot of actually taking over – certainly I’m happy things turned out the way they did overall – but oh what might have been.
On the Question:
It’s a weird one, because the null response – that you have made mistakes and learned from them but don’t regret a thing – seems highly valid. As does not having been at a job for a long time to have such a mistake to regret.
If someone does answer, there’s the obvious ‘maybe they tell me something awful’ aspect but mostly it’s asking what long term mistakes are most salient to people, what do they notice and learn from and consider important.
Do You Deserve To Succeed?
Variations of this that bring moral desert into it get weird.
Peter Thiel, the book reports, asks whether someone morally deserves to succeed, and uses that as part his funding decision. This matches my other experiences and knowledge of Thiel. I know of examples where he knew that someone or something was failing, could have offered that information to make failure less likely, but he did not do that, and my model of why is something like ‘if they can’t figure it out on their own they deserve to fail’ and it would be somehow wrong to stop that.
I don’t agree, but I do understand how one can get there. I have the instinct, at least a little. Part of this is that if you ‘rescue’ the situation, perhaps that means you’re going to have failure down the line instead when it’s more expensive? And part of it really is some sort of feeling of justice.
Questions like this seem kind of bizarre to me:
What does deliberate sin mean? It means you’re using a moralistic model of the world. There’s certainly a sense in which some things are accidents and some things are deliberate, although not as much as one would think – much of the time, if an ‘accident’ happened it was kind of accidentally on purpose or because you didn’t much care if it happened or not, or you wanted it to happen but couldn’t do it ‘on purpose,’ and so on. It’s a continuum.
Asking people to ‘tell on themselves’ in these ways seems like it goes against the principle of having a cooperative relaxed interaction where you get to know someone, as it puts someone in a game-style bind where they are induced to go into a tactical defensive optimization mode. If you can convince people to respond naturally and honestly to such questions they’ll have good value, but that seems super hard, and if you are good enough to do that you’ll be fine either way.
What Would You Ask Me?
When I was doing what preparing I have done, part of that is being told not to ask about the benefits package during the interview. The benefits package is for after you get the job offer, there will be plenty of time then to ask such questions or negotiate about them. Asking now shows you don’t understand this or are focused on the benefits side, which is a bad sign, so if someone is evaluating your prep I’d expect them to think you’d failed. At a minimum, it means you don’t have a better question, either in terms of info you want or a way to benefit yourself, or both. Which either means again a lack of preparation, or worse.
Asking for a question on a particular thing avoids the issue entirely, it is true. The honest answer most of the time would normally of course be ‘no, I don’t have questions, I don’t care’ but presumably a good candidate can decide to be curious and see where it takes them. In good interviews the questions and scenarios are interesting, so I ended up asking questions like ‘what did you actually end up doing in that spot and why?’ or asking about whether I’d made any mistakes or missed anything important or what not. Continuous improvement checks, of a sort.
Asking what criteria you would use for hiring is another of those could-be-walking-into-prep but maybe it’s fine class of questions. Written tests are not obviously less revealing than ones off the cuff, and I want to hear the answer. Might even learn something.
Seeking Different Talent
The Spark
The way I would put it is that if they create their own language they don’t live in the consensus reality. Sometimes that means they live in a different reality. Sometimes it even has a relationship to physical reality. Be especially on the lookout for when they can draw others into the one they see, and on the flip side be on the lookout for it not having any connection to physical reality at all.
If you’re hiring for a position requiring low creativity and high conscientiousness, this book’s techniques are mostly not for you. That’s a different puzzle with different solutions.
If anything the most interesting thing here is the suggestion that more than 1% of people are capable of creating something grand. That’s a lot. It would be very good news if true, even if only true at a relatively young age, but that does not match my experience. Yes, at least 1% of people I have met were at one point capable, but there’s a lot of selection effects going on there.
This makes more sense in the context of an existing search for genius-level talent, which is one way of describing the job of a venture capitalist. Within that group, your prior on someone being a pathbreaking genius is orders of magnitude higher (at least one, likely two or more) and thus a 1-2% chance is plausible, while noting that most people capable of it will ultimately fail even if supported.
Buying Low
A common mystery about discrimination is that if some group was being systematically discriminated against, you could hire them at discount prices and you’d make lots of money, so why don’t we see this happening?
This applies to the usual suspects, but also unusual ones.
Talent suggests thus seeking out ways to take advantage of this inefficiency.
That makes sense. This is where you can buy low and reap the benefits. The whole idea of the book is to expand this ‘buy low’ principle to a wide variety of attributes, including negative ones.
This assumes labor markets follow at least loosely some version of the law of one price when it comes to such attributes, but that the prices involved are often wrong. I think that’s basically correct, and the question is the magnitude of these effects and how best to take advantage.
I can verify that these strategies still work for groups like nerds and introverts. As a member of the Magic: The Gathering professional community, I knew firsthand that it was full of great talent that wasn’t being given good opportunities. These people had fierce desire to win and improve, were willing to work hard, had skills that transfer remarkably well to many domains, knew how to manage risk and loss and were wicked smart. And they’d chosen to pursue Magic rather than things that looked good on a resume. It was obvious they were massively underpriced.
I recruited a bunch of them, and they very much did not disappoint. Then when the time came to actually make a game I got to do it again, putting together a dream team by asking nicely.
They directly challenge the value of the Master’s Degree, which is increasingly becoming required for jobs.
I would double down and apply the same logic to college degrees. Talent doesn’t go into the problem with forcing everyone to go to college, which seems much larger in magnitude than the graduate school problem, or that the law forces employers’ hands here to a large extent – requiring a college (or post-college) degree has a highly disparate impact, yet alternative means of measuring the same attributes signaled by a college degree are illegal because of their disparate impacts.
The Usual Questions
File under the interesting question being the opposite of the one being implied.
Given how important the interview is in terms of one’s success, why is it unusual to have such a class as opposed to it being rare that this is only one class. And where is the class that’s for non-engineers? Where’s the one specifically for founders?
Why isn’t Introduction to Algorithms better described as Introduction to Algorithmic Interview Questions? Why don’t their tests take place on white boards?
Presumably some combination of focus on ‘real learning’ being considered superior and so the departments and students involved can tell a superior story. And because employers would be expected to eventually notice such systems and adjust, although my guess is they mostly wouldn’t.
I am, of course, happy that these classes mostly don’t exist.
The Usual Data
This seems important and reminds me of my concept of Zeroing Out, in the sense that all the AI-style legible measurements that everyone is making can kind of be combined together and then set aside or even count in the opposite direction, because they’re more known than average and thus likely overnoticed, overvalued and overpriced.
You would still then need to find a way to measure the ‘market price’ of the individual, in order to not have to pay attention to this info. In some sense that is extra incentive to let the AI do its job as a black box here, so you can verify and know that market price for context then focus on the unique information available.
Other Interesting Claims
Email Response Time
I’d seen this story before, but it’s worth repeating.
I am of two minds about this. With one of them, I super strongly agree.
Given you are realizing you have an email at all, and your state is being interrupted, responding immediately is absolutely the right thing to do if you are going to respond at all and you don’t have to (by nature of the email) do something else time intensive first before responding. People who are responsive in this way are much easier to coordinate with, much more likely to be worth engaging with further and generate positive feelings, it’s all great, more people should do this. That is all the more so for important connections like a founder’s connection to Sam Altman.
The flip side of that is the importance of deep work and maintaining state. There are times when it is vital that one not be disturbed, ideally for several hours or sometimes even days, and I don’t check or answer emails on Saturday for this reason. If you respond to emails within a few minutes, it means you’re getting a notification that will potentially interrupt your state or constantly checking email. So you either need to be setting up damn good filtering on when you get such an alert, and/or have someone whose job at any given time is to be the one who gets interrupted and then decides who else to interrupt.
Zoom Versus Zoomless
I don’t think this would have been exactly my hypothesis if I’d been asked, but it seems roughly right to me, and it’s at least close. I definitely have noticed that if you want a sufficiently ‘rich’ interaction, where creativity shines and people truly get to know each other and build trust and all that, it is much better in person.
Zoom still basically always beats the phone for such matters, in my book. Phone calls still have their place, but the bandwidth drop of having zero video is extreme.
For focused interaction with known goals between already-known parties, Zoom is fine and might even be superior to in person, as it lets everyone look at everyone else and have computer access with screen share. It also eliminates all travel and lets arrangements be made on the fly and so on, so there’s that too.
Then there’s the question of… relative status signals?
I wonder how the authors would compare this to a phone. On the phone, it is not always obvious who is higher status, but often it is at least very clear who is claiming higher status and who is not, through voice tone, who lets who speak and so on. If anything, Zoom perhaps obscures those signals by telling us to focus on the visual cues because we have video, then it takes the visual cues away and we still miss the audio ones.
I wonder if that has something to do with this move he uses:
The rest of us do this too, for the simple reason that being ‘on’ and paying attention to where you’re looking and your facial expressions and so on can be exhausting, but the status thing is there too. Most of the time, Daniel will be higher status than those with whom he is stalking. On Zoom he is prevented from enjoying the full benefits thereof, which is exhausting – getting and keeping high status is exhausting and not obviously worthwhile, butit’s good in the moment to be the king and have high status. By turning off the video, Daniel gets to be the king again.
Yet there is still projection of status of a sort, a lot of which is because many people feel compelled to somehow be honest about their assessments of their own status.
Another way of saying this is that the entire frame here is buying into the idea that flair is high status and substance is low status.
Thus, you can hire for low status jobs like engineers on Zoom but should be worried about high status jobs like salesman, which is high status because it requires you to project status, and caring about status is the essence of status?
What about the idea that Zoom has the advantage of leaving misleading information out?
What’s weird is that the book elsewhere encourages you to set up encounters with third parties exactly so you can observe them, yet here it questions whether those signals are accurate and maybe it would be better not to have them. If we are encouraging people to get out of the office and into the world, that’s the opposite of Zoom, creating a richer, more nuanced and detailed physical world to interact with as opposed to a sterile fixed interview room.
My model is that by default these signals are useful as long as you are not overconfident about your ability to read them, and you don’t treat them as more important than they are, but also they could be overpriced. There is value there, and even a relatively imprecise interpretation is going to be better than random – if you properly used your new info you’d be better off. The problem is that if you overvalue the information, and also others are overvaluing the information, and especially if your mistakes are often correlated, you’re going to be overpaying and your decisions get worse, whereas letting others all overpay largely at random while all you notice is the big payment will end up going well for you.
This also has the ‘we don’t know how accurate’ thing, which is odd because the whole book is full of things whose accuracy is unknown. That’s the wheelhouse, so it’s odd to make a fuss about it here.
There’s also the thought that face to face is inherently more resonant because of its inconvenience.
It’s hard to actually imagine the technology properly, with its full potential attached. You’d need to be feeling the breeze on your shoulder and the heat of the sun, to smell the Paris air, to have the visuals be indistinguishable from normal vision and so on, in order to get to this point. That’s not to say getting there is even implausible, I do expect the tech to get there eventually in worlds where AGI is difficult.
In that world, will the Oculus version be less resonant and memorable because it wasn’t inconvenient? Yes, at least somewhat, that’s definitely a thing, but I think a thing that is generally overvalued, and that would shrink in worlds where Oculus technology improved. We definitely have a bug in our system where having paid additional costs has this effect, but I consider it mostly to be a bug, and one we can correct over time as it makes even less sense and gets less reinforcement.
Who Are You Performing For?
I initially focused on the question of who you were trying to impress and the whole thing felt alien. When I reorient to who you are performing for, or even more than that who or what you think will pass judgment, suddenly it makes sense. It is still presented here largely in terms of trying to impress, though:
To me, if the judgement you are looking for is whether or not you impress then there is a problem no matter what is supposed to be impressed. ‘Look at how impressive this is’ seems like a wrong goal, whereas it makes sense to think of some algorithm or person or system as being the one that passes judgment. This passage makes me wonder whether that perspective is weirder than I realized.
If someone is stuck in the past, ask which past and in which ways they are stuck. Some pasts are in many ways superior.
Moral Non-Dilemma
This was the first statement in the book I strongly disagreed with:
Most of the time, in my experience, firing someone feels terrible and is stressful. Bryan Caplan believes, I think correctly, that this causes a lot of necessary firings not to happen. Yes, occasionally there is someone who richly deserves it, and it is indeed good to be willing to fire people when it is needed. Still, something is very wrong about you if “You’re Fired” is a line in which you take joy.
Hiring people is the opposite. It feels great to hire someone. It even feels great to try to hire someone. Even if they turn you down, as long as it was a decent offer, it usually feels like you made their day. When people offer me jobs, I’ve always taken it as a complement, even if there was never any danger I’d say yes. It’s nice to be asked.
Yes, the people you are not hiring are often going to be unhappy. But not all of them. If you’re ‘doing it right’ then many of those who you do not hire will also be happy with the experience.
If you do a good interview, often the assessment of a poor fit will be mutual. They won’t want the job any more than you want to offer them the job, both of which reinforce each other. At other times, they will be sad they didn’t get the job, but they’ll understand that they were unlikely to be the best candidate – they know how they did. In many cases, they should take not getting a job offer as evidence they should be happy not to have one. It’s not welcome news, but it’s fine. Sometimes the job interview was mostly for practice – either the job isn’t something they wanted and/or it was always out of reach – so there’s no harm in being turned down.
If you do a great interview, they’ll be happy they went through the process because the process was inherently worthwhile. The two great interview processes I went through fall into this category – one was a great use of time even though it didn’t get me a job offer, and the other was a great use of time that also ended in a job offer.
The book notices this phenomenon, that the great interview experiences I’ve had gave me very strong positive impressions of my time spent and also of the companies involved:
If the position you are offering is special everything is great. The more you are special and looking for special talent and using techniques like the one in the book, the more likely those who don’t make it are to consider this time well spent. One can think of this as a (stretch) goal, to make everyone happy in hindsight.
What about the ones who are unhappy? I would say that the people chose to come in for the job interview, which means they saw a lot of value in trying. When they get the interview, are most of them not happy? That’s a strong revealed preference. And I’d say that if you had to guess whether the average person interviews too rarely for a new job or too often, I’d be shocked if the answer wasn’t that they apply and interview far too rarely. The upside of a new job is huge, the downside of failing is low (aside from one’s pride) and in failing one often learns a lot or has an interesting experience or story.
In my experience, ‘I applied for the dream job and didn’t make it’ feels better than not applying. This is true even for the time I interviewed for what at the time was a dream job, left thinking I definitely had it, then got informed two weeks later that I didn’t. The finding out sucked, but I’m so happy that I cold emailed them and got my chance.
Even if I did believe I was making the people who applied but got turned down unhappy, I don’t see a moral dilemma or uneasiness here at all. Competitions are great, everyone entered knowing the odds were against them but the payoffs were worth it.
What about jobs that are, shall we say, less exciting? What if it’s a standard entry level position, and you have 100 applicants for one spot, and a lot of them would be fine and none of this is all that fun or interesting? If everyone involved is kind of desperate and worn down and unhappy about having to be there?
I still say it’s rather easy to hold your head high. Everyone is there for a reason. At the end of the day you got someone a job, and they got to stop looking for one. Even if it’s not the best job? That’s still pretty cool.
The Reference
The book isn’t big on references, but does offer some guidelines, advice and questions on what to do with them if one were so inclined.
A lot of the problem with references is legal worries or other ass covering. Making sure you are not sued and don’t cause an incident is often the order of the day. Beyond that, yeah, do what you can most of the time, I suppose.
None of these suggestions seem likely to move one into conversation mode. It all seems like a kind of a bidding game. The person does not want to lie too much and wants to not spend too long. You ask if they would be willing to bid a claim of how good the person is, you get how eagerly they do that or don’t do that. You keep going until you sense where they stop and where they start having to hedge or think about it, and you get your information. Your goal is to find the places they won’t bid, which become the weaknesses, and such.
Yet a lot of these questions seem like requests for very high bids. The last set are questions that should not get a genuine yes so often. How many people are there where, if they disagree with me, I think they are as likely right as wrong? Depends on context – some people are likely disagreeing to correct a spelling error or what not, so they’re usually right, but a meaningful disagreement that was sustained upon reflection? I’m not convinced anyone goes on that list.
Who is the Competition?
My general instinct is that if you have a counter-intuitive idea of who you are competing against, either that’s about your internal drive or you’re probably very wrong.
The most famous example of this is when Reed Hastings, the CEO of Netflix, said his competition was ‘sleep, and we’re winning.’
That’s an interesting sideways insight into an aspect of the business. It’s also insane.
Tyler and Daniel also have interesting less insane thoughts on their competition.
I think Tyler is mostly right.
I want Daniel to also be mostly right, but I am skeptical.
The thesis here implies that success as a founder is not so correlated with ability to convince venture capitalists to give you funding, at least in the current round. It’s saying that mostly other VCs are not interested in the same prospects as Pioneer. That would mean that Pioneer was providing lots of value, by identifying worthy founders and giving them a start they wouldn’t otherwise get.
Yet this is highly suspicious. If they are such top prospects why wouldn’t any other VCs be willing to fund them? Why would they be willing to give up? These imply that such would-be founders are not so driven and that they are not so good at fundraising and convincing VCs they are strong candidates. A true top prospect won’t have to and wouldn’t take the safe job offer.
So all this only makes sense if Pioneer is identifying counter-intuitive (to VCs and to themselves) candidates, and that is where they are getting their value. It’s not impossible, but I notice I am skeptical.
The best prospects are, I wager, mostly obviously great prospects, and mostly have no trouble finding interested VCs. The good deals are where VCs are directly competing to get into the round (or at least they are in good times like a few months ago) and they’ve managed to convince founders not to simply jack up the price to market clearing rates when this happens. If no one else is interested, that’s a reason to be worried on multiple levels.
I am doubly skeptical because the book reminds us that a primary motivator of VCs is the fear of missing out (FOMO). You get in trouble when you miss out on the next Facebook or Uber. Yet if someone doesn’t found the company, while you might have missed out on the next big thing, no one will ever know because it doesn’t exist. You won’t get the blame.
More generally, perhaps as a bit of a digression, I am very open to the idea of your competition not being what you think it is. As a game designer, I would often face people whose default assumption was that other games were ‘competition’ and that their success would be bad for Emergents. That the games were fighting over the same players. There isn’t none of this, but I firmly believe that mostly good games are good for games, and good collectable card games are good for collectable card games. They grow the market, they draw people in, they give us all a good name. Without Hearthstone having been created, I think Magic would have fewer players, not more. Whenever I discover a great game, like Storybook Brawl or Slay the Spire or Rogue Legacy, it makes it far more likely to play something else that is similar. That’s the sane version of the Netflix argument.
And just as I was writing this I came across another good example of the ‘you think you’re in competition with others like you but mostly you’re simply trying to be good enough’:
In my experience this is very right except that competition means there are existing pairings that are differentially likely to take the best pairings off the market, although they also induce market entry where people are willing to put themselves out there. Standards you have to meet go up and perhaps the quality of the pool facing you go down as competition increases, but you want to be where there’s more ‘competition’ not less.
I’d like to think of VC that way, where more better VCs means people get more excited to make more good companies (and to fund VC firms) and so more good VCs are good for VCs, and places like Google or Jane Street Capital with their good jobs are the true competition. I am not convinced.
Status-Seeking, Idea-Seeking and IQ
Note that both groups here are seekers at all, which is better than not seeking.
The broader claim is that you should seek out idea-seekers rather than status-seekers. For most purposes in life this is clearly true. If my friends were not idea-seekers I would urgently need new and better friends.
Alas, this can be difficult to put this into practice. When one needs to interact with the elite or do business, will one run into more people who are paying attention to ideas or to the elite?
Daniel Gross is a venture capitalist. He wrote a book with Tyler Cowen, and the story of the book shows him being drawn to Tyler’s idea-seeking. But no matter what he writes in a book, when he decides who to fund, does he reward idea-seekers or status-seekers? Which should he do if he wants to make good investments?
A repeated theme of Talent is that IQ is overrated and overpriced. Everyone is looking for and recruiting for IQ, so you’re not going to find neglected talent by looking for IQ.
It is pointed out that IQ doesn’t matter much for earnings.
Also offered as evidence is that Swedish CEOs, on average, are not so smart.
Higher IQ people have more tendency to idea-seek, and they have more comparative advantage in idea-seeking versus in status-seeking. CEOs are smart, but not so smart. The ‘CEOs of idea-land’ are much smarter in terms of raw IQ. Whey aren’t they rich?
Idea-seeking does not make you rich (in money, anyway) the way status-seeking makes you rich. It does not help you climb the corporate ladder and become CEO. It helps your start-up, but I would be surprised if it helps you as much as status-seeking. Status-seeking behavior helps you raise a round, in part by giving people the impression you will be able to raise another round.
Thus, while Tyler and Daniel both realize while idea-seeking that they should be looking for idea-seekers, they are still looking for attributes that predict outcomes that are correlated with (and caused by) status seeking.
I am reminded of Paul Graham and Y Combinator. Paul Graham often goes into idea-seeking mode, in which he writes excellent essays in which he believes one set of things. He gives advice to people to do the things that would be rewarded if he acted according to the principles in his essays. Then he goes out and (according to all reports I’ve seen, including occasionally his self-reports) does a different set of things in practice, occasionally admitting that he’d do better if he did more of the things he writes on paper. Part of the way he seeks talent is therefore seeing who can tell the difference.
Consider this quote as well:
What is a business plan for? In theory the business plan is so investors can decide whether or not they think the plan will work and whether it is something worth investing in. Sometimes that is even true.
More often, both in my experience and by admission of many VCs including Daniel here, they don’t care about the business plan for its own sake. The point of the business plan is to evaluate the founder, and its success is defined by its success at convincing you of its success, as part of systems of the form ‘we should fund this round if and only if we should fund another round.’
If you mess up making the case for the business plan, that’s important because it means you can’t make cases for things. Making cases for things is important. If you can’t make a proper polished business plan that does the things I’m expecting, you won’t be able to properly give people polished things that do what they are expecting to find. Being able and willing to do that is important too. And so on.
Is this a better version of the method, since recruiting talent is a key skill that isn’t self-referential? A little, but mostly I think the explanation here is a rationalization, a way to pretend one is ‘doing it for the talent.’ Yes, you have to sell recruits on your company, but that’s a shoehorn of the concern about whether the founder can be an effective salesperson of the company’s prospects in general. Yes, you can say that in particular recruits have to be sold on the company and its prospects and I can confirm this is to some extent true but mostly the ways in which you do that are very different from the Way of the Business Plan.
I also don’t see how it interacts with academia’s anti-interview claims, since those (as far as I can tell) refer to being anti-interview in situations where the person’s job won’t centrally be about salesmanship or about recruitment.
Avoiding the Usual
I expected the next line to be ‘Remember that this is in some sense an interview book.’
Testing someone’s preparation level and approach is indeed somewhat useful as a matching system. If you’re testing the results rather than asking for how they prepared (which as I mentioned earlier feels seriously ‘out of bounds’ to me), it is obviously in bounds but the same dynamic applies – do you want them to be ‘properly’ prepared, or not? If you ask a standard interview question, and get a good standard answer, is that a good or bad response? Which again, to me, goes back to whether you want someone who puts in effort to signal and to cram and to game systems in such ways, or if you want to avoid such folks.
Who Gets Funded?
The question is always what are the big winners? Or rather, what does your algorithm think are the big winners, or plausible enough candidates to get the funding?
This seems telling:
It should be a red flag when you notice that you are rewarding fear. Rewarding super-high levels of ambition and drive makes sense, but I strongly suspect that this is not what inspires the fear and thus is not the thing Daniel’s emotional responses are primarily measuring. It is instead the third thing Daniel mentions – that ‘they will do anything to succeed.’
This sense of ‘they will do anything’ is indeed a central reason I experience fear when encountering some people. And yes, a lot of the people who give off this vibe are in some central sense highly ‘successful’ in business or politics.
The mode of operation is highly useful on multiple levels.
When given the opportunity to betray someone, or their own principles and other preferences, for their own success, they will do so. That directly helps one succeed in the moment one does it.
In a good world, the intention to not keep one’s word, to not care about what is good and what is not so good, to reliably betray one’s friends and allies would it be convenient, to have every intention of selling out one’s cofounders, would be considered a negative when one is deciding whether to be one of the people to be later sold out.
Instead, in these worlds, it is considered a positive.
Others see that you are like this, and think you would be a good investment and a good ally, a good leader, someone to back.
Others see this, and they worry that you will become powerful or successful and they’ll miss out, and/or will use what power you already have against them if they don’t cooperate. Yes, they’ll screw you over anyway, but less, right?
They will constantly seek advantage, and use the threat of what they will do to constantly seek advantage in order to seek further advantage. They will alter the deal, you can’t turn your back on them for a second. But you can prey they don’t alter it any further.
Thus, such people actively work to trigger this fear response. They know that those deciding to whom to award power have learned to reward the invoking of fear, to reward those who show themselves to be this kind of monster and to lack any sort of other principle when it counts.
I have never met a truly successful public-facing politician, but it is clear that many successful ones are very much doing this – to avoid distraction I won’t name any and ask those commenting also not to do so.
I have met a number of billionaires and other rich businesspeople, and most of them show a central pattern of either being this thing, rewarding this thing or both. There are exceptions, in particular based on my experiences Sam Bankman-Fried has the drive and ambition but as far as I can tell is not doing this.
I’ve been an early employee or cofounder in three start-ups where I wasn’t the CEO that got as far as hiring a bunch of people. Two of them were very much this. Both, in the end, betrayed me and betrayed the business in massively destructive ways. The exception was, I strongly suspect, severely punished in fundraising for her not being this, and we had to give up most of the company to keep things going. It got us mostly expropriated.
This is a perfectly understandable algorithm to be using when everyone around you is using it – if your goal is to make money and others are going to reward a founder by funding them later and people will want to help them succeed then you want to be funding them now, even if there’s a good chance they’ll do everything in their power to screw you over later. The big hits will still often be quite big. So I get it.
Even if I was going to be part of another such company some day, the temptation to back another such person is not small – yes the downsides are obvious but at least you’re in the game.
Yet our eyes must be open about what types of people emerge at the end of such a process, and also what we are encouraging our most talented and ambitious people to become.
Invest in IQ? Overrated or Underrated?
An often repeated claim in the book is that IQ is, with notably rare exceptions, overrated and overpriced.
It’s worth saying up front that I am of course heavily biases against this claim.
I am surprised the 183% rise in inventors would be so low and it makes me wonder how many inventions are being misattributed. Invention seems to be super G-loaded to me.
In other places the correlations are not so high between ‘success’ and IQ, such as in the Swedish CEOs, and this is used as the main evidence that IQ is overpriced.
One could easily argue the opposite, if one has other reasons to believe that high IQs are valuable.
If IQ is not commanding a high price in the marketplace in the form of promotions and high pay, but we are confident it is highly valuable, that suggests that IQ is underpriced.
That seems like the more natural interpretation, that something commanding an unexpectedly high price is overpriced, and something commanding an unexpectedly low price is underrated.
Certainly in the jobs I have chosen to do I am confident that IQ is highly valuable and thus underpriced. That doesn’t mean it is the only factor, or that all other things combined don’t matter more – insufficiently high IQ would be a dealbreaker, but so would a bunch of other things – but if I wanted good work I would want to be long IQ.
As a clean example in the other direction, take height, which I was surprised wasn’t mentioned in the book that I can recall. Height is known to be correlated with ‘success’ and compensation. Taller political candidates win more often, taller employees win promotions more often and all that.
From that, we could conclude that being tall represents talent and thus is underpriced. Or we could conclude that there is a bias in favor of tall people, and that while this is worth knowing in contexts where you want to take advantage of that – you might well choose to run a political candidate or fund a startup founder because they are tall knowing others will respond positively to them being tall – you would mostly want to be suspicious that tall people were overpriced, and be excited to hire underpriced shorter people.
I’d be curious to see whether or not this could be used as a proxy for the degree to which merit is winning out at companies over status and superficiality and politics. Thus, the Long Short People ETF would evaluate the C-suite and board of companies, and invest in those that are on average unusually short. Would it have alpha? I’ve never seen the paper studying this.
The question comes down to why the high IQ people aren’t succeeding – a literal case of “If you’re so smart, why aren’t you rich?” How much of that is performance, versus smart people not being inclined to learn and play the social-political games, not being excited to participate in maze behaviors and so on, being different and thus seeming weird and so on.
And how much of it is active political selection against IQ? When selecting a CEO, I can imagine someone thinking that you wouldn’t want someone ‘too smart’ for any of several reasons.
What else do we have?
That makes sense, especially if you don’t expect doctor or lawyer to be the comparative advantage of high-IQ people and also if you think high-IQ is partly causing parental education. It’s something though.
So this is the flip side of the CEO study and other evidence that high IQ doesn’t pay – IQ actually pays quite a lot in this sample. But that’s directly contradicted here is perhaps simply an outlier?
I don’t defy the data – I believe that the smaller numbers are probably accurate. But that doesn’t mean accepting that this means the market is pricing this correctly, or that it means that IQ doesn’t much matter.
Instead, it tells me that smart people are not being properly utilized.
It is not obvious to me whether even the expensive value overprices or underprices IQ, as it depends on the circumstances. If you could ‘buy’ more IQ for your employees with higher pay at the super high price of 5% per point, would you? Certainly I agree it matters most at the very top because you can have dramatic oversized value generation, so I would happily pay 10x (or 100x, or if I could afford it 1000x) to get Von Neuman if I could put him to good use.
The more interesting question is if you buy your way from 110 to 120? In most jobs probably not, but in others definitely yes – I’d certainly rather have less engineers that were smarter. Other places my willingness to pay is a lot lower, but still mostly in the 1%+ range.
If I could buy IQ points at 0.5% per point? I’m a buyer for almost any task.
The earnings premium almost certainly overestimates the cost of ‘buying IQ’ because the higher pay comes in large part from what profession and job are chosen. So if the position is fixed, the marginal cost of more IQ is going to be far lower, if you can get it with higher pay. Alas, most of the time your pay scale is fixed and it’s more like you are trading off against other traits, so it’s harder to know if the deal is good.
I’ll go ahead and say that if the ‘market price’ of IQ is only 0.5%-1% per point, then I’m willing to go ahead and say this is crazy, IQ is dramatically underpriced, and you should try to get as long IQ as you can.
(Once again, reminder that I’m probably very biased.)
One hypothesis for this observation is that to become a not-very-top chess player, you have to be smart enough to play chess well, but you also have to not be smart enough to realize you have better life options than chess. Tyler often points out that being a chess player who isn’t at the very top is a poor life choice.
I had that choice to make. As a child, I was winning local tournaments and going to chess camp, once I came in 18th at the Elementary School Nationals, but I clearly had a choice to make. Either I could get super serious about my chess at the expense of other things, or I could accept that my life would consist of something else. I made the right choice, I like to think due in part to my (and my family’s) intelligence.
I do still notice I am surprised the link isn’t stronger, but chess-in-practice includes a lot of other skills that are not so IQ-bound, and also chess below the top mostly doesn’t reward innovation. My experience in Magic (where oh boy was success IQ loaded when I was playing professionally) is that it was the need for continuous innovation, the solving of new situations and finding new ideas, that was the big filter. Lots of players could practice in a static situation and get good, with success a lot more based on discipline and dedication and so on, which matches chess.
On the lower end, of course, for non-expert players, IQ is super correlated with chess ability with or without controlling for time spent on the game, and one does not need a study to know this.
That brings us to this claim.
I have not seen or searched for a study on this, but I would strongly disagree. If I knew a company was run by lots of unexpectedly smart people, yes I would be more excited to invest. Certainly it is no guarantee of anything and likely increases the chances of a true blow up – see Enron’s story being called The Smartest Guys In The Room. At one point the MetaMed team was called ‘the highest IQ team in the valley’ and it did not get us very far.
It’s still the way I would bet, so why am I challenging the EMH here?
The mechanics here imply to me the EMH will fail here, and also I mostly don’t buy the stronger versions of the EMH in general.
Note that the book uses the framework of EMH-style efficient markets in many places, but fundamentally says that the talent market is deeply inefficient and much or even most great talent isn’t even noticed.
There are, as far as I can tell, two basic ways to argue that there won’t be Alpha in investing in IQ.
The first is the idea that CEOs and other high-up assets of companies are hired in a competitive market, and IQ is being correctly priced, so they’re paying the cost in a combination of more compensation and sacrificing other traits.
I think that is simply wrong. These markets are not remotely efficient, and also one asset of companies is their ability to attract better talent and hire better. I think mostly smarter people at the top aren’t trading off against much of anything except perhaps slightly higher pay and the higher pay is priced in and also not that important.
The second is the idea I think is being claimed above, that the market has assessed the intelligence of these folks, that’s public information, so it’s already priced in or maybe even overpriced.
I think this is a great example of why we shouldn’t expect a lot of public information to be properly priced in. Most people who are actively investing aren’t looking at all at these questions, except maybe in exceptional cases of superstar CEOs. So most of the money moving the market around, and most (or all or almost all) of the mathematical models don’t have this as a variable. The money that does look is likely estimating its value roughly properly, so it won’t fully adjust the price. The way this wouldn’t be true is if the people looking often dramatically overvalue intelligence, but I doubt this.
The natural market dynamic outcome should be for this to be underpriced.
Not investment advice, of course, but I think this is much more of a ‘hidden virtue’ than it is being given credit for. I can get into the opposite mindset with effort, but it seems absurd.
The book holds out the most promise in terms of hiring for IQ when the market forces are not so much in play.
This does seem right to me, although I’d expand the mechanism beyond the one stated here. Intelligence is a lot of what lets someone expand and develop, which young people are much more likely to do than older people, and people who haven’t had opportunity are more likely to do than people who have already acclimated, which reinforces this effect. That’s on top of the ‘we already controlled for that and also adjusted the market price’ effect.
The Whole Package
This is a very real phenomenon, where the top achievers seem to be world class at many different things at once, and have lots of exceptional level traits at once as well, in ways that compound, and that they file away big win after big win. I have learned that if someone is exceptional in some ways, one shouldn’t expect a trade-off where they’re underwhelming in other ways. One should instead expect to find more ways they are exceptional, and for them to figure it out whatever it may be.
This happens often enough that ‘this is a remarkable confluence of skills’ seems like the wrong explanation. It’s no accident, the exceptionalism feeds into more exceptionalism (not always, but the ones where it doesn’t we don’t hear from them much after a while). Knowing that one can excel that much and having the drive required combines to keep them pushing forward.
Thus the Whole Package to me is more about package assembly than the package, plus noting whether there is a key flaw or weakness that the process isn’t going to be able to fix.
Not To Start a Political Argument
I think this is actually a big hint on what is going on. Intellectual smarts have huge advantages, but also increase distance between you and regular people.
It was before my time, but looking backwards Jimmy Carter always felt somewhat off. He was that thing called out of touch, where he didn’t appreciate how people thought or what they would think was important. Carter won the presidency (as I understand it) by being the first person to understand the importance of the primary schedule and focus on early states – being smart let him game the system in a way that later became standard – and then he won a general election under favorable conditions, but he was never a presidential-level politician. He essentially ‘forced his way up’ through intelligence, but didn’t have The Whole Package.
Richard Nixon had a similar thing, as I understand it, where he was fundamentally uncharismatic. No one liked Tricky Dick, he was the opposite of the Teflon that many great politicians seem to enjoy. Yet he still became president, again by using his abstract intelligence to punch above his weight in various ways, and being willing to do what it takes in various ways to make that happen – I think because he thought he could do enough good in power that the ends justified the means. Eventually it all caught up to him, but it’s not like plenty of other Presidents didn’t do pretty terrible stuff, and I suspect a lot of it really was that Nixon was fundamentally unlikable. And a bunch of the rankings I looked at didn’t have him as that smart overall.
Woodrow Wilson has been cancelled for being a racist, and also did various other things like throw the first amendment and other civil liberties onto a dumpster fire, but those were his choices and it’s weird to blame that on his being smart either way. Wilson’s big failure, the one that interacts strongly with his IQ, was the Treaty of Versailles. Wilson’s intelligence allowed him to propose an actually really good peace – the Fourteen Points were not perfect but were damn good, the problem was they were too good and overreached his ability to get his way at the conference, and the final outcome was quite bad when he likely could have done better by being more realistic and less ambitious. But it was very directionally right, and this seems like a case where he simply lacked the skills that mattered most.
Book smarts were not the right kinds of smarts.
Whereas the first hit on Google actually had Bill Clinton as smarter than all three of them. He wasn’t as ‘book smart’ but it was obvious he was super smart, and he often played things very smart, and it paid off in many ways. And when I look at the ‘lower ranks’ of the presidents, I see a lot of names where I’d say their lack of intelligence was a major handicap.
These are sufficiently partisan choices that I won’t go into details. I include the quote here for completeness.