Third-wave AI safety needs sociopolitical thinking

Richard_Ngo

LESSWRONG
LW

Comment Permalink

I had been considering the hypothesis that they would obviously not have kids, because look how much work they have to do. Just people most people have kids doesn't mean a really small set of strongly selected outliers do – I expected that the most successful people also have very few major illnesses, have 99th percentile IQs, have married parents, etc. (I'm not sure that the health and parental ones check out either, but I think the 99th percentile IQ probably does.)

I had anticipated that most great scientists, inventors and builders would not have children, and was actively surprised when I found out (before today) that Elon and Demis did (and a few other people I had thought to check). It changed my attitude toward having kids substantially.

See in context

54 Anna and Oliver discuss Children and X-Risk

by Raemon

1 min read

54

This Sunday, Anna Salamon and Oliver Habryka will discuss whether people who care about x-risk should have children.

A short summary of Oliver's position is:

If we want to reduce existential risk and protect the cosmic commons, we have some really tough challenges ahead and need to really succeed at some ambitious and world-scale plans. I have a sense that once people have children, they tend to stop trying to do ambitious things. It also dramatically reduces the flexibility of what plans or experiments they can try.

And a short summary of Anna's position is:

Most human activity is fake (pretends to be about one thing while being micro-governed by a different and unaligned process, e.g. somebody tries to "work on AI risk" while almost all of the micropulls come from wanting to keep fear at bay, or for a different person from wanting to feel busy and important, rather than from actually seeing and caring about AI risk). Returns from stuff that is less fake seem really a lot better than returns from stuff that is more fake -- particularly for areas that are easy to be confused about, such as AI risk. I suspect the desire for kids/lineage is really basic for a lot of people (almost everyone?), (despite being often not very conscious, though usually more conscious at 40 than at 30, and more at 30 than at 20), and that aligning with it leaves most of us with more of a shot at doing real things.

We'll be meeting in Zoom at 12pm PT

https://us02web.zoom.us/j/7151633248?pwd=RnpaUFNZbVYvTTV6RWJVZFFSQ0VPUT09

Posted on: 26th Feb 2021

54

Mentioned in

274Raising children on the eve of AI

53Enabling Children

17Should you have children? All LessWrong posts about the topic

New Comment

44 comments, sorted by

top scoring

Click to highlight new comments since: Today at 3:31 PM

[-]Owain_Evans4y590

I'd like to see discussion of data rather than mostly a priori argument ("I have a sense" ... "I suspect desire"). For aggregate data, there's SSC survey and there are studies of "ambitious" groups (e.g. the Harvard Men study, Benbow on precious math talent). There are also anecdata of the exceptionally ambitious. E.g. Musk had first child age ~30 and has many kids, Hassabis had first child aged ~29. It seems Jaan Tallinn had kids starting in his 20s before founding Skype (Wikipedia). Bezos has 4 kids (started age 37). Gates has 3 kids (started age ~40). Turing award-winners David Patterson and Judea Pearl had kids in their 20s before their biggest contributions. Yoshua Bengio in his 30s. etc

[-]mingyuan4y390

I looked at the 2020 Forbes list of most powerful women (which was the first thing that came up when I googled "powerful women") and spent about an hour total investigating whether they have children. The Forbes list had 100 women; I was able to find probably-reliable information about children for 95 of them (many of them are very private about their personal lives, which is reasonable). Of those, 25 were childless, or 26%.

This source indicates that 85% of women in the US have children in their lifetime – a lot of the women in question weren't from the US, but I'm not putting that much effort into the investigation, so close enough. So that's 26% of successful women being childless, vs the US average of 15%. Also (cherry-picking alert), out of the top three women on the list, two were childless (Angela Merkel and Kamala Harris).

[-]Daniel Kokotajlo4y30

Thanks so much for collecting this data -- I find those stats rather encouraging; 26% isn't that different from 15%.

[-]Gunnar_Zarncke4y30

We will see all kinds of strange things going on at the very tails of the distribution (as illustrated by the Merkel + Harris examples). I think that should not inform our strategy as a community (except in so far as the preparedness for taking extreme measures when needed later).

[-]jefftk3y20

Among the 26% who don't have children, I'd be curious what the age distribution is: I wonder whether some of them may still?

[-]Lukas_Gloor4y160

I'm also curious if Oliver or Anna think there's a difference between EA longtermist endeavors vs. the reference class you've drawn from ("scoring very highly on broadly accepted metrics of success"), and if so, how that difference manifests itself for having children.

[-]habryka4y90

I found this analysis one of the most useful I have found: https://academic.oup.com/psychsocgerontology/article/64B/6/767/550078

Abstract:

Compared with married parents, childless married couples tend to have slightly more income and about 5% more wealth. Unmarried childless men enjoy no income advantage over unmarried fathers but have 24%–33% more wealth. Compared with older unmarried mothers, unmarried childless women have 12%–31% more income and about 33% more wealth. The strength of these relationships increases as one moves up the distribution of income or wealth.

At the higher levels of wealth, the effects become quite strong.

And when I looked at random datasets like "what percentage of famous scientists according to this one random collection of biographies of scientists are married?" I found about 27% of them to be unmarried, which is a stark increase from about 15% population average. Traditional marriage here is used as a proxy for having children, partially because I think they have effects via the same mechanisms, and partially because they are heavily correlated.

[-]habryka4y40

There is also this paper, which aims to show that as soon as great scientists marry, they very quickly stop producing great achievements, but something about it irks me out and I don't currently put a ton of weight on it:

[-]Owain_Evans4y21

I haven't re-read the paper, although IIRC there are critiques online of this paper and the author's other statistical analyses. How strong do you think the evidence is for the counterfactual "If person has chooses to have kids, their chance of major achievement will drop substantially" (for a range of different people)? Ideally there'd be natural experiments (due to infertility or someone who didn't want kids raising their sibling's children etc).

These graphs aren't that different and (I'd guess) it wouldn't be hard to p-hack to get the intended result. Rate of being unmarried will vary over time and with country and this will correlate with age of achievements (e.g. if people in biology peak later than math/physics, if there's more biologists in UK and math/physics people in Germany and Italy). And there's the causal / counterfactual inference..

[-]habryka4y20

I am pretty confused on this, and as I said above, don't put much weight on this study because I also have some sense that the author isn't super trustworthy (though I haven't found any critique of this specific paper).

Overall, my current sense is that the effect on women in particular is quite strong, and women who choose to have children will reduce their chance of major achievement by at least 40% or so. For men it's probably weaker, and I am a lot less sure what the data says.

[-]David Hornbein4y41

My impression from many, many biographies of ambitious and world-changing people is that historically significant politicians and entrepreneurs nearly always have kids, to such an overwhelming extent that >50% of exceptions are biologically infertile or are gay. Among historically significant scientists, engineers, and philosophers, most have kids, but it's not nearly so overpowering and I'm not immediately sure if it's different from what you'd expect of baseline "successful people".

Of course this is purely descriptive rather than mechanistic, but there's overwhelming mountains of data suggesting that world-changing impact is at least consistent with having kids, and in some fields, being the-sort-of-person-who-chooses-to-have-kids seems like it's nearly a prerequisite.

[-]Dirichlet-to-Neumann4y300

When getting married meant "having a wife that would take care of you so you can focus on your works" or even "get a big fortune/political power from your wife" it was certainly beneficial for ambitious or want-to-be world changing MEN to get married. For women it was pretty different, and as a matter of fact the (rare) examples of women I know who were hugely influential before the 20th century often went unmarried (think Sophie Germain, Maria Gaetana Agnesi, Hypatia, or Johan of Arc, Catherine of Sienna...).

For women holding positions of power, a general trend seems to be that they held power either as a regent for an absent husband or underage child, or if holding the power in their own name kept unmarried. Elisabeth I of England is an obvious example.

[-]mingyuan4y560

Yes, thank you for pointing out the gender difference! As a woman working on x-risk and dating other people working on x-risk, I'm terrified by the stories of people like Laura Fermi and Clara Haber. Both were promising scientists in their own right, but once they married their famous scientist husbands, their own careers were abruptly ended. Laura felt like her life had become a footnote to Enrico's, was expected to take care of the children, and felt demeaned at social gatherings, and it wasn't until after her husband's death that she achieved any measure of recognition in her own right (as a writer). Clara was miserable and felt stifled in her marriage. From Wikipedia:

It has always been my attitude that a life has only been worth living if one has made full use of all one's abilities and tried to live out every kind of experience human life has to offer. It was under that impulse, among other things, that I decided to get married at that time... The life I got from it was very brief...and the main reasons for that was Fritz's oppressive way of putting himself first in our home and marriage, so that a less ruthlessly self-assertive personality was simply destroyed.

She had a son essentially against her will and ended up committing suicide – though to be fair, she had a lot of issues and it's not clear exactly why she killed herself.

(Note that the above is based on imperfect recollections of biographies I've read where these women played quite small parts, so the claims might not be exactly accurate, but that's the gist.)

Anyway I read those stories and I'm like, fuck no. I could so easily see that being me, especially if I lived a hundred years ago. Luckily my primary partner is Habryka so I'm not in danger of being pressured to have children, but also, those things are not unrelated. I'M ON YOUR SIDE OLI RIDE OR DIE

[-]Raemon4y160

I indeed hadn't been thinking much about the gender differences and am kinda embarrassed about it.

I'm kinda deliberately putting on my Cold Utilitarian Hat for this whole conversation because it seems like locally good practice for me (normally I'm the guy pushing against the Cold Utilitarianism), but within Cold Utilitarian Hat, if you have 15-30 years to stop the world from ending, having half of your people get consumed by childcare/family-rearing is a pretty big deal (whether or not it also ends up having gender dynamics attached to it)

[-]Raemon4y150

There's an important distinction between "be the sort of person who already was going to be majorly ambitious" and "be the sort of person who's sort of on the cusp of Try To Be Ambitious who could be part of an ambitious community if an ambitious community were there to support them, but might easily slide into mediocrity." Our community doesn't have the luxury of being built entirely out of Elon Musks.

But, I agree that I should update somehow about many famous ambitious people having kids.

[-]Zack_M_Davis4y140

being the-sort-of-person-who-chooses-to-have-kids

What years were most of these biographies about? Sexual marketplace and family dynamics have changed a lot since, say, 1970ish. (Such that a lot of people today who don't think of themselves as the-sort-of-person-who-chooses-to-have-kids would absolutely be married with children had someone with their genotype grown up in an earlier generation.)

[-]David Hornbein4y40

At conservative estimates, I've looked into dozens of significant pre-industrial people, dozens of significant people between the Industrial Revolution and 1970, and >100 significant post-1970 people. Among historically significant people and leaders-of-fields who get articles and books written about them, there has not been any change in who has kids large enough to jump out at me, except that in the past ~20 years there have been somewhat more openly gay entrepreneurs in the West.

[-]Steven Byrnes4y80

My impression is that difficult for two people to raise kids while both are pursuing intense, ambitious careers. If it's only one of the two people, no problem. See Anne-Marie Slaughter's writings on this, for example. I'm interested if you know of counterexamples to that.

[-]Owain_Evans4y40

Clintons? Obamas? There are many examples from academia. Nobel Laureates Banerjee and Duflo, or these two economists:

During the pregnancy, they employed a doula, or birth companion; after the birth, they hired a nanny named Ellen, who had a BA and was finishing her master's degree in education policy, and whom they paid $US50,000 (about $65,000) a year. "We didn't just want a warm body," Wolfers says, over his second beer. "Some people just want someone who'll keep their kids safe, but we wanted more than that."

[-]Steven Byrnes4y70

Were the Clintons and Obamas "raising kids while both were pursuing intense, ambitious careers"? They were raising kids at some point in their lives, and they were both pursuing intense, ambitious careers at some point in their lives, but was it simultaneous? My impression (could be wrong) is that Hillary had an ambitious career before kids, paused it for a decade or two, and then got back into it when her kids were grown up. Michelle "cut back to part time" when her kids were 7 & 10, and quit entirely sometime afterwards, if I'm understanding Wikipedia correctly. (Anne-Marie Slaughter's book says that power couples in USA can get through infancy and early childhood to some extent, but it gets really hard for somewhat older kids and teens.)

[-]David Hornbein4y40

Also the Curies, the Coris, the Durants, and others. What these all have in common is that they worked together on the same project. Offhand I can't think of any couples like this where both made historically-significant contributions to different projects.

[-]David Hornbein4y40

Oh also, woman politicians in living memory seem much less likely to have kids than woman politicians "from history". I would guess this is a consequence of the shift away from explicitly hereditary political power rather than a consequence of feminism or the pill or anything, but it's hard to untangle different hypotheses because there were so few woman politicians between the advent of industrialization and recent rise from ~1970-1990.

[-]habryka4y100

Yep, in general about 85% of people have kids, with something like half of the people who don't not doing so because they have fertility problems, or other things that tend to classify them as being "involuntarily childless" in a bunch of studies. So the population to study here (people who voluntarily don't have children) have historically only made up something like 7% of the population. So just looking through successful lists of people and seeing that most of them have kids isn't really going to provide a ton of evidence.

[-]Ben Pace4y50

[-]Owain_Evans4y90

Ability to succeed in building organizations or movements will correlate with ability to organize childcare (either through family, friends or paid help).

[-]remizidae4y50

You may be overestimating the amount of time and effort rich parents (especially rich fathers) put towards raising kids.

[-]Quinn3y20

Borlaug was a super absentee parent, his wife did everything herself and he (presumably) sent back cash while globetrotting. How many of these ambitious people with kids aren't super involved in their kids' lives?

[-]Vanessa Kosoy4y240

Not sure whether this is the right place to leave this comment, but:

I feel that having children increases my motivation to work on X-risk. Because, even if the timeline is s.t. it won't affect me personally, it can still affect my children (and is even more likely to affect my grandchildren if I will have any). That said, it definitely reduces the flexibility of e.g. giving up on income or moving to another country.

[-]Zack_M_Davis4y190

Two complementary pro-natalist considerations I'd like to see discussed:

Eugenics! It doesn't seem like there are any technical barriers to embryo selection for IQ today. If longtermist parents disproportionately become early adopters of this tech in the 2020s, could that help their children be a disproportionate share of up-and-coming AI researchers in the 2040s?
Escaping our Society's memetic collapse. We are the children of a memetic brood-parasite strategy. It's a lot easier to recruit new longtermists out of universal culture than it is from Mormonism, but universal culture triumphed not because its adherents had more children than everyone else, but by capturing the school and media institutions that socialize everyone else's children: horizontal meme transmission rather than vertical. If social-media-era universal culture is no longer as conducive to Reason as its 20th-century strain, maybe we need to switch to a more Mormon-like strategy (homeschooling, &c.) if we want there to be top reasoners in the 2040s.

[-]Ajeya Cotra4y120

Belatedly, I did a bit of outside-view research on the time and monetary costs of kids (though a couple parent friends kindly sanity-checked some of it). I presented it at my house's internal conference, but some folks suggested I share more broadly in case it's helpful to others: here is the slide deck. The assumptions are Bay Area, upper-middle-class parents (e.g. both programmers or something like that) who both want to keep their careers and are therefore willing to pay a lot for childcare.

[-]Adele Lopez4y110

FWIW: a lot of my motivation (in general) stems from my love of my future children.

[-]Ben Pace4y100

This was perhaps my favorite weekend event that we’ve done, both for the main event and the many hours of conversation afterward.

I’ll see if I can get the time to write up some of the thoughts and ideas I got from the discussion and put it into a post.

[-]Joe Collman4y10

I hope you find the time. I hadn't realised this was happening and would be interested in any thoughts and ideas. It's an issue that's high impact, broadly relevant, hard to get good data and easy to reason poorly about - so I'm glad to see some thoughtful discussion.

[-]lincolnquirk4y90

Is there a recording/transcript/anything to watch after-the-fact?

[-]Raemon4y20

There is not – it was a kinda delicate conversation and we decided it was better for Oli and Anna and everyone else to feel free to speak out loud. I also think much of the value was sort of a meandering sussing out of fuzzy positions that's just actually kinda hard to get if you weren't there at the time.

That said, one participant said they might write up some followup thoughts, that serve as something of a distillation.

[-]lincolnquirk4y50

Oh darn :( I was really looking forward to this event but ended up being busy then. One of the benefits of doing stuff like this on Zoom is to engage a broader community -- why stop at only people who can be there at the appointed time?

Would it have caused you to record it if I had asked nicely ahead of time? (I saw the announcement and knew I would likely be busy then. I could have thought to ask.)

[-]Ben Pace4y50

That sort of thing does encourage us to record the event, but I think we still wouldn't have recorded the 3+ hours of discussion afterwards, some of which leaned into being more personal or private. I got a lot out of that convo.

[-]TurnTrout4y30

I am now very sad that I assumed it would be recorded, for some unknown reason. Here’s to hoping for some distillation, at least!

[-]AnnaSalamon4y60

Acquiring/sharing data sounds good, but I am not personally planning to look at it before Sunday. Maybe someone else wishes to do this, and to bring it to the discussion? Relatedly, there are other good points here that I probably won't manage to personally go into -- maybe Oliver does, but if not, maybe we can have a discussion format that allows other people to also give short spiels/etc., and maybe if some are motivated to share such they can come with these? I am not sure what Ray and Oliver have planned.

[-]Raemon4y40

Overall my guess was "Anna and Oliver chat 1-on-1 for an hour, and then we open it up to other questions, without too much structure", but I also didn't have a strong sense of what would work best and kinda defer to whatever Anna and Oliver are excited by.

(In the world where neither Anna nor Oliver have a strong opinion, the default is "Hour of discussion after which Ray moderates some Q&A for awhile using his judgment, including making a call about whether it should remain one giant Zoom conversation or moving over to a Walled Garden afterparty where people can break into multiple groups")

[-]steven04614y40

guesses: 1. in most cases, children on net detract from other major projects for common-sense time/attention/optionality management reasons (as well as because they sometimes commit people to a world view of relatively slow change), 2. whether to have children isn't each other's business and pressure against doing normal human things like this is net socially harmful (conservatives in particular are alienated by a culture of childlessness, though maybe that's net strategically useful), 3. people conflate 2 with not-1 on an emotional level and feel 1 is false because 2 is true

[-]Dirichlet-to-Neumann4y20

Depending on your evaluation of AI timelines, children of AI-conscious parents have a much higher chance of being AI-conscious, so consider having kids a long-term investment in AI research...

[-]Stephen McAleese8mo10

I suspect the desire for kids/lineage is really basic for a lot of people (almost everyone?)

This seems like an important point. One of the arguments for the inner alignment problem is that evolution intended to select humans for inclusive genetic fitness (IGF) but humans were instead motivated by other goals (e.g. seeking sex) that were strongly correlated with IGF in the ancestral environment.

Then when humans' environment changed (e.g. the invention of birth control), the correlation between these proxy goals and IGF broke down resulting in low fitness and inner misalignment.

However this statement seems to suggest that modern humans really have internalized IGF as one of their primary objectives and that they're inner aligned with evolution's outer objective.

[-]leogao8mo64

Wanting to raise kids/have what would normally be considered a lineage is importantly different from IGF; most people would not consider sperm bank donation to satisfy their child-having goals very well despite this being very good for IGF.

Moderation Log