All of GeneSmith's Comments + Replies

Care to explain how you think it's being misused?

I find this argument fairly compelling. I also appreciate the fact that you've listed out some ways it could be wrong.

Your argument matches fairly closely with my own views as to why we exist, namely that we are computationally irreducible

It's hard to know what to do with such a conclusion. On the one hand it's somewhat comforting because it suggests even if we fuck up, there are other simulations or base realities out there that will continue. On the other hand, the thought that our universe will be terminated once sufficient data has been gathered is pretty sad.

DE-FACTO UPLOADING

Imagine for a moment you have a powerful AI that is aligned with your particular interests.

In areas where the AI is uncertain of your wants, it may query you as to your preferences in a given situation. But these queries will be "expensive" in the sense that you are a meat computer that runs slowly, and making copies of you is difficult.

So in order to carry out your interests at any kind of scale with speed, it will need to develop an increasingly robust model of your preferences.

Human values are context-dependent (see shard theory and ot... (read more)

I'm not sure I buy that they will be more cautious in the context of an "arms race" with a foreign power. The Soviet Union took a lot of risks their bioweapons program during the cold war.

My impression is the CCP's number one objective is preserving their own power over China. If they think creating ASI will help them with that, I fully expect them to pursue it (and in fact to make it their number one objective)

So in theory I think we could probably validate IQ scores of up to 150-170 at most. I had a conversation with the guys from Riot IQ and they think that with larger sample sizes the tests can probably extrapolate out that far.

We do have at least one example of a guy with a height +7 standard deviations above the mean actually showing up as a really extreme outlier due to additive genetic effects.

The outlier here is Shawn Bradley, a former NBA player. Study here

Granted, Shawn Bradley was chosen for this study because he is a very tall person who does not suf... (read more)

I'm being gaslit so hard right now

One data point that's highly relevant to this conversation is that, at least in Europe, intelligence has undergone quite significant selection in just the last 9000 years. As measured in a modern environment, average IQ went from ~70 to ~100 over that time period (the Y axis here is standard deviations on a polygenic score for IQ)

The above graph is from David Reich's paper

I don't have time to read the book "Innate", so please let me know if there are compelling arguments I am missing, but based on what I know the "IQ-increasing variants have been exhausted... (read more)

Sorry, I've been meaning to make an update on this for weeks now. We're going to open source all the code we used to generate these graphs and do a full write-up of our methodology.

Kman can comment on some of the more intricate details of our methodology (he's the one responsible for the graphs), but for now I'll just say that there are aspects of direct vs indirect effects that we still don't understand as well as we would like. In particular there are a few papers showing a negative correlation between direct and indirect effects in a way that is distinc... (read more)

I've been talking to people about this today. I've heard from two separate sources that it's not actually buyable right now, though I haven't yet gotten a straight answer as to why not.

Congrats on the new company! I think this is potentially quite an important effort, so I hope anyone browing this forum who has a way to get access to biobank data from various sources will reach out. 

One of my greatest hopes for these new models is that they will provide a way for us to predict the effects of novel genetic variants on traits like intelligence or disease risk.

When I wrote my post on adult intelligence enhancement at the end of 2023, one of the biggest issues was just how many edits we would need to make to achieve significant change i... (read more)

I’ve checked. Have heard from multiple people they “it’s not for sale in reality”

I don’t have any details yet. But obviously am interested.

I certainly hope we can do this one day. The biobanks that gather data used to make the predictors we used to identify variants for editing don't really focus on much besides disease. As a result, our predictors for personality and interpersonal behavior aren't yet very good.

I think as the popularity of embryo selection continues to increase, this kind of data will be gathered in exponentially increasing volumes, at which point we could start to think about editing or selecting for the kinds of traits you're describing.

There will be an additional question ... (read more)

In the last year it has really hit me at a personal level what graphs like these mean. I'm imagining driving down to Mountain View and a town once filled with people who had "made it" and seeing a ghost town. No more jobs, no more prestige, no more promise of a stable life. As the returns to capital grow exponentially and the returns to labor decline to zero, the gap between the haves and the have-nots will only grow.

If someone can actually get superintelligence to do what they want, then perhaps universal basic income can at the very least prevent actual ... (read more)

I spoke with one of the inventors of bridge recombinases at a dinner a few months ago and (at least according to him), they work in human cells.

I haven't verified this independently in my lab, but it's at least one data point.

On a broader note, I find the whole field of gene therapy very confusing. In many cases it seems like there are exceptionally powerful tools that are being ignored in favor of sloppy, dangerous, imprecise alternatives.

Why are we still using lentiviral vectors to insert working copies of genes when we can usually just fix the broken ge... (read more)

I mean... I think adult gene therapy is great! It can cure diseases and provide treatments that are otherwise impossible. So I think it's more impactful than heated seats.

So I'm obviously talking my own book here but my personal view is that one of the more neglected ways to potentially reduce x-risk is to make humans more capable of handling both technical and governance challenges associated with new technology.

There are a huge number of people who implicitly believe this, but almost all effort goes into things like educational initiatives or the formation of new companies to tackle specific problems. Some of these work pretty well, but the power of such initiatives is pretty small compared to what you could feasibly achi... (read more)

Cool! Are you working for an existing company or are you starting your own?

There is some overlap with adult enhancement. Specifically, if we could make a large number of changes to the genome with a single transfection, that would be quite helpful.

Population mean

I’ve seen this and will reply in the next couple of days. I want to give it the full proper response it deserves.

Also thanks for taking the time to write this. I don’t think I would get this level or quality of feedback anywhere else online outside of an academic journal.

I think superbabies would still have a massive positive impact on the world even if all we do is decrease disease risk and improve intelligence. But with this kind of thing I think the impact could be very robustly positive to an almost ridiculous degree.

My hope is as we scale operations and do more fundraising we can fund this kind of research.

It's possible I'm misunderstanding your comment, so please correct me if I am, but there's no reason you couldn't do superbabies at scale even if you care about alignment. In fact, the more capable people we have the better.

I'm having trouble understanding how concretely you think superbabies can lead to significantly improved chance of helping alignment.

Kman may have his own views, but my take is pretty simple; there are a lot of very technically challenging problems in the field of alignment and it seems likely smarter humans would have a much higher chance of solving them.

https://x.com/GeneSmi96946389/status/1892721828625264928

First of all, no one has really done large scale genetic engineering of animals before, so we wouldn't know.

Almost all mouse studies or genetic studies in other animals are very simple knockout experiments where they break a protein to try to assess its function.

We really haven't seen a lot of multiplex editing experiments in animals yet.

But even if someone were to do that it would be hard to evaluate the effects on intelligence in animals.

The genetic variants that control IQ in humans don't always have analogous sequences in animals. So you'd be working w... (read more)

Well we have it in cows. Just not in mice.

I think many people in academia more or less share your viewpoint.

Obviously genetic engineering does add SOME additional risk of people coming to see human children like commodities, but in my view it's massively outweighed by the potential benefits.

you end up with a child whose purpose is to fulfill the parameters of their human designers

I think whether or not people (and especially parents) view their children this way depends much more on cultural values and much less on technology.

There are already some parents who have very specific goals in mind ... (read more)

I would love to try this in mice.

Unfortunately our genetic predictors for mice are terrible. The way mouse research works is not at all like how one would want it to work if we planned to actually use them as a testbed for the efficacy of genetic engineering.

Mice are mostly clones. So we don't have the kind of massive GWAS datasets on which genes are doing what and how large the effect sizes are.

Instead we have a few hundred studies mostly on the effects of gene knockouts to determine the function of particular proteins.

But we're mostly not interested in k... (read more)

Yes, I pretty much agree with this

I'm not saying his experiments show germline editing is safe in humans. In fact He Jiankui's technique likely WASN'T safe. Based on some talks I heard from Dieter Egli at Colombia, He was likely deleting chromosomes in a lot of embryos, which is why (if I recall correctly) only 3 out of about ~30 embryos that were transferred resulted in live birth. Normally the live birth rate per transfer rate would be between 30 and 70%.

It's also not entirely clear how effective the editing was because the technique He used likely created a fair degree of mosaicism sinc... (read more)

I'd be interested in hearing where specifically you think we are doing that.

Yes, the two other approaches not really talked about in this thread that could also lead to superbabies are iterated meiotic selection and genome synthesis.

Both have advantages over editing (you don't need to have such precise knowledge of causal alleles with iterated meiotic selection or with genome synthesis), but my impression is they're both further off than an editing approach.

I'd like to write more about both in the future.

It's just very hard for me to believe there aren't huge gains possible from genetic engineering. It goes against everything we've seen from a millenia of animal breeding. It goes against the estimates we have for the fraction of variance that's linear for all these highly polygenic traits. It goes against data we've seen from statisitcal outliers like Shawn Bradley, who shows up as a 4.6 standard deviation outlier in graphs of height:

Do I buy that things will get noisier around the tails, and that we might not be able to push very far outside the +5 SD mar... (read more)

There is one saving grace for us which is that the predictor we used is significantly less powerful than ones we know to exist.

I think when you account for both the squaring issue, the indirect effect things, and the more powerful predictors, they're going to roughly cancel out.

Granted, the more powerful predictor itself isn't published, so we can't rigorously evaluate it either which isn't ideal. I think the way to deal with this is to show a few lines: one for the "current publicly available GWAS", one showing a rough estimate of the gain using the priva... (read more)

Ha, sadly it is a pseudonym. My parents were neither that lucky nor that prescient when it came to naming me.

A brief summary of the current state of the "making eggs from stem cells" field:

• We've done it in mice
• We have done parts of it in humans, but not all of it
• The main demand for eggs is from women who want to have kids but can't produce them naturally (usually because they're too old but sometimes because they have a medical issue). Nobody is taking the warning to not "Build A Method For Simulating Ovary Tissue Outside The Body To Harvest Eggs And Grow Clone Workers On Demand In Jars" because no one is planning on doing that.

Even if you could make eggs f... (read more)

Thanks for catching that! I hadn't heard. I will probably have to rewrite that section of the post.

What's your impression about the general finding about many autoimmune variants increasing protection against ancient plauges?

No, the problem really is technical right now.

There may be additional societal and political problems afterwards. But none of those problems actually matter unless the technology works.

Obviously we are going to do it in animals first. We have in fact DONE gene editing in animals many times (especially mice, but also some minor stuff in cows and other livestock). But you're correct that we need to test massive multiplex editing. My hope is we can have good data on this in cows in the next 1-3 years.

I don't understand your question

Agreed, though unfortunately it's going to take a while to make this tech available to everyone.

Also, if you want to prevent your children from getting hypertension, you can already do embryo selection right now! The reduction isn't always as large as what you can get for gene editing, but it's still noticeable. And it stacks generation after generation; your kids can use embryo selection to lower THEIR children's disease risk even more.

Kman and I probably differ somewhat here. I think it's >90% likely that if we continue along the current trajectory we'll get AGI before the superbabies grow up.

This technology only starts to become really important if there's some kind of big AI disaster or a war that takes down most of the world's chip fabs. I think that's more likely than people are giving it credit for and if it happens this will become the most important technology in the world.

Gene editing research is much less centralized than chip manufacturing. Basically all of the research can... (read more)

Yes you're right. With current technology there's no way you could get anywhere close to 500 embryos. I know a couple trying to get 100 and even that seems crazy to me.

5-20 is more realistic for most people (and 5 is actually quite good if you have fertility issues).

But we wanted to show 500 edits to compare scaling of gene editing and embryo selection and there wasn't any easy way to do that without extending the graph for embryo selection.

Currently, we have smart people who are using their intelligence mainly to push capabilities. If we want to grow superbabies into humans that aren't just using their intelligence to push capabilities, it would be worth looking at which kind of personality traits might select for actually working on alignment in a productive fashion.

I think we need to think more broadly than this. There's some set of human traits, which is a combination of the following:

• Able to distinguish prosocial from antisocial things
• Willing and able to take abstract ideas serious

I'm glad you liked the article!

Brain size is correlated with intelligence at maybe 0.3-0.4. If you were to just brain size max I think it would probably not yield the outcomes you actually want. It's better to optimize as directly as you can for the outcome you want.

I think almost everyone misunderstands the level of knowledge we have about what genetic variants will do.

Nature has literally run a randomized control trial for genes already. Every time two siblings are created, the set of genes they inherit from each parent are scrambled and (more or less) randomly assigned to each. That's INCREDIBLY powerful for assessing the effects of genes on life outcomes. Nature has run a literal multi-generational randomized control trial for the effect of genes on everything. We just need to collect the data.

This gives you a hug... (read more)

Agreed. I've actually had a post in draft for a couple of years that discusses some of the paralleles between alignment of AI agents and alignment of genetically engineered humans.

I think we have a huge advantage with humans simply because there isn't the same potential for runaway self-improvement. But in the long term (multiple generations), it would be a concern.

Do you have any estimate of how much more expensive testing in cynomolgus macaques or rhesus monkeys would be?

The issue is that it takes a long time for PGC-like cells to develop to eggs, if you're strictly following the natural developmental trajectory.

Thanks for the clarification. I'll amend the original post.

It's a fair concern. But the problem of predicting personality can be solved! We just need more data.

I also worry somewhat about brilliant psychopaths. But making your child a psychopath is not necessarily going to give them an advantage.

Also can you imagine how unpleasant raising a psychopath would be? I don't think many parents would willingly sign up for that.

Very little at the moment. Unlike intelligence and health, a lot of the variance in personality traits seems to be the result of combinations of genes rather than purely additive effects.

This is one of the few areas where AI could potentially make a big difference. You need more complex models to figure out the relationship between genes and personality.

But the actual limiting factor right now is not model complexity, but rather data. Even if you have more complex models, I don't think you're going to be able to actually train them until you have a lot mor... (read more)

It's a good question. The remarkable thing about human genetics is that most of the variants ARE additive.

This sounds overly simplistic, like it couldn't possible work, but it's one of the most widely replicated results in the field.

There ARE some exceptions. Personality traits seem to be mostly the result of gene-gene interactions, which is one reason why SNP heritability (additive variance explained by common variants) is so low.

But for nearly all diseases and for many other traits like height and intelligence, ~80% of variance is additive. somewhere bet... (read more)