Sure. One thing I might mention to someone with that utility function is that if humanity gets destroyed by an enhanced psychopath, that will probably happen right around the same time that enhanced scientists would be working to speed technological progress. So even someone with a relatively myopic utility function will in many cases favor caution.
I get the idea that FAI takes more intelligence than AGI, as AGI might be able to be brute-forced by reverse-engineering the brain or evolutionary approaches, whereas de novo AI is far harder, let alone AGI. This would mean that increasing intelligence would make the world safer. I don't see why enhanced psychopaths are more likely than enhanced empaths.
If you're the kind of person who gives up when faced with hard challenges, that's fine; I guess we're just different in that way.
No, I'm certainly not, however I am realistic and I do prioritise. I don't think the risk from genetic enhancement is all that great, and indeed it may be a net positive.
Anyway, so I think that mandatory enhancement is not going to be popular. However, other ideas do seem more plausible:
One way to prepare might be differential technological development. In particular, maybe it's possible to decrease the cost of gene editing/selection technologies while retarding advances in our knowledge of which genes contribute to intelligence.
So, this is a reasonable idea. Governments could prioritise research into stopping diseases above increasing intelligence, and indeed this is likely to be the case anyway, as this is less controversial. Increasing compassion or even docility could also be prioritised above increasing intelligence.
extend the benefits of designer babies to everyone for free regardless of their social class.
This is also a good idea. It seems inevitable that some of the rich will be early adopters before the technology is cheap enough to be made free to all. However, the cost of sequencing has been going down 5x per year, meaning that it is likely to quickly become widely available.
Overall, I would say the best strategy seems to be to take a more libertarian than authoritarian approach, but try to funnel money into researching the genetics of various antisocial personality disorders, try to make the technology free, and either don't patient the genes or ensure that the patients don't last that long.
I think sequencing is what lets you measure genes, not modify them.
I previously wrote a post hypothesizing that inter-group conflict is more common when most humans belong to readily identifiable, discrete factions.
This seems relevant to the recent human gene editing advance. Full human gene editing capability probably won't come soon, but this got me thinking anyway. Consider the following two scenarios:
1. Designer babies become socially acceptable and widespread some time in the near future. Because our knowledge of the human genome is still maturing, they initially aren't that much different than regular humans. As our knowledge matures, they get better and better. Fortunately, there's a large population of "semi-enhanced" humans from the early days of designer babies to keep the peace between the "fully enhanced" and "not at all enhanced" factions.
2. Designer babies are considered socially unacceptable in many parts of the world. Meanwhile, the technology needed to produce them continues to advance. At a certain point people start having them anyway. By this point the technology has advanced to the point where designer babies clearly outclass regular babies at everything, and there's a schism between "fully enhanced" and "not at all enhanced" humans.
Of course, there's another scenario where designer babies just never become widespread. But that seems like an unstable equilibrium given the 100+ sovereign countries in the world, each with their own set of laws, and the desire of parents everywhere to give birth to the best kids possible.
We already see tons of drama related to the current inequalities between individuals, especially inequality that's allegedly genetic in origin. Designer babies might shape up to be the greatest internet flame war of this century. This flame war could spill over in to real world violence. But since one of the parties has not arrived to the flame war yet, maybe we can prepare.
One way to prepare might be differential technological development. In particular, maybe it's possible to decrease the cost of gene editing/selection technologies while retarding advances in our knowledge of which genes contribute to intelligence. This could allow designer baby technology to become socially acceptable and widespread before "fully enhanced" humans were possible. Just as with emulations, a slow societal transition seems preferable to a fast one.
Other ideas (edit: speculative!): extend the benefits of designer babies to everyone for free regardless of their social class. Push for mandatory birth control technology so unwanted and therefore unenhanced babies are no longer a thing. (Imagine how lousy it would be to be born as an unwanted child in a world where everyone was enhanced except you.) Require designer babies to possess genes for compassion, benevolence, and reflectiveness by law, and try to discover those genes before we discover genes for intelligence. (Edit: leaning towards reflectiveness being the most important of these.) (Researching the genetic basis of psychopathy to prevent enhanced psychopaths also seems like a good idea... although I guess this would also create the knowledge necessary to deliberately create psychopaths?) Regulate the modification of genes like height if game theory suggests allowing arbitrary modifications to them would be a bad idea.
I don't know very much about the details of these technologies, and I'm open to radically revising my views if I'm missing something important. Please tell me if there's anything I got wrong in the comments.