In this video, Julian Savulescu from the Uehiro centre for Practical Ethics argues that human beings are "Unfit for the future" - that radical technological advance, liberal democracy and human nature will combine to make the 21st century the century of global catastropes, perpetrated by terrorists and psychopaths, with tools such as engineered viruses. He goes on to argue that enhanced intelligence and a reduced urge to violence and defection in large commons problems could be achieved using science, and may be a way out for humanity.
Skip to 1:30 to avoid the tedious introduction
Genetically enhance humanity or face extinction - PART 1 from Ethics of the New Biosciences on Vimeo.
Genetically enhance humanity or face extinction - PART 2 from Ethics of the New Biosciences on Vimeo.
Well, I have already said something rather like this. Perhaps this really is a good idea, more important, even, than coding a friendly AI? AI timelines where super-smart AI doesn't get invented until 2060+ would leave enough room for human intelligence enhancement to happen and have an effect. When I collected some SIAI volunteers' opinions on this, most thought that there was a very significant chance that super-smart AI will arrive sooner than that, though.
A large portion of the video consists of pointing out the very strong scientific case that our behavior is a result of the way our brains are structured, and that this means that changes in our behavior are the result of changes in the way our brains are wired.
Re: "Optimal by what measure? By the measure of maximizes the sort of things I value?"
No!
The basic idea is that some moral systems are better than other - in nature's eyes. I.e. they are more likely to exist in the universe. Invoking nature as arbitrator will probably not please those who think that nature favours the immoral - but they should at least agree that nature provides a yardstick with which to measure moral systems.
I don't have access to the details of which moral systems nature favours. If I did - and had a convincing supporting argument - there would probably be fewer debates about morality. However, the moral systems we have seen on the planet so far certainly seem to be pertinent evidence.
Measured by this standard, moral progress cannot fail to occur. In any case, that's a measure of progress quite orthogonal to what I value, and so of course gives me no reason to celebrate moral progress.