automation et.al aka AI scenarios and populations per se: less is more. We don't need or require 7billion+. the only reason this ideology of more ppl is for capitalism to create more docile consumers for rapacious share holders nothing else. capital should be invested in science of course, space exploration big time, build underwater cities for fun and get rid of planet-city destroying cars: eg Australia. it might be urbanized but really us sub-urbanized. an eco nightmare, disturbia through wrong technology [the car] deciding how ppl live. This has to stop even if we get down to 1 billion humans [one can dream]. However if AI really takes off and goes beyond the Event Horizon [false known as singularity whatever that is] than all ISMS are defunct be it socialism/capitalism/mercantilism/religionism hopefully they all vanish as humans don't work anymore on Earth. Let the AI run riot. There is always an off button.

I think the -intelligence- in the -artificial- is overrated. It doesn't need anything per se to control it. All these scenario's pretend that -it- has volition which it has not. As long as that is the case all this -re:above- is what the Germans call Spiegelfechterei- fencing with a mirror image. Esp social integration. That is so Neanderthal. I rest my case. And my mind.

being psychotic is not some alpha out of control testosterone malfunction per se. You see I have been diagnosed as [among other madness-es] psychotic. I have an innate hatred of all humans [though when meeting them it's cool] but still I'd rather disengage than engage. An AI with intelligence could easily deduce that most humans are morons and a waste of space. Social media forums proves this abundantly by the vapid vomit espoused in such multitudinal mind numbing stupidity. Real intelligence is rare so perhaps an AI will be as stupid as their creators. From what I gathered a lot of logic and maths is involved which is intelligent-specific which is of a different order to eastern definition of comprehensive if not cosmic wisdom. So perhaps we have nothing to fear except stupidity itself.

Amerikans seem to think AIs will morph into DALEKS. Perhaps they will. We don't know this though. At the moment machine-intelligences are stupid and that is the way best to work with them. The late great Arthur C Clarke had 3 futures re AIs 1] we rule 2] they rule but keep us for interests' sake. We become the samples, the experiment, the lab-rats. 3] they rule and realize we are a pest and this is the end of the human race 4] DALEKS win [my contribution]

obviously AIs in the future haven't gotten in to time travel [yet] as we are still here building the systems that will doom us. This has been a prevalent Amerikan view. The Japanese are embracing AIs and robots. The TV series -and movie spin offs - Ghost in the Shell, Stand alone Complex explored the future of us being integrated with machine intelligences beautifully and it really is a romantic possibility. All depends on the intelligence - stupidity of the programmers.

Maybe it is time to explore this and reincarnate on another advanced [relative to Earth] planet and study the results there in situ.

Self improvement: oh dear we are sinking into pop psych 101 here. Something with some sub-urbanites that resonates in their mental echo chamber. Add technology and the cargo cult mentality comes in: problems solved of whatever. Not a solid solution. As for cognitive enhancement the European led the way with primarily the secular mind and philosophers from the French and the Germans complementing each other. The Brits were basically logicians which proves nothing. But they did start the Industrial Revolution without self improvement or self help books or self help life gurus. That the Europeans achieved by independent thinking so that today's aides won't work. During the shamanistic stage of human development psychtoropic drugs boosted humanity into the cosmic city state. Quite an achievement with mental achievement. Yet it was the beer sodden wine sodden northern Europe that defined our current society. The great mental release was to go secular. Write divinity and it's mental ordure out of the holistic equation. And today other paradigms are needed: Philip K Dick et.al created possibilities that really did, along with Arthur C Clarke push the cosmological horizon into near infinity. In-sanity the next mindful step.

In practice do you expect a system's values to change with its intelligence?

An AI that escapes is truly compelling. Given the hopeless mess humans create we are not the best of mentally applied solution solvers. Too many people. Too many outdated technologies such as cars & ango-amerikan suburbs. Outdated social ideas such as capitalism, socialism & religious manias. Maybe it will take an AI to lead us to equitable solutions globally applied. Arthur Clarke viewed it this way. And we must stop breeding for a few decades. All this smart thinking is really self proscribed [even if ironically in a global network].

The Manhattan Project [Tubes] managed to remain secret even though scientists and researchers themselves thrive on the open community which in turn feeds new ideas through innovation and response to continue the scientific dialectic. A spy in the works [Fuchs] saved years re the Soviets to get it's bomb together. Then there was the Soviet sputnik which too surprised esp Amerika. And went paranoid because they thought in cold-war terms not scientific ones. Not at first anyway. Now AI research and breakthroughs-spyware aside-can keep researchers autonomous. Computers are shrinking. What took electro-shock-rock bands a full room of hardware to synthesize their sound is now done in a tiny nifty box top model the size of a thick book. Research will be done in secret [spyware aside]. After all terrorists nearly get away with their secret schemes so devoted scientists can and will do the same in the interest of glory which they fully deserve. We will be in for some surprises I hope

The problem is is that superintelligence has been re-defined. Under current models it aint gonna happen no matter how sexy the presentation and thoughts inputted. Eating potatoes or whatever doesn't make you smarter. Take all the German scientists back end of 19th-20th century. Stodgy European food. And it worked. They delivered. Now Asians have much better cuisine but they are now catching up and may yet end up as the best is yet to come. Drugs are good. Esp Opium. Cognitive enhancement big time. Yet again those cultures that enjoyed it staid within their cultural milieu for ages. Again things have changed but that option should be made available to all everywhere. The rest is gimmicky. Interesting in itself as itself. Take whole brain emulation. I mean that is what babies are. Re-creating brains and or designing wetware is silly. Plus the brain so emulated may have ideas of its own and turn DALEK on us. The essence of intelligence is resonant sentience. Meditation techniques esp Ch'an-Zen [and related] Buddhist practices will give results but this takes years and even then no guarantee. But this self enhancement seems more logical than a lot of machines thinking they think by merely puzzling out solutions that are in essence given anyway. The Mind. That is the centre of everything. Molecular enhancement will for now be the best bet.

I hate to spoil the party but the author has redefined superintelligence. whilst the possibilities are there to go further deeper and broader in scope real superintelligence is raising the paradigms and boundary thresholds of current intelligence. To be super is to be in another level of cognitively boosted consciousness. If the mind is in toto resonating at level 1 then superintelligence has to resonate above that. The closest humanity has come to that is external and wrapped in ancient Hindu mythologies of their gods and godesses. Anything less is simply where we are now.