@Recovering irrationalist, good points, thank you - I just wanted to save time and space by linking to relevant stuff on my blog without repeating myself over and over. My apologies for overdoing it. I guess I feel like talking to a wall or being deliberately ignored due to the lack of feedback. I shall curb my enthusiasm and let things take its course. You know where to find me.

@J Thomas, the trick lies in ensuring continued co-existence.

@TGGP: This forum really is not the right place to get into details. It would not be fair towards Eliezer and that I posted something at all is an embarrassing revelation in regards to my intellectual vanity. Mea culpa.

@Tiiba, trust me - I am quite certain that I do, but this is not the right forum - PM me if you want to continue off this blog.

logicnazi, he is making progress ;-)

Humans certainly aren't perfect at imagining. In fact if you ask most people to imagine a heavy object and a much heavier object falling they will predict the much heavier object hits first and I can give a host of other examples of the same thing.

When you ask someone to imagine something he is controlling his imagination which is equivalent to conscious though. What one can think of however is controlled by ones beliefs - what is skewed in humans is their beliefs, not their imagination. Once beliefs are being controlled by science one will imagine the falling consistent with the scientific theory forming ones beliefs. I would reword your sentence to: humans not usually form their beliefs on the basis of science.

Caledonian, sorry - do you mean that humanity needs to be superseded?

Gray Area, did you read my paper on friendly AI yet? I must be sounding like a broken record by now ;-)

I justify my statement 'that is good what increases fitness' with the axiomatic belief of 'to exist is preferable over not to exist'

The phenomena created by evolution that seem like horrors to us (parasitic wasps) must be that particular wasp's pinnacle of joy. It is a matter of perspective. I am not saying: eat crap - millions of flies can't be wrong! I am taking the human perspective - not that of a wasp or a fly or of random horror inducing entity but can understand other entities points of view and see the general principle - what increases fitness of an entity is good for that entity. Generally put: that is good what increases fitness.

Caledonian, yes - I agree 100% - the tricky part is getting to post humanity - avoiding a non-friendly AI. That would be a future where we have a place in the sense that we will have evolved further.

gutzperson, today you are gutzperson - tomorrow you are post-gutzperson yesterday - ensuring your continued existence in that sense will lead to your eventual transcendence. Same for everyone else - just don't extinguish that strand.

Aaron Luchko, I argue that morality can be universally defined. You can find my thoughts in my paper on friendly AI theory? Would love to hear your comments.

Somehow the links in my earlier comment got messed up.

For the link behind 'cognitive evolution' see: http://www.jame5.com/?p=23 For the link behind 'make sure we will have a place' see: http://www.jame5.com/?p=17

gutzperson: good points - it is all about increasing fitness and social control. You will find reading the following paper quite interesting: Selection of Organization at the Social level: obstacles and facilitators of metasystem transitions. Particularly chapter four: Social Control Mechanisms.

Evolution does not stop on the genetic level but continues on the cognitive level allowing for a far higher complexity and speed. As a result group selection becomes intuitively obvious although on the cognitive level members of weaker groups have of cause in principle the chance to change their minds aka evolve their beliefs before physical annihilation.

"If we can't see clearly the result of a single monotone optimization criterion"

We can project where ever increasing fitness leads up to and it is up to us to make sure we will have a place in such a future.