Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: tygorton 19 May 2012 09:02:34AM *  1 point [-]

This logic assumes that a beyond human intelligence in a redesigned world would still find inherent value in free will. Isn't it possible that such an intelligence would move beyond the need to experience pain in order to comprehend the value of pleasure?

According to the bible, god created different aspects of the world across six days and after each creation he "saw that it was good". Yet nothing ELSE existed. If there had never been a "world" before, and evil had not yet been unleashed, by what method was this god able to measure that his creation was good? One must assume that god's superior intelligence simply KNEW it to be good and had no need to measure it against something "bad" in order to know it. Couldn't the eventual result of AI be the attainment of the same ability... the ability to KNOW pleasure without the existence of its opposite?

Isn't the hope (or should I say fun?) of considering the potential of AI that such a vast intelligence would move life BEYOND the anchors to which we now find ourselves locked? If AI is simply going to be filled with the same needs and methods of measuring "happiness" as we currently deal with, what is the point of hoping for it at all?

This is a bit of an afterthought, but even at our current level of intelligence, humans have no way of knowing if we would value pleasure if pain did not exist. Pain does now and has always existed. "Evil" (or what we perceive as evil) has existed since the dawn of recorded human existence. How can we assume that we are not already capable of recognizing pleasure as pleasure and good as good without their opposites to compare them to? We have never had the opportunity to try.

Comment author: thespianic 21 September 2012 02:57:29PM 0 points [-]

I beg to differ on the aspect of there being non-existence predating the creation. A subtle nuance in the first verse of Genesis offers an insight into this. Gen 1:1 "In the beginning God created the heavens and the earth. And the earth was without form, and void; and darkness was upon the face of the deep." Original manuscripts offer a translation that is closer to "and the earth 'became' without form (sic), and void". It may so very well be that in the assumption that God looked on his creation and saw that it was good, there was a pre-existential basis for this. Also to point out another simple example, there would be no record of wrong without a sort of legal system that says that an act is actually defined as wrong. I agree with the idea that there had to be an apple in the garden to bring to the front the difference between good and bad. Utopia can therefore only exists where there is an understanding or mere knowledge of dystopia.