I don't think that an AI that goes FOOM would be exactly the same as any of the "Gods" humanity has been envisioning and may not even resemble such a God (especially because, if it were a success, it would theoretically not behave in self-contradictory ways like making sinful people, knowing exactly what they're going to do, making them to do just that, telling them not to act like what they are and then punishing them for behaving the way it designed them to). I don't see a reason to believe that it is possible for any intellect to be omniscient, omnipotent or perfect. That includes an AI. These, to me, would be the main differences.
Robert Wright appears to be aware of this, as his specific wording was "It seems to me that in some sense what you're saying is that we need to build a God."
If you are taking this as a question about what to CALL the thing, then I agree completely that the AI should not be called a God. But he said "in some sense" which means that his question is about something deeper than choosing a word. The wording he's using is asking something more like "Do you think we should build something similar to a God?"
The way that I interpret this question is not "What do we call this thing?" but more "You think we should build a WHAT?" with the connotations of "What are you thinking?" because the salient thing is that building something even remotely similar to a God would be very, very dangerous.
The reason I interpreted it this way is partly because instead of interpreting everything I hear literally, I will often interpret wording based on what's salient about it in the context of the situation. For instance, if I saw a scene where someone was running toward someone else with a knife and I asked "Are you about to commit murder?" I would NOT accept "Why don't we call it knife relocation?" as an acceptable answer.
Afterward, Robert Wright says that Eliezer is being euphemistic. This perception that Eliezer's answer was an attempt to substitute nice sounding wording for something awful confirms, to me, that Robert's intent was not to ask "What word should we use for this?" but was intended more like "You think we should build a WHAT? What are you thinking?"
Now, it could be argued that Eliezer accidentally failed to detect the salient connotation. It could be argued, and probably fairly effectively (against me anyway) that the reason for Eliezer's mistake is that he was having one of his arrogant moments and he genuinely thought that, because of a gigantic intelligence difference between Robert and himself, that Robert was asking a moronic question based on the stupid perception that a super powerful AI would be exactly the same as a real God (whatever that means). In this case, I would classify that as a "social skills / character flaw induced faux pas".
In my personal interpretation of Eliezer's behavior, I'm giving him more credit than that - I am assuming that he has previously encountered people by that point (2010) who have flipped out about the possibility that he wants to build a God and have voiced valid and poignant concerns like "Why do you believe it is possible to succeed at controlling something a bazillion times smarter than you?" or "Why would you want us imperfect humans to make something so insanely powerful if it's more or less guaranteed to be flawed?" I'm assuming that Eliezer interpreted correctly when the salient part of someone's question is not in it's literal wording but in connotations relating to the situation.
This is why it looks, to me, like Eliezer's intent was to brush him off by choosing to answer this question as if it were a question about what word to use and hoping that Robert didn't have the nerve to go for the throat with valid and poignant questions like the examples above.
The topic of whether this was an unintentional faux pas or an intentional brush-off isn't the most important thing here.
The most important questions, in my opinion, are:
"Does Eliezer intend to build something this powerful?"
"Does Eliezer really think that something a bazillion times as intelligent as himself can be controlled?"
"Do you and I agree/disagree that it's a good idea to build something this powerful / that it can be controlled?"
If you are taking this as a question about what to CALL the thing, then I agree completely that the AI should not be called a God. But he said "in some sense" which means that his question is about something deeper than choosing a word. The wording he's using is asking something more like "Do you think we should build something similar to a God?"
If forced to use that term and answer the question as you ask it, with a "Yes" or "No" then the correct answer would be "No". He is not trying to create a God, h...
Various people raised concerns that growth might ruin the culture after reading my "LessWrong could grow a lot" thread. There has been some discussion about whether endless September, a phenomenon that kills online discussion groups, is a significant threat to LessWrong and what can be done. I really care about it, so I volunteered to code a solution myself for free if needed. Luke invited debate on the subject (the debate is here) and will be sent the results of this poll and asked to make a decision. It was suggested by him in an email that I wait a little while and then post my poll (meta threads are apparently annoying to some, so we let people cool off). Here it is, preceded by a Cliff's notes summary of the concerns.
Why this is worth your consideration:
- Yvain and I checked the IQ figures in the survey against other data this time, and the good news is that it's more believable that the average LessWronger is gifted. The bad news is that LessWrong's IQ average has decreased on each survey. It can be argued that it's not decreasing by a lot or we don't have enough data, but if the data is good, LessWrong's average has lost 52% of it's giftedness since March of 2009.
- Eliezer documented the arrival of poseurs (people who superficially copycat cultural behaviors - they are reported to over-run subcultures) which he termed "Undiscriminating Skeptics".
- Efforts to grow LessWrong could trigger an overwhelming deluge of newbies.
- LessWrong registrations have been increasing fast and it's possible that growth could outstrip acculturation capacity. (Chart here)
- The Singularity Summit appears to cause a deluge of new users that may have similar effect to the September deluges of college freshman that endless September is named after. (This chart shows a spike correlated with the 2011 summit where 921 users joined that month, which is roughly equal to the total number of active users LW tends to have in a month if you go by the surveys or Vladmir's wget.)
- A Slashdot effect could result in a tsunami of new users if a publication with lots of readers like the Wall Street Journal (they used LessWrong data in this article) decides to write an article on LessWrong.
- The sequences contain a lot of the culture and are long meaning that "TLDR" may make LessWrong vulnerable to cultural disintegration. (New users may not know how detailed LW culture is or that the sequences contain so much culture. I didn't.)
- Eliezer said in August that the site was "seriously going to hell" due to trolls.
- A lot of people raised concerns.
Two Theories on How Online Cultures Die:
Overwhelming user influx.
There are too many new users to be acculturated by older members, so they form their own, larger new culture and dominate the group.
Trending toward the mean.
A group forms because people who are very different want a place to be different together. The group attracts more people that are closer to mainstream than people who are equally different because there are more mainstream people than different people. The larger group attracts people who are even less different in the original group's way for similar reasons. The original group is slowly overwhelmed by people who will never understand because they are too different.
Poll Link:
Endless September Poll.
Request for Feedback:
In addition to constructive criticism, I'd also like the following:
Your observations of a decline or increase in quality, culture or enjoyment at LessWrong, if any.
Ideas to protect the culture.
Ideas for tracking cultural erosion.