I went out looking for myself and I just watched the bloggingheads video (6:42) where Robert Wright says to Eliezer "It sounds like what you're saying is we need to build a God" and Eliezer is like "Why don't we call it a very powerful optimizing agent?" and grins like he's just fooled someone
Like he's just fooled someone? I see him talking like he's patiently humoring an ignorant child who is struggling to distinguish between "Any person who gives presents at Christmas time" and "The literal freaking Santa Claus, complete with magical flying reindeer". He isn't acting like he has 'fooled' anyone or acting in any way 'sneaky'.
and Robert Wright thinks and he's like "Why don't we call that a euphemism for God?" which destroys Eliezer's grin.
While I wouldn't have been grinning previously whatever my expression had been it would change in response to that question in the direction of irritation and impatience. The answer to "Why don't we call that a euphemism for God?" is "Because that'd be wrong and totally muddled thinking". When your mission is to create an actual very powerful optimization agent and that---and not gods---is actually what you spend your time researching then a very powerful optimization agent isn't a 'euphemism' for anything. It's the actual core goal. Maybe, at a stretch, "God" can be used as a euphemism for "very powerful optimizing agent" but never the reverse.
I'm not commenting here on the question of whether there is a legitimate PR concern regarding people pattern matching to religious themes having dire, hysterical and murderous reactions. Let's even assume that kind of PR concern legitimate for the purpose of this comment. Even then there is a distinct difference between "failure to successfully fool people" and "failure to educate fools". It would be the latter task that Eliezer has failed at here and the former charge would be invalid. (I felt the paragraph I quoted to be unfair on Eliezer with respect to blurring that distinction.)
I don't think that an AI that goes FOOM would be exactly the same as any of the "Gods" humanity has been envisioning and may not even resemble such a God (especially because, if it were a success, it would theoretically not behave in self-contradictory ways like making sinful people, knowing exactly what they're going to do, making them to do just that, telling them not to act like what they are and then punishing them for behaving the way it designed them to). I don't see a reason to believe that it is possible for any intellect to be omniscien...
Various people raised concerns that growth might ruin the culture after reading my "LessWrong could grow a lot" thread. There has been some discussion about whether endless September, a phenomenon that kills online discussion groups, is a significant threat to LessWrong and what can be done. I really care about it, so I volunteered to code a solution myself for free if needed. Luke invited debate on the subject (the debate is here) and will be sent the results of this poll and asked to make a decision. It was suggested by him in an email that I wait a little while and then post my poll (meta threads are apparently annoying to some, so we let people cool off). Here it is, preceded by a Cliff's notes summary of the concerns.
Why this is worth your consideration:
- Yvain and I checked the IQ figures in the survey against other data this time, and the good news is that it's more believable that the average LessWronger is gifted. The bad news is that LessWrong's IQ average has decreased on each survey. It can be argued that it's not decreasing by a lot or we don't have enough data, but if the data is good, LessWrong's average has lost 52% of it's giftedness since March of 2009.
- Eliezer documented the arrival of poseurs (people who superficially copycat cultural behaviors - they are reported to over-run subcultures) which he termed "Undiscriminating Skeptics".
- Efforts to grow LessWrong could trigger an overwhelming deluge of newbies.
- LessWrong registrations have been increasing fast and it's possible that growth could outstrip acculturation capacity. (Chart here)
- The Singularity Summit appears to cause a deluge of new users that may have similar effect to the September deluges of college freshman that endless September is named after. (This chart shows a spike correlated with the 2011 summit where 921 users joined that month, which is roughly equal to the total number of active users LW tends to have in a month if you go by the surveys or Vladmir's wget.)
- A Slashdot effect could result in a tsunami of new users if a publication with lots of readers like the Wall Street Journal (they used LessWrong data in this article) decides to write an article on LessWrong.
- The sequences contain a lot of the culture and are long meaning that "TLDR" may make LessWrong vulnerable to cultural disintegration. (New users may not know how detailed LW culture is or that the sequences contain so much culture. I didn't.)
- Eliezer said in August that the site was "seriously going to hell" due to trolls.
- A lot of people raised concerns.
Two Theories on How Online Cultures Die:
Overwhelming user influx.
There are too many new users to be acculturated by older members, so they form their own, larger new culture and dominate the group.
Trending toward the mean.
A group forms because people who are very different want a place to be different together. The group attracts more people that are closer to mainstream than people who are equally different because there are more mainstream people than different people. The larger group attracts people who are even less different in the original group's way for similar reasons. The original group is slowly overwhelmed by people who will never understand because they are too different.
Poll Link:
Endless September Poll.
Request for Feedback:
In addition to constructive criticism, I'd also like the following:
Your observations of a decline or increase in quality, culture or enjoyment at LessWrong, if any.
Ideas to protect the culture.
Ideas for tracking cultural erosion.