Yudkowsky seemed to me simplistic in his understanding of moral norms. “You would not kill a baby,” he said to me, implying that was one norm that could easily be programmed into a machine.
“Some people do,” I pointed out, but he didn’t see the full significance. SS officers killed babies routinely because of an adjustment in the society from which they sprang in the form of Nazism. Machines would be much more radically adjusted away from human social norms, however we programmed them.
Wow. This particular mistake seems to be an unlikely and even difficult mistake to make in good faith,
as opposed to, for example, by outright dishonesty.
Update: I told Appleyard of his mistake, and he simply denied that his article has made a mistake on this matter.
Never mind, it seems they don't even try to be honest.
As of an hour ago, I had not yet heard of the Centre for the Study of Existential Risk.
Luke announced it to Less Wrong, as The University of Cambridge announced it to the world, back in April:
CSER is scheduled to launch next year.
Here is a small selection of CSER press coverage from the last two days:
http://www.bbc.co.uk/news/technology-20501091
http://www.guardian.co.uk/education/shortcuts/2012/nov/26/cambridge-university-terminator-studies
http://www.dailymail.co.uk/news/article-2238152/Cambridge-University-open-Terminator-centre-study-threat-humans-artificial-intelligence.html
http://www.theregister.co.uk/2012/11/26/new_centre_human_extinction_risks/
http://www.slashgear.com/new-ai-think-tank-hopes-to-get-real-on-existential-risk-26258246/
http://www.techradar.com/news/world-of-tech/super-brains-to-guard-against-robot-apocalypse-1115293
http://www.hindustantimes.com/world-news/Europe/Cambridge-to-study-risks-from-robots-at-Terminator-Centre/Article1-964746.aspx
http://economictimes.indiatimes.com/news/news-by-industry/et-cetera/cambridge-to-study-risks-from-robots-at-terminator-centre/articleshow/17372042.cms
http://www.extremetech.com/extreme/141372-judgment-day-update-disneys-grenade-catching-robot-and-the-burger-flipping-robot-that-could-replace-2-million-us-workers
http://slashdot.org/topic/bi/cambridge-university-vs-skynet/
http://www.businessinsider.com/researchers-robots-risk-human-civilization-2012-11
http://www.newscientist.com/article/dn22534-megarisks-that-could-drive-us-to-extinction.html
http://news.cnet.com/8301-11386_3-57553993-76/killer-robots-cambridge-brains-to-assess-ai-risk/
http://www.globalpost.com/dispatches/globalpost-blogs/weird-wide-web/cambridge-university-opens-so-called-termintor-centre-stu
http://www.washingtonpost.com/world/europe/cambridge-university-to-open-center-studying-the-risks-of-technology-to-humans/2012/11/25/e551f4d0-3733-11e2-9258-ac7c78d5c680_story.html
http://www.foxnews.com/tech/2012/11/26/terminator-center-to-open-at-cambridge-university/
Google News: All 119 news sources...
Here's an excerpt from one quite typical story appearing in tech-tabloid theregister.co.uk today:
Humanity’s last invention and our uncertain future
http://www.cam.ac.uk/research/news/humanitys-last-invention-and-our-uncertain-future/
ThreeFour quick observations: