= 27d567bc2d48d9f40e9ccc20ea370b15)
Vladimir_Nesov comments on Open Thread: April 2010 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (524)
David Chalmers has written up a paper based on the talk he gave at 2009 Singularity Summit:
From the blog post where he announced the paper:
Rather sad to see Chalmers embracing the dopey "singularity" terminology.
He seems to have toned down his ideas about development under conditions of isolation:
"Confining a superintelligence to a virtual world is almost certainly impossible: if it wants to escape, it almost certainly will."
Still, the ideas he expresses here are not very realistic, IMO. People want machine intelligence to help them to attain their goals. Machines can't do that if they are isolated off in virtual worlds. Sure there will be test harnesses - but of course we won't keep these things permanently restrained on grounds of sheer paranoia - that would stop us from using them.
53 pages with only 2 mentions of zombies - yay.
We can't test for values -- we don't know what they are. A negative test might be possible ("this thing surely has wrong values"), as a precaution, but not a positive test.
Testing often doesn't identify all possible classes of flaw - e.g. see:
http://en.wikipedia.org/wiki/Unit_testing#Unit_testing_limitations
It is still very useful, nonetheless.