= 27d567bc2d48d9f40e9ccc20ea370b15)
Perplexed comments on David Chalmers' "The Singularity: A Philosophical Analysis" - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (202)
Roko combined the conccept with the (rather less sensible) idea of promoting those instrumental values into terminal values - and was met with a chorus of "Unfriendly AI".
Hollerith produced several pages on the topic.
Probably the best-known continuation is via Omohundro.
"Universal Instrumental Values" is much the same idea as "Basic AI drives" dressed up a little differently:
http://selfawaresystems.com/2007/11/30/paper-on-the-basic-ai-drives/
http://selfawaresystems.com/2007/10/05/paper-on-the-nature-of-self-improving-artificial-intelligence/
You are right. I hadn't made that connection. Now I have a little more respect for Omohundro's work.
I was a little bit concerned about your initial Omohundro reaction.
Omohundro's material is mostly fine and interesting. It's a bit of a shame that there isn't more maths - but it is a difficult area where it is tricky to prove things. Plus, IMO, he has the occasional zany idea that takes your brain to interesting places it didn't dream of before.
I maintain some Omohundro links here.
As a side point, you could also re-read "Basic AI drives" as "Basic Replicator Drives" - it's systemic evolution.