= 27d567bc2d48d9f40e9ccc20ea370b15)
turchin comments on The virtual AI within its virtual world - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (34)
I have thought about something similar with respect to an oracle AI. You program it to try to answer the question assuming no new inputs and everything works to spec. Since spec doesn't include things like the AI escaping and converting the world to computronium to deliver the answer to the box, it won't bother trying that.
I kind of feel like anything short of friendly AI is living on borrowed time. Sure the AI won't take over the world to convert it to paperclips, but that won't stop some idiot from asking it how to make paperclips. I suppose it could still be helpful. It could at the very least confirm that AIs are dangerous and get people to worry about them. But people might be too quick to ask for something that they'd say is a good idea after asking about it for a while or something like that.
I greatly dislike the term "friendly AI". The mechanisms behind "friendly AI" have nothing to do with friendship or mutual benefit. It would be more accurate to call it "slave AI".
I prefer term "Safe AI" as it more self explaining for the outsider.
I think it's more accurate, though the term "safe" has a much larger positive valence than is justified, and is so accurate but misleading. Particularly since it smuggles in EY's presumptions about whom it's safe for, and so whom we're supposed to be rooting for, humans or transhumans. Safer is not always better. I'd rather get the concept of stasis or homogeneity in there. Stasis and homogeneity are, if not the values at the core of EY's scheme, at least the most salient products of it.
Safe AI sounds like it does what you say as long as it isn't stupid. Friendly AIs are supposed to do whatever's best.
For me Safe AI is one that is not existential risk. "Friendly" reminds me about "friendly user interface", that is something superficial for core function.