= 27d567bc2d48d9f40e9ccc20ea370b15)
Wei_Dai comments on The virtual AI within its virtual world - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (34)
I have thought about something similar with respect to an oracle AI. You program it to try to answer the question assuming no new inputs and everything works to spec. Since spec doesn't include things like the AI escaping and converting the world to computronium to deliver the answer to the box, it won't bother trying that.
I kind of feel like anything short of friendly AI is living on borrowed time. Sure the AI won't take over the world to convert it to paperclips, but that won't stop some idiot from asking it how to make paperclips. I suppose it could still be helpful. It could at the very least confirm that AIs are dangerous and get people to worry about them. But people might be too quick to ask for something that they'd say is a good idea after asking about it for a while or something like that.
I agree with this. Working on "how can we safely use a powerful optimization process to cure cancer" (where "cure cancer" stands for some technical problem that we can clearly define, as opposed to the sort of fuzzy philosophical problems involved in building FAI) does not seem like the highest value for one's time. Once such a powerful optimization process exists, there is only a very limited amount of time before, as you say, some idiot tries to use it in an unsafe way. How much does it really help the world to get a cure for cancer during this time?