Incorrect comments on Yet another safe oracle AI proposal - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (33)
Because it is safe to ask a non-agent Oracle questions which do not have dangerous answers (or create dangerous information while calculating the answer). On the other hand, an Oracle that behaves as an agent is unsafe to ask any question because it might convert the planet to computronium to calculate the answer.
OK, so you're using "behaves as an agent" to mean what I mean by "cares about our universe". It doesn't sound like we disagree on substance (i.e. what such a program would do if it worked as proposed).