amcknight comments on A taxonomy of Oracle AIs - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (51)
There may be ways around having an FAI-complete Advisor if you ask somewhat less generic questions. Maybe questions of the form: "Which of these 3 options would it be best for me to do if I want to satisfy this utility function?" Where "this" is the best you've found so far. You probably can't ask questions like this indefinitely, but with a smart ordering of questions you could probably get a lot done quite safely. This all assumes you can build an initial Advisor AI safely. How do you create a powerful AI without recursive self-improvement or that doesn't have goals?