My idea was the question was not intended "Run all of these questions to completion and tell me which takes the least time." Which would definitely cause problems. The question was "Stop all programs, and give me an answer, once you hit an answer any of these conditions."
Although, that brings up ANOTHER problem. The Oracle AI has to interpret grammar. If it interprets grammar the wrong way, then large amounts of unexpected behavior can occur. Since there are no guarantees the Oracle understands your grammar correctly, there IS no safe question to ask a powerful Oracle AI without having verified it's grammar first.
So in retrospect, yes, that question could get me into a lot of trouble, and you are correct to point that out.
The Future of Humanity Institute wants to pick the brains of the less wrongers :-)
Do you have suggestions for safe questions to ask an Oracle? Interpret the question as narrowly or broadly as you want; new or unusual ideas especially welcome.