I do not understand what the word "erroneous" is supposed to mean in this context.
For the sake of argument, I will go ahead and ask what sort of nonconstructive entities you think an AI needs to reason about, in order to function properly.
Aaaand this makes me curious. Eliezer, for the sake of argument, do you really think we'd do good by prohibiting the AI from using reductio ad absurdum?
[edit: sorry, the formatting of links and italics in this is all screwy. I've tried editing both the rich-text and the HTML and either way it looks ok while i'm editing it but the formatted terms either come out with no surrounding spaces or two surrounding spaces]
In the latest Rationality Quotes thread, CronoDAS quoted Paul Graham: