Dead right. It would seem very silly to believe that rationality hits a glass ceiling at human level intelligence. Unlikely though it is, if the AI could predict the number in my head by looking at my facial expressions, then told me to cut my arm off for the good of the human race, I'd suddenly feel very conflicted indeed.