The posts you linked to are injunctive, not descriptive, and I note that humans are certainly prone to this kind of cheating, or else we wouldn't have cocaine addicts (and yes, I've even seen very smart and rational people develop addictions, although they tend to be somewhat quicker to break them). It would depend on design, of course, but why should we expect that AIs would not also do this? Or, more to the point, how could we design them so that they don't?
(On the other hand, maybe this is more a question about human neuropsychology than about AI design.)
People are only in favor of shortcuts in some areas - generally, where the "point" of that reward isn't the person's own goal. So, people will use contraceptives if they want to have sex for pleasure, even if the reward exists to make them reproduce (to anthropomorphize evolution). People might use drugs to feel okay because they are trying to feel okay, not accomplish goals by feeling okay (only an example). On the other hand, many (most?) people interested in, say, solving world hunger would reject a pill that gives them a false sense of having...
bentarm writes:
I'm just echoing everyone else here, but I don't understand why the AI would do anything at all other than just immediately find the INT_MAX utility and halt - you can't put intermediate problems with some positive utility because the AI is smarter than you and will immediately devote all its energy to finding INT_MAX.
Now, this is in response to a proposed AI who gets maximum utility when inside its box. Such an AI would effectively be a utility junkie, unable to abandon its addiction and, consequently, unable to do much of anything.
(EDIT: this is a misunderstanding of the original idea by jimrandomh. See comment here.)
However, doesn't the same argument apply to any AI? Under the supposition that it would be able to modify its own source code, the quickest and easiest way to maximize utility would be to simply set its utility function to infinity (or whatever the maximum is) and then halt. Are there ways around this? It seems to me that any AI will need to be divided against itself if it's ever going to get anything done, but maybe I'm missing something?