gwern comments on AI prediction case study 5: Omohundro's AI drives - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (5)
I think it's the same thing as before. AI drives is about a particular set of behaviors being an instrumental value for a large subset of all plausible agents; rationality is one of these instrumental (and not terminal) drives.
Providing an instance where an agent trades off an instrumental good (rationality) for a terminal good (pleasure) is simply not a counter-example - what else would an agent do when offered such a tradeoff? It would be like saying "supposedly, people earn money so as to spend it on things they want; but look! they're spending money on things like trips to Tahiti! Clearly that is not why they really earn money..."