I'm claiming that it is possible to define the utility function of any agent.

It is trivially possible to do that. Since no choice is strictly identical, you just add enough details to make each choice unique, and then choose a utility function that will always reach that choice

My formalism doesn't work like that since the utility function is a function over possible universes, not over possible choices. There is no trivial way to construct a utility function wrt which the given agent's intelligence is close to maximal. However it still might be the case we need to give larger weight to simple utility functions (otherwise we're left with selecting a maximum in an infinite set and it's not clear why it exists). As I said, I don't have the final formula.

A good simple model of human behaviour is that of different modules expressing preferences and short-circuiting the decision making in some circumstances, and a more rational system ("system 2") occasionally intervening to prevent loss through money pumps. So people are transitive in their ultimate decisions, often and to some extent, but their actual decisions depend strongly on which choices are presented first (ie their low level preferences are intransitive, but the rational part of them prevents loops). Would you say these beings have no preferences?

I'd say they have a utility function. Image a chess AI that selects moves by one of two strategies. The first strategy ("system 1") uses simple heuristics like "check when you can" that produce an answer quickly and save precious time. The second strategy ("system 2") runs a minimax algorithm with a 10-move deep search tree. Are all of the agent's decisions perfectly rational? No. Does it have a utility function? Yes: winning the game.