That's easy: If p and q are two policies and alpha in [0,1], the mixed policy where today you randomly choose policy p with probability alpha and policy q with probability 1-alpha, is also a policy.

But what if the value an agent assigns to life events depends on the probabilities as well as the final results? Wei Dai gave one example, a concern for fairness, that can make this true, but simple risk aversion or thrill seeking would also suffice. Then it seems to me that the interpretation of the "linear aggregation function" becomes problematic.