representation of preference order (lattices vs. probabilities vs. graphical models vs. other mathematical structures), relation and conversions between different representations of the state space (variables/predicates/etc.)

There's actually quite a lot of this in James Joyce's The foundations of causal decision theory, at what appears to me to be a gratuitiously high math level.