Solomonoff Induction where sometimes you really can be helped by thinking formally about Occam's Razor.
That's curious, because Solomonoff Induction is something not even an enormously powerful (but computable) AI can evaluate.
Yes, but my point is that thinking about SI or MML in the abstract helps because people sometimes gain insight from asking "How complex is that computer program?" I haven't seen appeal-to-CEV produce much insight in practice, and any insight it could produce can probably be better produced by appealing to the relevant component principle of CEV instead. (Nor yet is this a critique of CEV, because it's meant as an AI design, not as a moral intuition pump.)
r/HPMOR readers on heroic responsibility - not the OP, the comments. Holy snorkels this is good.