By being internally inconsistent, and only saved by your mistakes in A.

For example it can be argued that proper D should treat risk of dying in all possible ways the same way. If person's D considers dying of shark attack worse than dying of infection (given similar level of suffering etc), and their A has completely wrong idea of how likely shark attacks and infections are, they might take precautions about sharks and infections that are exactly correct. If they find out what A is really like, and start using it, their decisions suddenly become inconsistent.

Of course you can argue from fundamentalist position that utility function is "never wrong", but if you can be trivially Dutch booked, or have ridiculously inconsistent preferences between essentially equivalent outcomes (like dying), then it's "wrong" as far as I'm concerned.