I thought of the idea that maybe the human decision maker has multiple utility functions that when you try to combine them into one function some parts of the original functions don't necessarily translate well... sounds like the "shards of desire" are actually a bunch of different utility functions.

This is an interesting perspective, and I would agree that we humans typically have multiple decision criteria which often can't be combined well. However, I don't think it is quite right to call then utility functions. Humans are adaptation-executers, not fitness-maximizers - so it is more like we each have a bunch of behavioral patterns that we apply when appropriate, and potential conflicts arise when multiple patterns apply.