I think we have almost reached agreement, just a few more nitpicks I seem to have with your current post.
the independence principle doesn't strictly hold in the real world, like there are no strictly right angle in the real world
Its pedantic, but these two statements aren't analogous. A better analogy would be
"the independence principle doesn't strictly hold in the real world, like the axiom that all right angles are equal doesn't hold in the real world"
"there are no strictly identical outcomes in the real world, like there are no strictly right angle in the real world"
Personally I prefer the second phrasing. The independence principle and the right angle principle do hold in the real world, or at least they would if the objects they talked about ever actually appeared, which they don't.
I'm in general uncomfortable with talk of the empirical status of mathematical statements, maybe this makes me a Platonist or something. I'm much happier with talk of whether idealised mathematical objects exist in the real world, or whether things similar to them do.
What this means is we don't apply VNM when we think independence is relatively true, we apply them when we think the outcomes we are facing are relatively similar to each other, enough that any difference can be assumed away.
But do we have any way to measure the degree of error introduced by this approximation?
This is an interesting problem. As far as I can tell, its a special case of the interesting problem of "how do we know/decide our utility function?".
Do we have ways to recognize the cases where we shouldn't apply the expected utility theory
I've suggested one heuristic that I think is quite good. Any ideas for others?
(Once again, I want to nitpick the language. "Do we have ways to recognize the cases where two outcomes look equal but aren't" is the correct phrasing.
Followup to : Is risk aversion really irrational?
After reading the decision theory FAQ and re-reading The Allais Paradox I realized I still don't accept the VNM axioms, especially the independence one, and I started thinking about what my true rejection could be. And then I realized I already somewhat explained it here, in my Is risk aversion really irrational? article, but it didn't make it obvious in the article how it relates to VNM - it wasn't obvious to me at that time.
Here is the core idea: information has value. Uncertainty therefore has a cost. And that cost is not linear to uncertainty.
Let's take a first example: A is being offered a trip to Ecuador, B is being offered a great new laptop and C is being offered a trip to Iceland. My own preference is: A > B > C. I love Ecuador - it's a fantastic country. But I prefer a laptop over a trip to Iceland, because I'm not fond of cold weather (well, actually Iceland is pretty cool too, but let's assume for the sake of the article that A > B > C is my preference).
But now, I'm offered D = (50% chance of A, 50% chance of B) or E = (50% chance of A, 50% chance of C). The VNM independence principle says I should prefer D > E. But doing so, it forgets the cost of information/uncertainty. By choosing E, I'm sure I'll be offered a trip - I don't know where, but I know I'll be offered a trip, not a laptop. By choosing D, I'm no idea on the nature of the present. I've much less information on my future - and that lack of information has a cost. If I know I'll be offered a trip, I can already ask for days off at work, I can go buy a backpack, I can start doing the paperwork to get my passport. And if I know I won't be offered a laptop, I may decide to buy one, maybe not as great as one I would have been offered, but I can still buy one. But if I chose D, I've much less information about my future, and I can't optimize it as much.
The same goes for the Allais paradox: having certitude of receiving a significant amount of money ($24 000) has a value, which is present in choice 1A, but not in all others (1B, 2A, 2B).
And I don't see why a "rational agent" should neglect the value of this information, as the VNM axioms imply. Any thought about that?