to their benefit
You assert this as if it were an axiom. It doesn't look like one to me. Show me the benefit.
And I still don't understand why would I want to become an ideal utility maximizer.
Here's a puzzle based on something I used to be confused about:
It is known that utility functions are equivalent (i.e. produce the same preferences over actions) up to a positive affine transformation: u'(x) = au(x) + b where a is positive.
Suppose I have u(vanilla) = 3, u(chocolate) = 8. I prefer an action that yields a 50% chance of chocolate over an action that yields a 100% chance of vanilla, because 0.5(8) > 1.0(3).
Under the positive affine transformation a = 1, b = 4; we get that u'(vanilla) = 7 and u'(chocolate) = 12. Therefore I now prefer the action that yields a 100% chance of vanilla, because 1.0(7) > 0.5(12).
How to resolve the contradiction?
to their benefit
You assert this as if it were an axiom. It doesn't look like one to me. Show me the benefit.
And I still don't understand why would I want to become an ideal utility maximizer.
For the sake of organization, I suggest discussing such things on the comment threads of Sequence posts.
you can always use meta factors to argue why your revealed preferences actually were coherent.
Three observations. First, those aren't meta factors, those are just normal positive terms in the utility function that one formulation ignores and another one includes. Second, "you can always use" does not necessarily imply that the argument is wrong. Third, we are not arguing about coherency -- why would the claim that, say, I value the perception of myself as someone who votes for X more than 10c be incoherent?
we know that humans are born being somewhat farther-than-optimal from the ideal utility maximizer, and practicing the art of rationality adds value to their lives by getting them somewhat closer to the ideal than where they started.
I disagree, both with the claim that getting closer to the ideal of a perfect utility maximizer necessarily adds value to people's lives, and with the interpretation of the art of rationality as the art of getting people to be more like that utility maximizer.
Besides, there is still the original point: even if you posit some entilty as a perfect utility maximizer, what would its utility function include? Can you use the utility function to figure out which terms should go into the utility function? Colour me doubtful. In crude terms, how do you know what to maximize?
Well I guess I'll focus on what seems to be our most fundamental disagreement, my claim that getting value from studying rationality usually involves getting yourself to be closer to an ideal utility maximizer (not necessarily all the way there).
Reading the Allais Paradox post can make a reader notice their contradictory preferences, and reflect on it, and subsequently be a little less contradictory, to their benefit. That seems like a good representative example of what studying rationality looks like and how it adds value.
with no logistical inconveniences
I would argue all those values are irrational.
Please do.
The expression "irrational values" sounds like a category mistake to me.
You're right that "those values are irrational" is a category mistake, if we're being precise. But Houshalter has an important point...
Any time you violate the axioms of a coherent utility-maximization agent, e.g. falling for the Allais paradox, you can always use meta factors to argue why your revealed preferences actually were coherent.
Like, "Yes the money pump just took some of my money, but you haven't considered that the pump made a pleasing whirring sound which I enjoyed, which definitely outweighed the value of the money it pumped from me."
While that may be a coherent response, we know that humans are born being somewhat farther-than-optimal from the ideal utility maximizer, and practicing the art of rationality adds value to their lives by getting them somewhat closer to the ideal than where they started.
A "rationality test" is a test that provides Bayesian evidence to distinguish people earlier vs. later on this path toward a more reflectively coherent utility function.
Having so grounded all the terms, I mostly agree with pwno and Houshalter.
I guess I'll be the first one to offer a steel man interpretation of pwno's post:
Assuming you're anti-Trump... If voting for Trump could be done with no logistical inconveniences, and somehow legally pay you a reward of say 10 cents, and you didn't believe this offer was being made to anyone else, then a good rationality test is whether you would take that offer.
The expected monetary value of insurance is negative (or rather, negative in real dollars. It can be positive in nominal dollars but underperform inflation.)
But the utility is not linear in money. Losing e.g. $10,000 might be 20 times as bad as losing $1,000. If so, you should pay $1,000 100% of the time to avoid paying $10,000 8% of the time.
The insurance company averages out over many buyers, so their utility is roughly linear.
Insurance is just trading against different utility scales.
Yep. Also note that if you had $1M in the bank, you would then not prefer to buy insurance for something on the order of $10k.
I made this joke site: https://flashcash.money
It's often rational to burn cash on positional goods like Rolexes and bottle service at clubs, but FlashCash.money takes that value proposition to the logical extreme.
This is really good. I think that a summary of what the presentation covers would be useful as well. I wrote a draft one below:
'You Are A Brain' is a presentation by Liron Shapira that is tailored for a general audience and provides an introduction to some of the the core LessWrong concepts including:
ignorance exists in the map, not in the territory. If I am ignorant about a phenomenon, that is a fact about my own state of mind, not a fact about the phenomenon itself.
Awesome thanks.
View more: Next
And why should we be utility maximization agents?
Assume the following situation. You are very rich. You meet a poor old lady in a dark alley who carries a purse with her, with some money which is a lot from her perspective. Maybe it's all her savings, maybe she just got lucky once and received it as a gift or as alms. If you mug her, nobody will ever find it out and you get to keep that money. Would you do it? As a utility maximization agent, based on what you just wrote, you should.
Would you?
Have you read the LW sequences? Because like gjm explained, your question reveals a simple and objective misunderstanding of what utility functions look like when they model realistic people's preferences.