There are fairly elementary arguments that, in the absence of uncertainty, any preferences not described by a utility function are problematic—this is the circular preferences argument.
No, it is not the circular preferences argument!
Arguments against circularity of preferences—that is, against violations of the axiom of transitivity—are all well and good. But (in the VNM formalism) preferences cannot be described by a utility function if they violate any of the axioms—not just transitivity! Transitive preferences can fail to be describable by a utility function!
I wrote a comment about this on the post of Eliezer’s which you linked. It would really be very nice if we did not perpetuate misconceptions after they’ve been pointed out.
We're not actually talking about the VNM formalism here. That's why the "in the absence of uncertainty" part is important.
We have a finite set of world-states and preferences over those world-states. We do not care about preferences over random mixtures of world-states, we don't even have a notion of random mixtures, just the deterministic states themselves. We want a utility function which encodes our preferences over those deterministic world-states.
In the absence of uncertainty, we don't actually need the continuity assumption or the independence assumption for anything. They don't even make sense; we need a notion of random mixtures just to state those assumptions. VNM utility needs those because it's trying to get expected utility maximization right out the door. But we're not starting from VNM utility, we're starting from deterministic utility.
Whether we need completeness or not is more debatable. It depends on how we're interpreting missing preferences. If we interpret missing preferences as "I don't know", then it seems natural to allow the utility function to give any possible preference for that pair. In that case, lack of completeness may mean our utility function isn't unique, but it won't prevent a utility function from existing.
It's exactly the same in Eliezer's post. His circular preferences argument comes before random outcomes are even introduced. There's no notion of randomness at that point, no notion of lotteries, so he's not talking about VNM utility. The circular preferences argument is not the VNM utility theorem, it is a separate thing which makes a different claim under weaker assumptions. That does not make it incorrect.
I've tried to minimize the technical prerequisites for this post, but it's still very abstract and mathy. If you understand it and can write well, please consider writing up a more human-readable version which builds around a concrete example or two rather than keeping everything abstract. Alternatively, if you are Eliezer Yudkowsky, consider integrating the FTAP into that great intro I linked above.
I'll probably get around to writing a more concrete version of this post eventually, but I wanted to get the idea out there, since hardly anyone seems to know about it.
How does this interact with time preference ? As stated, an elementary consequence of this theorem is that either lending (and pretty much every other capitalist activity) is unprofitable, or arbitrage is possible.
Great question. The setup here assumes zero interest rates - in particular, I'm implicitly allowing borrowing without interest via short sales (real-world short sales charge interest). Once we allow for nonzero interest, there's a rate charged to borrow, and the price of each asset is its discounted expected value rather than just expected value. That's one of several modifications needed in order to use this theorem in real-world finance. (The same applies to the usual presentation of the Dutch Book arguments, and the same modification is possible.)
Assumed background: Acyclic preferences, Dutch Book theorems
There are fairly elementary arguments that, in the absence of uncertainty, any preferences not described by a utility function are problematic - this is the circular preferences argument. There are also fairly elementary arguments that, if we handle uncertainty by taking weighted sums of utilities of different outcomes, then the weights should follow the usual rules of probability - these are the Dutch Book arguments. But in the middle there’s a jump: we need to assume that taking weighted sums of utilities makes sense for some reason. There are some high-powered theorems which make that jump (specifically the complete class theorem), but they’re not very mathematically accessible.
(If any of that sounds new, you should read Yudkowsky’s excellent intro to this stuff before reading this post.)
It turns out that there is a relatively simple theorem which bridges the gap between deterministic utility and Dutch Book arguments. But rather than hanging out in decision theory textbooks, it’s been living it up in finance. It’s called the Fundamental Theorem of Asset Pricing (FTAP).
Here’s the setup. Just like the Dutch Book arguments, we have a bunch of tradable assets - i.e. betting contracts, like stock options or horse race bets. We have a bunch of possible outcomes - i.e. possible prices of an underlying stock at expiry, or possible winners of the horse race. Each asset's final value will depend on the outcome. Then the FTAP states that either:
Note that this is exactly what we need to round out the Dutch Book arguments: either there exists an arbitrage opportunity, or we compare assets using a weighted sum of possible outcome values.
Let’s prove it. First, we’ll name some variables:
FTAP says that either:
I’ll state the proof informally - if you know a little linear algebra, it’s easy but tedious to formalize and see that it works. The key question is: how many assets, and how many possible outcomes? With N assets and M outcomes, our arbitrage condition has N variables (the q’s) and M+1 equations (one for each outcome plus the current cost constraint). Conversely, our probability distribution condition has M variables (the p’s) and N equations. We generally expect the system to be solvable when the number of variables is at least as large as the number of equations. So, either:
I’m brushing some stuff under the rug here - i.e. maybe there are more assets than outcomes, but the prices line up perfectly. That’s where the linear algebra comes in - the above works for full-rank V, but rank-deficient V requires checking the usual corner cases. If you take a math finance class, you’ll probably go through that tedium in its full glory, along with some more interesting extensions of the theorem.
Anyway, what have we shown? We actually haven’t established that the “probability distribution” p_j is a probability distribution - we’ve shown that the prices are described by some weighted sum of outcome values, but the weights could still be negative or not sum to 1. That’s fine - the usual Dutch Book arguments show that the weights are a probability distribution (or else there’s an arbitrage opportunity). We’ve bridged the gap.
All the usual considerations of the Dutch Book theorems still apply. “Arbitrage” means exactly the same thing here that it means in the Dutch Book theorems. As usual, we’re formulating things with “bets” and “contracts” and “arbitrage” and “prices”, but that can model a much wider range of phenomena.
One interesting point: the probability distribution may not be unique. There may be more than one possible distribution which satisfies the conditions. This works fine with the Dutch Book arguments: each possible distribution corresponds to a different prior.