I suspect the cycles of the directed cyclic graph take a bit longer to run through than an obvious and quick cycle as this seems to imply.
I have a vague hypothesis that the cyclic nature of human preferences is something evolution hit upon to keep us working, working, working on enhancing our descendants' chances - that that's what the money pump actually is. Remember that a common failure mode in real-world AIs is them getting stuck. So if this is the case, then something won't capture human value until it captures the cyclic, pumpable nature of it. This has obvious and annoying implications for coherently extrapolating it, of course.
Intransitive preferences are a demonstrable characteristic of human behaviour. So why am I having such trouble coming up with real-world examples of money-pumping?
"Because I'm not smart or imaginative enough" is a perfectly plausible answer, but I've been mulling this one over on-and-off for a few months now, and I haven't come up with a single example that really captures what I consider to be the salient features of the scenario: a tangled hierarchy of preferences, and exploitation of that tangled hierarchy by an agent who cyclically trades the objects in that hierarchy, generating trade surplus on each transaction.
It's possible that I am in fact thinking about money-pumping all wrong. All the nearly-but-not-quite examples I came up with (amongst which were bank overdraft fees, Weight Watchers, and exploitation of addiction) had the characteristics of looking like swindles or the result of personal failings, but from the inside, money-pumping must presumably feel like a series of gratifying transactions. We would want any cases of money-pumping we were vulnerable to.
At the moment, I have the following hypotheses for the poverty of real-world money-pumping cases:
Does anyone have anything to add, or any good/arguable cases of real-world money-pumping?