= 27d567bc2d48d9f40e9ccc20ea370b15)
Grognor comments on Real World Solutions to Prisoners' Dilemmas - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (87)
There are many problems here.
At the end of paragraph 2 and the other examples, you say
But it doesn't, as you point out later in the post, because the payoff matrix isn't D-C > C-C > D-D, as you explain, but rather C-C > D-C > C-D, because of reputational effects, which is not a prisoner's dilemma. "Prisoner's dilemma" is a very specific term, and you are inflating it.
I doubt that quite strongly!
That is not tit-for-tat! Tit-for-tat is start with cooperate and then parrot the opponent's previous move. It does not do what it "expects" the opponent to do. Furthermore, if you categorically expect your opponent to cooperate, you should defect (just like you should if you expect him to defect). You only cooperate if you expect your opponent to cooperate if he expects you to cooperate ad nauseum.
That is not superrationality! Superrationality achieves cooperation by reasoning that you and your opponent will get the same result for the same reasons, so you should cooperate in order to logically bind your result to C-C (since C-C and D-D are the only two options). What is with all this misuse of terminology? You write like the agents in the examples of this game are using causal decision theory (which defects all the time no matter what) and then bring up elements that cannot possibly be implemented in causal decision theory, and it grinds my gears!
This is in direct violation of one of the themes of Less Wrong. If "rational expected utility maximizers" are doing worse than "irrational emotional hangups", then you're using a wrong definition of "rational". You do this throughout the post, and it's especially jarring because you are or were one of the best writers for this website.
9_9
"The good kind of irrationality" is like "the good kind of bad thing". An oxymoron, by definition.
Bullshit. A rational agent is going to do what works. We know this because we stipulated that it was rational. If you mean to say a "stupid number crunching robot that misses obvious details like how to play ultimatum games" then sure it might do as you describe. But don't call it "rational".
You think?
Downvoted.
Great critique!
The first time I read the post, I stopped reading when "tit-for-tat" and "superrationality" were misused in two consecutive sentences. Sadly, that part seems to be still inaccurate after Yvain edited it, because TFT is not dominant in the 100-fold repeated PD, if the strategy pool contains strategies that feed on TFT.
To be fair he doesn't seem to make the claim that TFT is dominant in the fixed length iterated PD. (I noticed how outraged I was that Yvain was making such a basic error so I thought I should double check before agreeing emphatically!) Even so I'm not comfortable with just saying TFT is "evolutionarily dominant" in completely unspecified circumstances.
I agree with pretty much everything you've said here, except:
You don't actually need to continue this chain - if you're playing against any opponent which cooperates iff you cooperate, then you want to cooperate - even if the opponent would also cooperate against someone who cooperated no matter what, so your statement is also true without the "ad nauseum" (provided the opponent would defect if you defected).
You're right. I assumed symmetry, which was wrong.
You're reading this uncharitably. There are also parts that are unclear on Yvain's part, sure, but not to the extent that you claim.
The original group project situation Yvain explores does mirror the Prisoner's Dilemma. Then, later, he introduces reputational effects to illustrate one of the Real World Solutions to the Prisoner's Dilemma that we have already developed.
It's not made crystal clear....
Well, actually it is.
...
I understood Yvain to be speaking metaphorically, or perhaps tongue-in-cheek, when talking about what evolution would take note of. I believe this was his intention, and furthermore is a reasonable reading given our knowledge of Yvain.
I expect that Yvain used 'rational' against the theme of LW on purpose, to create a tension - rationality failing to outperform emotional hangups is a contradiction, that would motivate readers to find the false premise or re-analyse the situation.
I do concur with your point about tit-for-tat. Similarly for super-rationality; although it's possible Yvain is not familiar with Hofstadter's definition and was using 'super' as an intensifier, it seems unlikely.
I had this downvoted based on on form and irritating tone before I looked closely and decided enough of the quotes from Yvain are, indeed, plainly wrong and I encourage hearty dismissal.
Agree. Who is he and what has he done to the real Yvain?
You'll notice I used scare quotes around most of the words you objected to. I'm trying to point out the apparent paradox, using the language that game theorists and other people not already on this website would use, without claiming that the paradox is real or unsolvable.
"This so-called "superrationality” " in the post is still wrong, I think. Would work without "so-called", since the meaning is clear from the context, but it's not conventional usage.
By definition?
I am largely appreciative of your overall comment, but "rational" is a historically legitimate term to describe naive utility-maximisers in this manner. The original post introduced it in inverted commas, suggesting a special usage of the term. While there are less ambiguous ways this could have been expressed, it seems to me the main benefit of doing so would be to pre-empt people complaining about an unfavourable usage of the term "rational". Your response to it seems excessive.
Agreed.
I want to point out that Eliezer's (and LW's general) use of the word 'rationality' is entirely different from the use of the word in the game theory literature, where it usually means VNM-rationaliity, or is used to elaborate concepts like sequential rationality in SPNE-type equilibria.
ETA: Reading Grognor's reply to the parent, it seems that much of the negative affect is due to inconsistent use of the word 'rational(ity)' on LW. Maybe it's time to try yet again to taboo LW's 'rationality' to avoid the namespace collision with academic literature.
And the common usage of 'rational' on lesswrong should be different to what is used in a significant proportion of game theory literature. Said literature gives advice, reasoning and conclusions that is epistemically, instrumentally and normatively bad. According to the basic principles of the site it is in fact stupid and not-rational to defect against a clone of yourself in a true Prisoner's Dilemma. A kind of stupidity that is not too much different to being 'rational' like Spock.
No. The themes of epistemic and instrumental rationality are the foundational premise of the site. It is right there in the tagline on the top of the page. I oppose all attempts to replace instrumental rationality with something that involves doing stupid things.
I do endorse avoiding excessive use of the word.
This is a recurring issue, so perhaps my instructor and textbooks were atypical: we never discussed or even cared whether someone should defect on PD in my game theory course. The bounds were made clear to us in lecture – game theory studies concepts like Nash equilibria and backward induction (using the term 'rationality' to mean VNM-rationality) and applies them to situations like PD; that is all. The use of any normative language in homework sets or exams was pretty much automatically marked incorrect. What one 'should' or 'ought' to do were instead relegated to other courses in, e.g, economics, philosophy, political science. I'd like to know from others if this is a typical experience from a game theory course (and if anyone happens to be working in the field: if this is representative of the literature).
Upon reflection, I tend to agree with these statements. In this case, perhaps we should taboo 'rationality' in its game theoretic meaning – use the phrase 'VNM-rationality' whenever that is meant instead of LW's 'rationality'.
The normative claim is one I am making now about the 'rationality' theories in question. It is the same kind of normative claim I make when I say "empirical tests are better than beliefs from ad baculum".
I could agree to that---conditional on confirmation from one of the Vladimirs that the axioms in question do, in fact, imply the faux-rational (CDT like) conclusions the term would be used to represent. I don't actually see it at a glance and would expect another hidden assumption to be required. I wouldn't be comfortable using the term without confirmation.
I quoted badly; I believe there was a misunderstanding. The first quote in the parent to this should be taken in the context of your sentence segment that "Said literature gives advice". In my paragraph, I was objecting to this from my experiences in my course, where I did not receive any advice on what to do in games like PD. Instead, the type of advice that I received was on how to calculate Nash equilibria and find SPNEs.
Otherwise, I am mostly in agreement with the latter part of that sentence. (ETA: That is, I agree that if current game theoretic equilibrium solutions are taken as advice on what one ought to do, then that is often epistemically, instrumentally, and normatively bad.)
More ETA:
You are correct – VNM-rationality is incredibly weak (though humans don't satisfy it). It is, after all, logically equivalent to the existence of a utility function (the proof of this by von Neumann and Morgenstern led to the eponymous VNM theorem). The faux-rationality on LW and in popular culture requires much stronger assumptions. But again, I don't think these assumptions are made in the game theory literature – I think that faux-rationality is misattributed to game theory. The game theory I was taught used only VNM-rationality, and gave no advice.
I remember hearing about studies where economics and game theory students ended up less "moral" by many usual measures after completing their courses. Less inclined to co-operate, more likely to lie and cheat, more concerned about money, more likely to excuse overtly selfish behaviour and so on. And then these fine, new, upstanding citizens, went on to become the next generation of bankers, traders, stock-brokers, and advisers to politicians and industry. The rest as they say is history.
Said literature makes statements about what is game-theory-rational. Those statements are only epistemically, instrumentally or normatively bad if you take them to be statements about what is LW-rational or "rational" in the layperson's sense.
Ideally we'd use different terms for game-theory-rational and LW-rational, but in the meantime we just need to keep the distinction clear in our heads so that we don't accidentally equivocate between the two.
Disagree on instrumentally and normatively. Agree regarding epistemically---at least when the works are careful with what claims are made. Also disagree with the "game-theory-rational", although I understand the principle you are trying to get at. A more limited claim needs to be made or more precise terminology.
I would be interested in reading about the bases for your disagreement. Game theory is essentially the exploration of what happens if you postulate entities who are perfectly informed, personal utility-maximisers who do not care at all either way about other entities. There's no explicit or implicit claim that people ought to behave like those entities, thus no normative content whatsoever. So I can't see how the game theory literature could be said to give normatively bad advice, unless the speaker misunderstood the definition of rationality being used, and thought that some definition of rationality was being used in which rationality is normative.
I'm not sure what negative epistemic or instrumental outcomes you foresee either, but I'm open to the possibility that there are some.
Is there a term you prefer to "game-theory-rational" that captures the same meaning? As stated above, game theory is the exploration of what happens when entities that are "rational" by that specific definition interact with the world or each other, so it seems like the ideal term to me.
Under this definition you can't claim epistemic accuracy either. In particular the 'perfectly informed' assumption when combined with the personal utility maximization leads to different behaviors to those described as 'rational'. (It needs to be weakened to "perfectly informed about everything except those parts of the universe that are the other agent.)
This isn't about the agents having selfish desires (in fact, they don't even have to "not care at all about other entities"---altruism determines what the utility function is, not how to maximise it.) No, this is about shoddy claims about decision theory that are either connotatively misleading or erroneous depending on how they are framed. All those poor paperclip maximisers who read such sources and take them at face value will end up producing less paperclips than they could have if they knew the correct way to interact with the staples maximisers in contrived scenarios.
I think this was a legitimate use of "by definition", since it's the definition we use on this website. You're right that "rational" has often meant "blindly crunching numbers without looking at all available information &c." but I thought we had a widespread agreement here not to use the word like that.
You're right that my response seems excessive, but I don't know if it actually is excessive rather than merely seeming so.
A term "bad rationality" is also used on this website. It is a partial rationality, and it may be harmful. On the other hand, as humans, partial rationality is all we have, don't we?
But now I am discussing labels on the map, not the territory.
You're attaching a negative connotation where there doesn't have to be one. In econ and game theory literature, "rational" means something else, not necessarily something bad. It also refers to something specific. If we want to talk about that specific referent, we have limited options.
I would propose suffixing alternative uses of the word "rational" with a disambiguating particle. Thus above, Yvain could have used "econ-rational". If we ever have cause to talk about the Rationalist philosophical tradition, they can be "p-Rationalists". Annoyingly, I don't actually believe we need to do this for disambiguation purposes.
<retracted>
This is disturbing. I had been looking forward to the sequence, but based on this comment and some others I'm starting to wonder how accurate it is overall.
Overall, it is accurate. There are nits to pick, sure, and some loose language, but the actual math and theory is solid so far. The generalisations to real life
are correct as well.
Upvoted. By the way, I think you meant to type "C-C > D-C > D-D" instead of "C-C > D-C > D-C", and you might also want to include C-D. Not sure whether C-D is supposed to be last or second last in Yvain's example, because of the reputational effects.