First of all, congratulations, Eliezer! That's great work. When I read your 3-line description, I thought it would never be computable. I'm glad to see you can actually test it.

Eliezer_Yudkowsky wrote on 19 August 2009 03:05:15PM

... Moving second is a disadvantage (at least it seems to always work out that way, counterexamples requested if you can find them)

Rock-paper-scissors ?
Negotiating to buy a car?

I would like to begin by saying that I don't believe my own statements are True, and I suggest you don't either. I do request that you try thinking WITH them before attacking them. It's really hard to think with an idea AFTER you've attacked it. I've been told my writing sounds preachy or even fanatical. I don't say "In My Opinion" enough. Please imagine "IMO" in front of every one of my statements. Thanks!

Having more information (not incorrect "information") on the opponent's decisions is beneficial.

Let's distinguish Secret Commit & Simultaneous Effect (SCSE) from Commit First & Simultaneous Effect (CFSE) and from Act & Effect First (AEF). That's just a few categories from a coarse categorization of board war games.

The classic gunfight at high noon is AEF (to a first approximation, not counting watching his face & guessing when his reaction time will be lengthened). The fighter who draws first has a serious advantage, the fighter who hits first has a tremendous advantage, but not certain victory. (Hollywood not withstanding, people sometimes keep fighting after taking handgun hits, even a dozen of them.) I contend that all AEFs give advantage to the first actor. Chess is AEF.

My understanding of the Prisoner's Dilemma is that it is SCSE as presented. On this thread, it seems to have mutated into a CFSE (otherwise, there just isn't any "first", in the ordinary, inside-the-Box-Universe, timeful sense). If Prisoner A has managed to get information on Prisoner B's commitment before he commits, this has to be useful. Even if PA is a near-Omega, it can be a reality check on his Visualization of the Cosmic All. In realistic July 2009 circumstances, it identifies PB as one of the 40% of humans who choose 'cooperate' in one-shot PD. PA now has a choice whether to be an economist or a friend.

And now we get down to something fundamental. Some humans are better people than the economic definition of rationality, which " ... assume that each player cares only about minimizing his or her own time in jail". " ... cooperating is strictly dominated) by defecting ... " even with leaked information.

"I don't care what happens to my partner in crime. I don't and I won't. You can't make me care. On the advice of my economist... " That gets both prisoners a 5-year sentence when they could have had 6 months.

That is NOT wisdom! That will make us extinct. (In My Opinion)

Now try on "an injury to one is an injury to all". Or maybe "an injury to one is an (discounted) injury to ME". We just might be able to see that the big nuclear arsenals are a BAD IDEA!

Taking that on, the payoff matrix offered by Wei Dai's Omega (19 August 2009 07:08:23AM)

* cooperate 5/5 0/6
* defect 6/0 1/1

is now transformed into PA's Internal Payoff Matrix (IPM)

* cooperate 5+5κ/5 0+6κ/6
* defect 6+0κ/0 1+1κ/1

In other words, his utility function has a term for the freedom of Prisoner B. (Economists be damned! Some of us do, sometimes.)

"I'll set κ=0.3 ," Says PA (well, he is a thief). Now PA's IPM is:

* cooperate 6.5/5 1.8/6
* Defect 6/0 1.3/1

Lo and behold! 'cooperate' now strictly dominates!

When over 6 billion people are affected, it doesn't take much of a κ to swing my decisions around. If I'm not working to save humanity, I must have a very low κ for each distant person unknown to me.

People say, "Human life is precious!" Show it to me in results. Show it to me in how people budget their time and money. THAT is why Friendly AI is our only hope. We will 'defect' our way into thwarting any plan that requires a lot of people to change their beliefs or actions. That sub-microscopic κ for unknown strangers is evolved-in, it's not going away. We need a program that can be carried out by a tiny number of people.

.

.

.

IMO.

---=

Maybe I missed the point. Maybe the whole point of TDT is to derive some sort of reduced-selfishness decision norm without an ad-hoc utility function adjustment (is that what "rained down from heaven" means?). I can derive the κ needed in order to save humanity, if there were a way to propagate it through the population. I cannot derive The One True κ from absolute principles, nor have I shown a derivation of "we should save humanity". I certainly fell short of " ... looking at which agents walk away with huge heaps of money and then working out how to do it systematically ... ". I would RATHER look at which agents get their species through their singularity alive. Then, and only then, can we look at something grander than survival. I don't grok in fullness "reflective consistency", but from extinction we won't be doing a lot of reflecting on what went wrong.

IMO.

Now, back to one-shot PD and "going first". For some values of κ and some external payoff matrices (not this one), the resulting IPM is not strictly dominated, and having knowledge of PB's commitment actually determines whether 'cooperate' or 'defect' produces a better world in PA's internal not-quite-so-selfish world-view. Is that a disadvantage? (That's a serious, non-rhetorical question. I'm a neophyte and I may not see some things in the depths where Eliezer & Wei think.)

Now let's look at that game of chicken. Was "throw out the steering wheel" in the definition of the thought experiment? If not, that player just changed the universe-under-consideration, which is a fairly impressive effect in an AEF, not a CFSE.

If re-engineering was included, then Driver A may complete his wheel-throwing (while in motion!) only to look up and see Driver B's steering gear on a ballistic trajectory. Each will have a few moments to reflect on "always get away with it."

If Driver A successfully defenestrates first, is Driver B at a disadvantage? Among humans, the game may be determined more by autonomic systems than by conscious computation, and B now knows that A won't be flinching away. However, B now has information and choices. One that occurs to me is to stop the car and get out. "Your move, A." A truly intelligent player (in which category I do not, alas, qualify) would think up better, or funnier, choices.

Hmmm... to even play Chicken you have to either be irrational or have a damned strange IPM. We should establish that before proceeding further.

I challenge anyone to show me a CFSE game that gives a disadvantage to the second player.

I'm not too proud to beg: I request your votes. I've got an article I'd like to post, and I need the karma.

Thanks for your time and attention.

RickJS
Saving Humanity from Homo Sapiens

08/28/2009 ~20:10 Edit: formatting ... learning formatting ... grumble ... GDSOB tab-deleter ... Fine. I'll create the HTML for tables, but this is a LOT of work for 3 simple tables ... COMMENT TOO LONG!?!? ... one last try ... now I can't quit, I'm hooked! ... NAILED that sucker! ... ~22:40 : added one more example *YAWN*