Overview for Eliezer_Yudkowsky - Less Wrong

Eliezer_Yudkowsky on Giving Now Currently Seems to Beat Giving Later

2013-06-19T22:58:31.314591+00:00

Isn't this true only if there are sharply diminishing returns on recruitment?

Eliezer_Yudkowsky on Newcomb's Problem and Regret of Rationality

2013-06-19T20:29:38.476405+00:00

Also: You can't simultaneously claim that any rational being ought to two-box, this being the obvious and overdetermined answer, and also claim that it's impossible for anyone to figure out that you're going to two-box.

Eliezer_Yudkowsky on X-Risk Roll Call

2013-06-19T17:46:10.639161+00:00

Mihaly Barasz is an IMO gold medalist perfect scorer. From what I've seen personally, I'd guess that Paul Christiano is better than him at math. I forget what Marcello's prodigy points were in but I think it was some sort of computing olympiad. All should have some sort of verified performance feat far in excess of the listed educational attainment.

These days I'd describe myself as a decision theorist, with a strong interest in human rationality. Boasting of my mathematical talent in that company seems inappropriate; I don't have comparable prodigy markers (well, some very early ones of similar statistical rareness, but that was at easier problems at a younger age, and I was not properly developed as a pure math prodigy since then). I've often played a key role in figuring out which math to invent, but have relatively less comparative advantage at proving things within a given system once invented, unless the key happens to be checking laws against a concrete example which I seem to do earlier than most mathematicians. What I really do doesn't seem to have very much of a name, and can't realistically be described in a document like this one.

Anna Salamon has an Erdos number of 2.

Eliezer_Yudkowsky on Why do theists, undergrads, and Less Wrongers favor one-boxing on Newcomb?

2013-06-19T17:12:59.820231+00:00

Timeless decision theory. UDT = Updateless decision theory.

Eliezer_Yudkowsky on Why do theists, undergrads, and Less Wrongers favor one-boxing on Newcomb?

2013-06-19T16:28:33.906440+00:00

TDT's reply to this is a bit more specific.

Informally: Since Omega represents a setup which rewards agents who make a certain decision X, and reality doesn't care why or by what exact algorithm you arrive at X so long as you arrive at X, the problem is fair. Unfair would be "We'll examine your source code and punish you iff you're a CDT agent, but we won't punish another agent who two-boxes as the output of a different algorithm even though your two algorithms had the same output." The problem should not care whether you arrive at your decisions by maximizing expected utility or by picking the first option in English alphabetical order, so long as you arrive at the same decision either way.

More formally: TDT corresponds to maximizing on the class of problems whose payoff is determined by 'the sort of decision you make in the world that you actually encounter, having the algorithm that you do'. CDT corresponds to maximizing over a fair problem class consisting of scenarios whose payoff is determined only by your physical act, and would be a good strategy in the real world if no other agent ever had an algorithm similar to yours (you must be the only CDT-agent in the universe, so that your algorithm only acts at one physical point) and where no other agent could gain any info about your algorithm except by observing your controllable physical acts (tallness being correlated with intelligence is not allowed). UDT allows for maximizing over classes of scenarios where your payoff can depend on actions you would have taken in universes you could have encountered but didn't, i.e., the Counterfactual Mugging. (Parfit's Hitchhiker is outside TDT's problem class, and in UDT, because the car-driver asks "What will this hitchhiker do if I take them to town? so that a dishonorable hitchhiker who is left in the desert is getting a payoff which depends on what they would have done in a situation they did not actually encounter. Likewise the transparent Newcomb's Box. We can clearly see how to maximize on the problem but it's in UDT's class of 'fair' scenarios, not TDT's class.)

If the scenario handed to the TDT algorithm is that only one copy of your algorithm exists within the scenario, acting at one physical point, and no other agent in the scenario has any knowledge of your algorithm apart from acts you can maximize over, then TDT reduces to CDT and outputs the same action as CDT, which is implied by CDT maximizing over its problem class and TDT's class of 'fair' problems strictly including all CDT-fair problems.

If Omega rewards having particular algorithms independently of their outputs, by examining the source code without running it, the only way to maximize is to have the most rewarded algorithm regardless of its output. But this is uninteresting.

If a setup rewards some algorithms more than others because of their different outputs, this is just life. You might as well claim that a cliff punishes people who rationally choose to jump off it.

This situation is interestingly blurred in modal combat where an algorithm may perhaps do better than another because its properties were more transparent (more provable) to another algorithm examining it. Of this I can only say that if, in real life, we end up with AIs examining each other's source code and trying to prove things about each other, calling this 'unfair' is uninteresting. Reality is always the most important domain to maximize over.

Eliezer_Yudkowsky on Why do theists, undergrads, and Less Wrongers favor one-boxing on Newcomb?

2013-06-19T16:20:14.164265+00:00

What if Box B contains $1,500 instead of $1,000,000 but Omega has still been right 999 times out of 1000?

Eliezer_Yudkowsky on Why do theists, undergrads, and Less Wrongers favor one-boxing on Newcomb?

2013-06-19T16:18:21.298417+00:00

One-boxing is normal and does not call out for an explanation. :)

Eliezer_Yudkowsky on Why do theists, undergrads, and Less Wrongers favor one-boxing on Newcomb?

2013-06-19T03:45:16.082661+00:00

The most charitable interpretation would just be that there happened to be a convincing technical theory which said you should two-box, because it took an even more technical theory to explain why you should one-box and this was not constructed, along with the rest of the edifice to explain what one-boxing means in terms of epistemic models, concepts of instrumental rationality, the relation to traditional philosophy's 'free will problem', etcetera. In other words, they simply bad-lucked onto an edifice of persuasive, technical, but ultimately incorrect argument.

We could guess other motives for people to two-box, like memetic pressure for partial counterintuitiveness, but why go to that effort now? Better TDT writeups are on the way, and eventually we'll get to see what the field says about the improved TDT writeups. If it's important to know what other hidden motives might be at work, we'll have a better idea after we negate the usually-stated motive of, "The only good technical theory we have says you should two-box." Perhaps the field will experience a large conversion once presented with a good enough writeup and then we'll know there weren't any other significant motives.

Eliezer_Yudkowsky on Why do theists, undergrads, and Less Wrongers favor one-boxing on Newcomb?

2013-06-19T03:40:19.720459+00:00

It is unlikely that a mere human without computational assistance could simulate someone in sufficient detail to reliably make one boxing the best option.

(Plausible, but then the mere human should have a low accuracy / discrimination rate. You can't have this and a high accuracy rate at the same time. Also in practice there are plenty of one-boxers out there.)

Eliezer_Yudkowsky on Do Earths with slower economic growth have a better chance at FAI?

2013-06-19T09:37:56.968597+10:00

Yep.