Followup to:  Newcomb's Problem and Regret of Rationality, Towards a New Decision Theory

Wei Dai asked:

"Why didn't you mention earlier that your timeless decision theory mainly had to do with logical uncertainty? It would have saved people a lot of time trying to guess what you were talking about."

...

All right, fine, here's a fast summary of the most important ingredients that go into my "timeless decision theory".  This isn't so much an explanation of TDT, as a list of starting ideas that you could use to recreate TDT given sufficient background knowledge.  It seems to me that this sort of thing really takes a mini-book, but perhaps I shall be proven wrong.

The one-sentence version is:  Choose as though controlling the logical output of the abstract computation you implement, including the output of all other instantiations and simulations of that computation.

The three-sentence version is:  Factor your uncertainty over (impossible) possible worlds into a causal graph that includes nodes corresponding to the unknown outputs of known computations; condition on the known initial conditions of your decision computation to screen off factors influencing the decision-setup; compute the counterfactuals in your expected utility formula by surgery on the node representing the logical output of that computation.

There are three great besetting sins of rationalists in particular, and the third of these is underconfidence.  Michael Vassar regularly accuses me of this sin, which makes him unique among the entire population of the Earth.

But he's actually quite right to worry, and I worry too, and any adept rationalist will probably spend a fair amount of time worying about it.  When subjects know about a bias or are warned about a bias, overcorrection is not unheard of as an experimental result.  That's what makes a lot of cognitive subtasks so troublesome—you know you're biased but you're not sure how much, and you don't know if you're correcting enough—and so perhaps you ought to correct a little more, and then a little more, but is that enough?  Or have you, perhaps, far overshot?  Are you now perhaps worse off than if you hadn't tried any correction?

You contemplate the matter, feeling more and more lost, and the very task of estimation begins to feel increasingly futile...

And when it comes to the particular questions of confidence, overconfidence, and underconfidence—being interpreted now in the broader sense, not just calibrated confidence intervals—then there is a natural tendency to cast overconfidence as the sin of pride, out of that other list which never warned against the improper use of humility or the abuse of doubt.  To place yourself too high—to overreach your proper place—to think too much of yourself—to put yourself forward—to put down your fellows by implicit comparison—and the consequences of humiliation and being cast down, perhaps publicly—are these not loathesome and fearsome things?

To be too modest—seems lighter by comparison; it wouldn't be so humiliating to be called on it publicly, indeed, finding out that you're better than you imagined might come as a warm surprise; and to put yourself down, and others implicitly above, has a positive tinge of niceness about it, it's the sort of thing that Gandalf would do.

So if you have learned a thousand ways that humans fall into error and read a hundred experimental results in which anonymous subjects are humiliated of their overconfidence—heck, even if you've just read a couple of dozen—and you don't know exactly how overconfident you are—then yes, you might genuinely be in danger of nudging yourself a step too far down.

So... the longer I think about this Wiki thing, the more it seems like a really good idea - a missing piece falling into place.

Here's my proposal, which I turn over to this, the larger community that suggested the Wiki in the first place:

The Wiki should consist mainly of short concept introductions plus links to longer posts, rather than original writing.  Original writing goes in a post on Less Wrong, which may get voted up and down, or commented on; and this post should reference previous work by linking to the Wiki rather than other posts, to the extent that the concepts referred to can be given short summaries.  The intent is to set up a resonance that bounces back and forth between the Wiki (short concept summaries that can be read standalone, and links to more info for in-depth exploration) and the posts (which make the actual arguments and do the actual analyses).

My role model here is TV Tropes, which manages to be, shall we say, really explorable, because of the resonance between the tropes, and the shows/events in which those tropes occur, and the other tropes that occur in those shows/events.  And furthermore, you know that the trope explanation itself will be a short bite of joy, and that reading the further references is optional.